Review Mate: A Cutting-Edge Model for Analyzing the Sentiment of Online Customer Product Reviews using ML.NET

Year 2024, Volume: 5 Issue: 2, 74 - 88

Özge Cömert , Nurcan Yücel

https://doi.org/10.55195/jscai.1595798

Abstract

E-commerce has become increasingly important in recent years due to several factors such as convenience, global reach, lower costs, personalization and uninterrupted access. In e-commerce, product reviews by customers can significantly impact purchasing behavior by providing social proof, establishing trust, aiding decision-making, improving search engine optimization, and increasing sales. Conducting an evaluation of the primary impacts of customer reviews on purchasing behavior through automated machine learning techniques has the potential to facilitate the advancement of diverse online business models. In this scope, we come with a new machine-learning model for evaluating customer sentiment based on product reviews. To this aim, a dataset consisting of 1000 positive and 1000 negative customer reviews was created by collecting publicly shared comments from online shopping websites serving in Turkey with a data collection tool developed by our research group. The model development was carried out on ML.NET, an open-source and cross-platform machine learning framework. In order to reach the most efficient model, a total of 36 machine learning models were explored for the solution of the problem within the scope of the experimental study. As a result, the model named Lbfgs Logistic Regression Binary was found to be the most efficient. The related model provided an accuracy rate of 94.76%. An API service called Review Mate has been developed to expand the potential impact of the proposed machine learning model and enable its use in different online business models. According to the findings, the proposed method outperforms the previous approach in terms of classification performance and also provides avenues for the discovery of new product ideas.

Keywords

Sentiment analysis, Machine learning, Classification, ML.NET

Project Number

-

References

• M. Guha Majumder, S. Dutta Gupta, and J. Paul, “Perceived usefulness of online customer reviews: A review mining approach using machine learning & exploratory data analysis,” J. Bus. Res., vol. 150, pp. 147–164, 2022, doi: https://doi.org/10.1016/j.jbusres.2022.06.012.
• S.-H. Chin, C. Lu, P.-T. Ho, Y.-F. Shiao, and T.-J. Wu, “Commodity anti-counterfeiting decision in e-commerce trade based on machine learning and Internet of Things,” Comput. Stand. Interfaces, vol. 76, p. 103504, 2021, doi: https://doi.org/10.1016/j.csi.2020.103504.
• F. Liu, S. Liu, and G. Jiang, “Consumers’ decision-making process in redeeming and sharing behaviors toward app-based mobile coupons in social commerce,” Int. J. Inf. Manage., vol. 67, p. 102550, 2022, doi: https://doi.org/10.1016/j.ijinfomgt.2022.102550.
• E. M. Mercha and H. Benbrahim, “Machine learning and deep learning for sentiment analysis across languages: A survey,” Neurocomputing, vol. 531, pp. 195–216, 2023, doi: https://doi.org/10.1016/j.neucom.2023.02.015.
• N. Meilatinova, “Social commerce: Factors affecting customer repurchase and word-of-mouth intentions,” Int. J. Inf. Manage., vol. 57, p. 102300, 2021, doi: https://doi.org/10.1016/j.ijinfomgt.2020.102300.
• X. Xu, R. Fan, D. Wang, Y. Wang, and Y. Wang, “The role of consumer reviews in e-commerce platform credit supervision: A signaling game model based on complex network,” Electron. Commer. Res. Appl., vol. 63, p. 101347, 2024, doi: https://doi.org/10.1016/j.elerap.2023.101347.
• D. Zhang, L. G. Pee, and L. Cui, “Artificial intelligence in E-commerce fulfillment: A case study of resource orchestration at Alibaba’s Smart Warehouse,” Int. J. Inf. Manage., vol. 57, p. 102304, 2021, doi: https://doi.org/10.1016/j.ijinfomgt.2020.102304.
• O. Araque, J. F. Sánchez-Rada, and C. A. Iglesias, “GSITK: A sentiment analysis framework for agile replication and development,” SoftwareX, vol. 17, p. 100921, 2022, doi: https://doi.org/10.1016/j.softx.2021.100921.
• J. Joung and H. Kim, “Interpretable machine learning-based approach for customer segmentation for new product development from online product reviews,” Int. J. Inf. Manage., vol. 70, p. 102641, 2023, doi: https://doi.org/10.1016/j.ijinfomgt.2023.102641.
• A. Patel, P. Oza, and S. Agrawal, “Sentiment Analysis of Customer Feedback and Reviews for Airline Services using Language Representation Model,” Procedia Comput. Sci., vol. 218, pp. 2459–2467, 2023, doi: https://doi.org/10.1016/j.procs.2023.01.221.
• P. Savci and B. Das, “Prediction of the customers’ interests using sentiment analysis in e-commerce data for comparison of Arabic, English, and Turkish languages,” J. King Saud Univ. - Comput. Inf. Sci., vol. 35, no. 3, pp. 227–237, 2023, doi: https://doi.org/10.1016/j.jksuci.2023.02.017.
• M. Nilashi, R. A. Abumalloh, S. Samad, M. Alrizq, S. Alyami, and A. Alghamdi, “Analysis of customers’ satisfaction with baby products: The moderating role of brand image,” J. Retail. Consum. Serv., vol. 73, p. 103334, 2023, doi: https://doi.org/10.1016/j.jretconser.2023.103334.
• R. Haque, N. Islam, M. Tasneem, and A. K. Das, “Multi-class sentiment classification on Bengali social media comments using machine learning,” Int. J. Cogn. Comput. Eng., vol. 4, pp. 21–35, 2023, doi: https://doi.org/10.1016/j.ijcce.2023.01.001.
• H. Karayiğit, Ç. İnan Acı, and A. Akdağlı, “Detecting abusive Instagram comments in Turkish using convolutional Neural network and machine learning methods,” Expert Syst. Appl., vol. 174, p. 114802, 2021, doi: https://doi.org/10.1016/j.eswa.2021.114802.
• M. Demircan, A. Seller, F. Abut, and M. F. Akay, “Developing Turkish sentiment analysis models using machine learning and e-commerce data,” Int. J. Cogn. Comput. Eng., vol. 2, pp. 202–207, 2021, doi: https://doi.org/10.1016/j.ijcce.2021.11.003.
• H. Zhao, Z. Liu, X. Yao, and Q. Yang, “A machine learning-based sentiment analysis of online product reviews with a novel term weighting and feature selection approach,” Inf. Process. Manag., vol. 58, no. 5, p. 102656, 2021, doi: https://doi.org/10.1016/j.ipm.2021.102656.
• G. Cosma and G. Acampora, “A computational intelligence approach to efficiently predicting review ratings in e-commerce,” Appl. Soft Comput., vol. 44, pp. 153–162, 2016, doi: https://doi.org/10.1016/j.asoc.2016.02.024.
• N. Yücel and Ö. Cömert, “Müşteri Duyarlılığını Keşfetmek İçin Yapay Zeka Destekli Analiz ile Çevrimiçi Ürün İncelemelerinden Anlamlı Bilgiler Elde Etme,” Fırat Üniversitesi Mühendislik Bilim. Derg., vol. 35, no. 2, pp. 679–690, 2023, doi: 10.35234/fumbd.1305932.
• M. Kilinc and C. Aydin, “Feature selection for Turkish Crowdfunding projects with using filtering and wrapping methods,” Electron. Commer. Res. Appl., vol. 62, p. 101340, 2023, doi: https://doi.org/10.1016/j.elerap.2023.101340.
• A. R. Webb, Statistical pattern recognition. John Wiley & Sons, 2003.
• I. Guyon and A. Elisseeff, “An Introduction to Variable and Feature Selection,” J. Mach. Learn. Res., vol. 3, no. 3, pp. 1157–1182, 2003, doi: 10.1016/j.aca.2011.07.027.
• Z. Cömert and A. F. Kocamaz, “A Study of Artificial Neural Network Training Algorithms for Classification of Cardiotocography Signals,” Bitlis Eren Univ. J. Sci. Technol., vol. 7, no. 2, pp. 93–103, 2017, doi: 10.17678/beuscitech.338085.
• Microsoft, “ML.NET An open source and cross-platform machine learning framework,” 2024. https://dotnet.microsoft.com/en-us/apps/machinelearning-ai/ml-dotnet
• Z. Cömert, A. F. Kocamaz, and V. Subha, “Prognostic model based on image-based time-frequency features and genetic algorithm for fetal hypoxia assessment,” Comput. Biol. Med., vol. 99, pp. 85–97, Aug. 2018, doi: 10.1016/J.COMPBIOMED.2018.06.003.
• Z. Cömert and Ö. Cömert, “A Study of Technologies Used in Learning Management Systems and Evaluation of New Trend Algorithms,” Bitlis Eren Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, vol. 7. Bitlis Eren Üniversitesi, pp. 286–297, 2018.
• A. Sills, M. Ahmed, Dotthatcom.com, F. Boumphrey, and J. Ortiz, “Chapter 6 - Web Development Using XML and ASP.NET,” A. Sills, M. Ahmed, Dotthatcom.com, F. Boumphrey, and J. B. T.-X. M. L. N. E. T. D. G. Ortiz, Eds., Rockland: Syngress, 2002, pp. 231–282. doi: https://doi.org/10.1016/B978-192899447-3/50009-3.
• P. Rasappan, M. Premkumar, G. Sinha, and K. Chandrasekaran, “Transforming sentiment analysis for e-commerce product reviews: Hybrid deep learning model with an innovative term weighting and feature selection,” Inf. Process. Manag., vol. 61, no. 3, p. 103654, 2024, doi: https://doi.org/10.1016/j.ipm.2024.103654.
• A. Yucel, A. Dag, A. Oztekin, and M. Carpenter, “A novel text analytic methodology for classification of product and service reviews,” J. Bus. Res., vol. 151, pp. 287–297, 2022, doi: https://doi.org/10.1016/j.jbusres.2022.06.062.
• P. Vijayaragavan, R. Ponnusamy, and M. Aramudhan, “An optimal support vector machine based classification model for sentimental analysis of online product reviews,” Futur. Gener. Comput. Syst., vol. 111, pp. 234–240, 2020, doi: https://doi.org/10.1016/j.future.2020.04.046.
• Y. Huang, X. Bai, Q. Liu, H. Peng, Q. Yang, and J. Wang, “Sentence-level sentiment classification based on multi-attention bidirectional gated spiking neural P systems,” Appl. Soft Comput., vol. 152, p. 111231, 2024, doi: https://doi.org/10.1016/j.asoc.2024.111231.
• L. Gui, Y. Zhou, R. Xu, Y. He, and Q. Lu, “Learning representations from heterogeneous network for sentiment classification of product reviews,” Knowledge-Based Syst., vol. 124, pp. 34–45, 2017, doi: https://doi.org/10.1016/j.knosys.2017.02.030.
• M. Aydoğan and A. Karci, “Improving the accuracy using pre-trained word embeddings on deep neural networks for Turkish text classification,” Phys. A Stat. Mech. its Appl., vol. 541, p. 123288, 2020, doi: https://doi.org/10.1016/j.physa.2019.123288.