The Effect of Word Representation Methods on Aspect-Based Sentiment Analysis

Year 2022, , 443 - 452, 31.10.2022

Mesut Polatgil , Murat Fatih Tuna , Oğuz Kaynar

https://doi.org/10.17671/gazibtd.1114901

Abstract

Unlike classical sentiment analysis methods, Aspect-Based Sentiment Analysis (ABSA) can demonstrate a more successful performance in evaluating complex online consumer feedbacks including more than one category. As a matter of fact, consumer feedbacks on a platform can be referred to more than one aspect regarding a product, and standard sentiment analysis method is insufficient to analyse these comments. When the developments in the literature are reviewed, it is understood that HDTA studies are very popular among other studies focusing on sentiment analysis. In the SemEval ABSA-2016 competition, datasets were published in 8 different languages for HTDA and the teams competed for sentiment analysis. There are different subtasks in the competition, determining sub-categories such as aspect term, category and sentiment class. One of these subtasks is to determine the aspect term. HTDA studies for Turkish language are quite limited. There are studies using different languages and different word representation methods. There is no study examining the effect of word representation methods for the Turkish data set of SemEval Absa 2016 competition. This study was carried out to examine the success of different word representation methods in identifying aspect terms in customer comments. This study was carried out with the aim of examining the success of different word representation methods in identifying target terms in customer comments. Word2Vec, Glove and Fasttext word representation methods were examined within the scope of the analysis and it was seen that the method that could detect the aspect term most successfully was the Fasttext word representation method. The highest classification success for Turkish dataset in the literature with a success rate of 77% in terms of the F-1 score was also achieved in the study.

Keywords

aspect based sentiment analysis, aspect term, word representation methods, consumer feedbacks, Turkish texts, Word2Vec, Glove, Fasttext

Kelime Temsil Yöntemlerinin Hedef Tabanlı Duygu Analizine Etkisi

Abstract

Klasik duygu analizi yöntemlerinden farklı olarak hedef tabanlı duygu analizi (HTDA), birden fazla kategorinin olduğu karmaşık yapıdaki çevrimiçi tüketici geribildirimlerini değerlendirmede daha başarılı bir performans ortaya koyabilmektedir. Nitekim bir platformda yer alan tüketici geri bildirimleri bir ürüne ilişkin birden farklı hedefe atfedilebilmektedir ve standart duygu analizleri bu geribildirimleri analiz etmede yetersiz kalmaktadır. Literatürdeki gelişmeler gözden geçirildiğinde, HDTA çalışmalarının, duygu analizine odaklanan diğer çalışmalar içinde oldukça popüler olduğu anlaşılmaktadır. SemEval ABSA-2016 yarışmasında, HTDA için 8 farklı dilde veri setleri yayınlanmış ve ekipler duygu analizi için yarışmışlardır. Yarışmada hedef terim, kategori ve duygu sınıfı tespit etmek gibi farklı alt görevler bulunmaktadır. Bu alt görevlerin içindekilerden biri, hedef terimin tespit edilmesidir. Türkçe dili için HTDA çalışmaları oldukça sınırlıdır. Farklı diller ve farklı kelime temsil yöntemleri kullanan çalışmalar vardır. SemEval Absa 2016 yarışması Türkçe veri seti için kelime temsil yöntemlerinin etkisini inceleyen çalışma bulunmamaktadır. Bu çalışma, müşteri yorumlarındaki hedef terimlerin tespitinde farklı kelime temsil yöntemlerinin başarısının incelenmesi amacıyla gerçekleştirilmiştir. Word2Vec, Glove ve Fasttext kelime temsil yöntemleri analiz kapsamında incelenmiş ve hedef terimi en başarılı tespit edebilen yöntemin Fasttext kelime temsil yöntemi olduğu görülmüştür. Çalışmada ayrıca F-1 sınıflandırma ölçütü açısından %77 başarı oranı ile Türkçe veri seti için literatürdeki en yüksek sınıflandırma başarısı elde edilmiştir.

Keywords

hedef tabanlı duygu analizi, hedef terim, kelime temsil yöntemleri, müşteri geribildirimleri, Türkçe metinler, Word2Vec, Glove, Fasttext

References

• F. S. Çeti̇n, G. Eryi̇ği̇t, “Türkçe Hedef Tabanlı Duygu Analizi İçin Alt Görevlerin İncelenmesi – Hedef Terim, Hedef Kategori ve Duygu Sınıfı Belirleme”, Bilişim Teknolojileri Dergisi, 11(1), 43–56, 2018.
• O. Kaynar, Y. Görmez, M. Yıldız, A. Albayrak, “Makine öğrenmesi yöntemleri ile Duygu Analizi”, International Artificial Intelligence and Data Processing Symposium (IDAP’16), Malatya, 234–241, September 17-18, 2016.
• Z. Khan, T. Vorley, “Big data Text Analytics: An Enabler of Knowledge Management”, Journal of Knowledge Management, 21(1), 18–34, 2017.
• G. Zaltman, L. H. Zaltman, Marketing Metaphoria: What Deep Metaphors Reveal About the Minds of Consumers (1st edition), Harvard Business Review Press, Boston, 2008.
• D. Westerman, P. R. Spence, B. Van Der Heide, “Social Media as Information Source: Recency of Updates and Credibility of Information”, Journal of Computer-Mediated Communication, 19(2), 171–183, 2014.
• V. Ahuja, Y. Medury, “Corporate Blogs as e-CRM Tools – Building Consumer Engagement through Content Management”, Journal of Database Marketing & Customer Strategy Management, 17(2), 91–105, 2010.
• P.-Y. Chen, S. Wu, J. Yoon, “The Impact of Online Recommendations and Consumer Feedback on Sales”, in ICIS 2004 Proceedings, 58, 2004.
• A. A. Thorp, G. N. Healy, E. Winkler, B. K. Clark, P. A. Gardliner, N. Owen, D. W. Dunstan, “Prolonged Sedentary Time and Physical Activity in Workplace and Non-Work Contexts: A Cross-Sectional Study of Office, Customer Service and Call Centre Employees”, International Journal of Behavioral Nutrition and Physical Activity, 9(128), 1–9, 2012.
• J. Cotte, S. Ratneshwar, D. G. Mick, “The Times of Their Lives: Phenomenological and Metaphorical Characteristics of Consumer Timestyles”, Journal of Consumer Research, 31(2), 333–345, 2004.
• T. Y. Lee, E. T. Bradlow, “Automated Marketing Research Using Online Customer Reviews”, Journal of Marketing Research, 48(5), 881–894, 2011.
• S. A. Bhat, M. A. Darzi, “Service, People and Customer Orientation: A Capability View to CRM and Sustainable Competitive Advantage”, Vision, 22(2), 163–173, 2018.
• A. Stelzer, F. Englert, S. Hörold, C. Mayas, “Improving Service Quality in Public Transportation Systems Using Automated Customer Feedback”, Transportation Research Part E: Logistics and Transportation Review, 89, 259–271, 2016.
• V. Barger, J. W. Peltier, D. E. Schultz, “Social Media and Consumer Engagement: A Review and Research Agenda”, Journal of Research in Interactive Marketing, 10(4). 268–287, 2016.
• M. F. Tuna, O. Kaynar, M. Ş. Akdoğan, “Otellere İlişkin Çevrimiçi Geribildirimlerin Makine Öğrenmesi Yöntemleriyle Duygu Analizi”, İşletme Araştırmaları Dergisi, 13(3), 2232–2241, 2021.
• M. U. Salur, İ. Aydın, M. Jamous, “An ensemble approach for aspect term extraction in Turkish texts”, Pamukkale University Journal of Engineering Sciences, Ahead of Print, 2022.
• Ł. Augustyniak, T. Kajdanowicz, P. Kazienko, “Comprehensive Analysis of Aspect Term Extraction Methods using Various Text Embeddings”, arXiv:1909.04917 [cs], 2020.
• S. G. Barbounaki, K. Gourounti, A. Sarantaki, “Advances of Sentiment Analysis Applications in Obstetrics/Gynecology and Midwifery”, Materia Socio Medica, 33(3), 225–230, 2021.
• M. Al-Smadi, O. Qawasmeh, M. Al-Ayyoub, Y. Jararweh, B. Gupta, “Deep Recurrent Neural Network vs. Support Vector Machine for Aspect-Based Sentiment Analysis of Arabic Hotels’ Reviews”, Journal of Computational Science, 27, 386–393, 2018.
• B. Liu, “Sentiment Analysis and Opinion Mining”, Synthesis Lectures on Human Language Technologies, 5(1), 1–167, 2012.
• Y. Wang, M. Huang, X. Zhu, L. Zhao, “Attention-Based LSTM for Aspect-Level Sentiment Classification”, Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin (Texas):Association for Computational Linguistics, 606–615.
• W. Wang, S. J. Pan, D. Dahlmeier, X. Xiao, “Recursive Neural Conditional Random Fields for Aspect-based Sentiment Analysis”, Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin (Texas):Association for Computational Linguistics, 616–626.
• B. Pang, L. Lee, S. Vaithyanathan, “Thumbs up? Sentiment Classification using Machine Learning Techniques”, Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002), Pennsylvania(Philadelphia): Association for Computational Linguistics, 79–86, 2002.
• T. T. Thet, J.-C. Na, C. S. G. Khoo, “Aspect-Based Sentiment Analysis of Movie Reviews on Discussion Boards”, Journal of Information Science, 36(6), 823–848, 2010.
• S. Brody, N. Elhadad, “An Unsupervised Aspect-Sentiment Model for Online Reviews”, Program No: HLT-NAACL, 2010. URL: https://openreview.net/forum?id=HJ-8d7-_bH, 04.04.2022.
• I. Titov, R. McDonald, “A Joint Model of Text and Aspect Ratings for Sentiment Summarization”, Proceedings of ACL-08: HLT, Columbus(Ohio): Association for Computational Linguistics, 308–316, 2008.
• J. Zhu, H. Wang, B. K. Tsou, M. Zhu, “Multi-Aspect Opinion Polling from Textual Reviews”, Proceedings of the 18th ACM Conference on Information and Knowledge Management, New York: ACM Digital Library, 1799–1802, 2009.
• J. Wang, B. Xu, Y. Zu, “Deep Learning for Aspect-Based Sentiment Analysis”, International Conference on Machine Learning and Intelligent Systems Engineering (MLISE), Chongqing: IEEE, 267–271, 2021.
• D. Anand, D. Naorem, “Semi-Supervised Aspect Based Sentiment Analysis for Movies Using Review Filtering”, Procedia Computer Science, 84, 86–93, 2016.
• T. Tran, H. Ba, V.-N. Huynh, “Measuring Hotel Review Sentiment: An Aspect-Based Sentiment Analysis Approach”, In Integrated Uncertainty in Knowledge Modelling and Decision Making, Cham, 393–405, 2019.
• D. Ekawati, M. L. Khodra, “Aspect-Based Sentiment Analysis for Indonesian Restaurant Reviews”, International Conference on Advanced Informatics, Concepts, Theory, and Applications (ICAICTA-17), Kuta: IEEE, 1–6, 2017.
• P. Blinov, E. Kotelnikov, “Blinov: Distributed Representations of Words for Aspect-Based Sentiment Analysis at SemEval 2014”, Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), Dublin:Association for Computational Linguistics, 140–144, 2014.
• W. Wang, G. Tan, H. Wang, “Cross-Domain Comparison of Algorithm Performance in Extracting Aspect-Based Opinions from Chinese Online Reviews”, International Journal of Machine Learning & Cybernetics, 8(3), 1053–1070, 2017.
• M. Pontiki, D. Galanis, H. Papageorgiou, I. Androutsopoulos, S. Manandhar, M. Al-Smadi, M. Al-Ayyoub, Y. Zhao, B. Qin, O. D. Clercq, V. Hoste, M. Apidianaki, X. Tannier, N. Loukachevitich, E. Kotelnikov, N. Bel, S. M. Jiménez-Zafra, G. Eryiğit, “SemEval-2016 Task 5: Aspect Based Sentiment Analysis”, Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), San Diego (California): Association for Computational Linguistics, 19–30, 2016.
• B. Kama, M. Ozturk, P. Karagoz, I. H. Toroslu, O. Ozay, “A Web Search Enhanced Feature Extraction Method for Aspect-Based Sentiment Analysis for Turkish Informal Texts”, In Big Data Analytics and Knowledge Discovery, Cham, 225–238, 2016.
• K. Bayraktar, U. Yavanoglu, A. Ozbilen, “A Rule-Based Holistic Approach for Turkish Aspect-Based Sentiment Analysis”, IEEE International Conference on Big Data (Big Data), Los Angeles:IEEE, 2154–2158, 2019.
• B. Ozyurt, M. A. Akcayol, “A New Topic Modeling Based Approach for Aspect Extraction in Aspect Based Sentiment Analysis: SS-LDA”, Expert Systems with Applications, 168, 114231, 2021.
• M. U. Salur, İ. Aydin, “An Annotated Turkish Aspect Based Sentiment Analysis Corpus for Smart Tourism”, 2021 Innovations in Intelligent Systems and Applications Conference (ASYU), Elazığ: IEEE 1–6, 2021.
• Internet: A. Köksal, Github, https://github.com/akoksal/Turkish-Word2Vec, 15.02.2022.
• O. Çiftçi, GitHub, https://github.com/inzva/Turkish-GloVe, 15.02.2022.
• E. Grave, P. Bojanowski, P. Gupta, A. Joulin, T. Mikolov, “Learning Word Vectors for 157 Languages”, Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, European Language Resources Association (ELRA), 3483–3487, 2018.