Farklı Sınıflandırma Algoritmaları ve Metin Temsil Yöntemlerinin Duygu Analizinde Performans Karşılaştırılması

Abstract

Son yıllarda internete erişim imkanlarının artması ve kullanıcılardaki akıllı telefon kullanımının yaygınlaşması sebebiyle sosyal medya olarak adlandırılan ve insanların çeşitli konulardaki fikirlerini paylaştığı servisler çok yaygın olarak kullanılmaktadır. Sosyal medya verilerinin analiz edilmesiyle insanların farklı konulardaki duygularına dair anlamlı çıkarımlarda bulunulması anlamına gelen ve temelde bir sınıflandırma işlemi olan Duygu Analizi çalışmaları son yıllarda öne çıkan çalışma alanlarından biridir. Bu çalışmada, Python programlama dili içindeki kütüphaneler kullanılarak Naive Bayes (NB), Logistic Regression (LR), Support Vector Machine (SVM), Decision Tree (DT), Random Forest (RF) ve Artificial Neural Network (ANN) gibi 6 adet sınıflandırma algoritmasının Duygu Analizi kapsamında, performans karşılaştırması yapılmıştır. Veri seti olarak, açık kaynaklı, IMDB sitesinde yer alan etiketli kullanıcı yorumları kullanılmıştır. Doğal Dil İşleme yöntemleri kullanılarak temizlenen veri setinin sayısal olarak temsil edilebilmesi için Bag of Words (BoW), TF-IDF, FastText ve Word2Vec metin temsil yöntemleri kullanılmıştır. Veri setinin eğitimi ve test edilmesi aşamasında k=5 olacak şekilde k-fold cross validation yöntemi kullanılmıştır. 6 farklı sınıflandırma yöntemi için elde edilen sonuçlar accuracy, precision, recall ve f1 score hesaplanarak ayrıntılı bir karşılaştırma yapılmış ve sonuçlar kaydedilmiştir. En yüksek accuracy değerleri olarak LR ve SVM sırasıyla BOW’da %86, TF-IDF’te %87, word2Vec’de %87 ve FastText’te %83 seviyelerinde benzer sonuçlar vermiştir.

Keywords

• Doğal Dil İşleme
• Duygu Analizi
• Makine Öğrenmesi
• Metin Temsil
• Sınıflandırma
• Veri Madenciliği

Performance Comparison of Different Classification Algorithms and Text Representation Methods in Sentiment Analysis

Abstract

Due to the increase in internet access opportunities and the widespread use of smartphones in recent years, services called social media where people share their opinions on various issues are widely used. Sentiment Analysis studies, which means making meaningful inferences about people's emotions on different subjects by analyzing social media data, and which is basically a classification process, is one of the prominent fields of study in recent years. In this study, 6 classification methods such as Naive Bayes (NB), Logistic Regression (LR), Support Vector Machine (SVM), Decision Tree (DT), Random Forest (RF) and Artificial Neural Network (ANN) were used by using libraries in Python programming language. Within the scope of Sentiment Analysis of the algorithm, performance comparison was made. As the dataset, open source, labeled user comments on the IMDB site were used. Bag of Words (BoW), TF-IDF, FastText and Word2Vec text representation methods were used to represent the data set that was cleaned using Natural Language Processing methods. During the training and testing of the data set, the k-fold cross validation method was used, with k=5. The results obtained for 6 different classification methods were calculated by calculating accuracy, precision, recall and f1 score, and a detailed comparison was made and the results were recorded. As the highest accuracy values, LR and SVM gave similar results at 86% in BOW, 87% in TF-IDF, 87% in word2Vec and 83% in FastText, respectively.

Keywords

• Natural Language Processing
• Sentiment Analysis
• Machine Learning
• Text Representation
• Classification
• Data Mining

References

1. [1] B. Agarwal, N. Mittal, P. Bansal, and S. Garg, “Sentiment analysis using common-sense and context information,” Computational Intelligence and Neuroscience, vol. 2015, pp. 1–9, 2015.
2. [2] N. Mishra and C. K. Jha, “Classification of opinion mining techniques,” International Journal of Computer Applications, vol. 56, no. 13, pp. 1–6, 2012.
3. [3] B. Bansal ve S. Srivastava, “Sentiment classification of online consumer reviews using word vector representations”, Procedia Computer Science, vol. 132, pp. 1147–1153, 2018.
4. [4] S. Symeonidis, D. Effrosynidis, ve A. Arampatzis, “A comparative evaluation of pre-processing techniques and their interactions for twitter sentiment analysis”, Expert Systems with Applications, vol. 110, pp. 298–310, 2018.
5. [5] B. Haryanto, Y. Ruldeviyani, F. Rohman, T. N. Julius Dimas, R. Magdalena, ve F. Muhamad Yasil, “Facebook analysis of community sentiment on 2019 Indonesian presidential candidates from Facebook opinion data”, Procedia Computer Science, vol. 161, pp. 715–722, 2019.
6. [6] E. D’Andrea, P. Ducange, A. Bechini, A. Renda, ve F. Marcelloni, “Monitoring the public opinion about the vaccination topic from tweets analysis”, Expert Systems with Applications, vol. 116, pp. 209–226, 2019.
7. [7] A. Alsaeedi and M. Z. Khan, “A study on sentiment analysis techniques of Twitter data,” International Journal of Advanced Computer Science and Applications, vol. 10, no. 2, pp. 361–374, 2019.
8. [8] J. Khairnar and M. Kinikar, “Machine learning algorithms for opinion mining and sentiment classification,” International Journal of Scientific and Research Publications, vol. 3, no. 6, pp. 1–6, 2013.

9. [9] A. Tyagi and N. Sharma, “Sentiment Analysis using logistic regression and effective word score heuristic,” International Journal of Engineering and Technology (UAE), vol. 7, no. 2, pp. 20–23, 2018.
10. [10] H. Kaur, V. Mangat, and Nidhi, “A survey of sentiment analysis techniques,” Proceedings of the International Conference on IoT in Social, Mobile, Analytics and Cloud, I-SMAC 2017, 2017, pp. 921–925.
11. [11] M. M and S. Mehla, “Sentiment analysis of movie reviews using machine learning classifiers,” International Journal of Computer Applications, vol. 182, no. 50, pp. 25–28, 2019.
12. [12] F. Hemmatian and M. K. Sohrabi, “A survey on classification techniques for opinion mining and sentiment analysis,” Artificial Intelligence Review, vol. 52, no. 3, pp. 1495–1545, 2019.
13. [13] A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts, “Learning word vectors for sentiment analysis,” ACL-HLT 2011 - Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 142–150, 2011.