Social Media User Opinion Analysis Using Deep Learning and Machine Learning Methods: A Case Study on Airlines

Abstract

ABsTRACT. The rapid surge in social media usage has augmented the significance and value of data available on these platforms. As a result, analyzing community sentiment and opinions related to various topics and events using social media data has become increasingly crucial. However, the sheer volume of data produced on social media platforms surpasses human processing capabilities. Consequently, artificial intelligence-based models became frequently employed in social media analysis. In this study, deep learning (DL) and machine learning (ML) methods are applied to assess user opinions regarding airlines, and the effectiveness of these methods in social media analysis is comparatively discussed based on the performance results obtained. Due to the imbalanced nature of the dataset, synthetic data is produced using the Synthetic Minority Over-Sampling Technique (SMOTE) to enhance model performance. Before the SMOTE process, the dataset containing 14640 data points expanded to 27534 data points after the SMOTE process. The experimental results demonstrate that Support Vector Machines (SVM) achieved the highest performance among all methods with accuracy, precision, recall, and F-score values of 0.79 in the pre-SMOTE (imbalanced dataset). In contrast, Random Forest (RF) obtained the best performance among all methods, with accuracy, precision, recall, and F-score values of 0.88 in the post-SMOTE (balanced data set). Moreover, experimental findings demonstrate that SMOTE led to performance improvements in ML and DL models, ranging from a minimum of 3% to a maximum of 24% increase in F-Score metric.

Keywords

• Social media
• sentiment analysis
• deep learning
• machine learning

References

1. Aljedaani, W. et al., Sentiment analysis on Twitter data integrating TextBlob and deep learning models: The case of US airline industry, Knowledge Based Systems, 255(2022), 109780.
2. Al-Qahtani, R., Bint Abdulrahman, P.N., Predict sentiment of airline tweets using ML models, EasyChair, 5228(2021).
3. Atacak, İ., Şencan, Ö .A., Mamdani ve Sugeno tip bulanık çıkarım sistemleri ile sosyal medya haber popülerliğinin tahmini, Uluslararası Muhendislik Araştırma ve Geliştirme Dergisi, 14(3)(2022), 303–320.
4. Bibi, M. et al., A novel unsupervised ensemble framework using concept-based linguistic methods and machine learning for twitter sentiment analysis, Pattern Recognition Letters, 158(2022), 80–86.
5. Chaw, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P., SMOTE: Synthetic Minority Over-sampling Technique, Journal of Artificial Intelligence Research, 16(2002), 321–357.
6. Figure Eight, Twitter US Airline Sentiment, (2023), Accessed: Oct 25, 2023, https://www.kaggle.com/datasets/crowdflower/twitter-airlinesentiment.
7. Garciai, K., Berton, L., Topic detection and sentiment analysis in Twitter content related to COVID-19 from Brazil and the USA, Applied Soft Computing 101(2021), 107057.
8. Greer, C.R., Lei, D., Collaborative innovation with customers: A review of the literature and suggestions for future research, International Journal of Management Reviews, 14(1) (2012), 63–84.

9. Guan, H., Zhao, L., Dong, X., Chen, C., Extended natural neighborhood for SMOTE and its variants in imbalanced classification, Engineering Applications of Artificial Intelligence, 124(2023), 106570.
10. Guo, W., Wang, G., Wang, C., Wang, Y., Distribution network topology identification based on gradient boosting decision tree and attribute weighted naive Bayes, Energy Reports, 9(2023), 727–736.
11. Hasib, K. Md., Habib, Md. A., Towhid, N.A., Showrov, Md. I.H., A novel deep learning based sentiment analysis of Twitter data for US airline service, International Conference on Information and Communication Technology for Sustainable Development (ICICT4SD), (2021), 450–455.
12. Heba, H., Aljarah, I., Al-Shboul, B., Online social media-based sentiment analysis for US airline companies, Proceedings of the New Trends in Information Technology (NTIT-2017), (2017), 176–181.
13. Khairnar, J., Kinikar, M., Machine learning algorithms for opinion mining and sentiment classification, Citeseer, 3(6)(2013), Accessed: May 25, 2023, https://citeseerx.ist.psu.edu/document repid=rep1&type=pdf&doi=269d91e79049092bdf0651241d0d66830aa9fafc.
14. Kong, J.,Wang, J., Zhang, X., Hierarchical BERT with an adaptive fine-tuning strategy for document classification, Knowledge Based Systems, 238(2022), 107872.
15. Li, W.C., Jiang, L., Learning from crowds with robust logistic regression, Information Sciences, 639(2023), 119010.
16. Liu, Y., Bi, J.-W., Fan, Z.-P., Multi-class sentiment classification: The experimental comparisons of feature selection and machine learning algorithms, Expert Systems with Applications, 80(2017), 323–339.
17. Liu, F., Zheng, J., Zheng, L., Chen, C., Combining attention-based bidirectional gated recurrent neural network and two-dimensional convolutional neural network for document-level sentiment classification, Neurocomputing, 371(2020), 39–50.
18. Mercha, E.M., Benbrahim, H., Machine learning and deep learning for sentiment analysis across languages: A survey, Neurocomputing, 531(2023), 195–216.
19. Pang, B., Lee, L., Vaithyanathan, S., Thumbs up? Sentiment classification using machine learning techniques, Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing, (2002), 79–86.
20. Pavitha, N. et al., Movie recommendation and sentiment analysis using machine learning, Global Transitions Proceedings, 3(1)(2022), 279– 284.
21. Qiu, J., et al., GCC: Graph contrastive coding for graph neural network pre-training, Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, New York, NY, USA: ACM, (2020), 1150–1160.
22. Rodriguez-Ibanez, M., Casanez-Ventura, A., Castej´on-Mateos, F., Cuenca-Jim´enez, P.-M., A review on sentiment analysis from social media platforms, Expert Systems with Applications, 223(2023), 119862.
23. Statista, Number of worldwide social network users 2027, (2023), https://www.statista.com/statistics/278414/number-of-worldwide-socialnetwork- users/, accessed Mar. 10, 2023.
24. Svetnik, V., Liaw, A., Ton, C., Christopher Culberson, J., Sheridan, R.P., Feuston, B.P., Random forest: A classification and regression tool for compound classification and QSAR modeling, Journal of Chemical Information and Modeling, 43(6)(2003), 1947–1958.
25. Şencan, Ö .A., Atacak, İ., Doğru, İ.A., Systematic literature review of detecting topics and communities in social networks, Bilişim Teknolojileri Dergisi, 15(3)(2022), 317–329.
26. Vapnik, V.N., Lerner, A.Y., Recognition of patterns with help of generalized portraits, (1963).
27. Wen, S. et al., Memristive LSTM network for sentiment analysis, IEEE Transactions on Systems, Man, and Cybernetics, 51(3)(2021), 1794–1804.
28. Wen-wen, G., Lv, Y., Jia-yu, Y., Wang, Z., Yuan-hai, S., Fast support vector classifier with generalization-memorization kernel, Procedia Comput Sci, 214(2022), 55–62.
29. Wilson, T., Wiebe, J., Hoffmann, P., Recognizing contextual polarity: An exploration of features for phrase-level sentiment analysis, Computational Linguistics, 35(3)(2009), 399–433.
30. Zhou, L., Zhao, C., Liu, N., Yao, X., Cheng, Z., Improved LSTM-based deep learning model for COVID-19 prediction using optimized approach, Engineering Applications of Artificial Intelligence, 122(2023), 106157.