Machine Learning based Emotion classification in the COVID-19 Real World Worry Dataset

Year 2021, Volume: 6 Issue: 1, 24 - 31, 01.03.2021

Hakan Çakar , Abdulkadir Sengur

Abstract

COVID-19 pandemic has a dramatic impact on economies and communities all around the world. With social distancing in place and various measures of lockdowns, it becomes significant to understand emotional responses on a great scale. In this paper, a study is presented that determines human emotions during COVID-19 using various machine learning (ML) approaches. To this end, various techniques such as Decision Trees (DT), Support Vector Machines (SVM), k-nearest neighbor (k-NN), Neural Networks (NN) and Naïve Bayes (NB) methods are used in determination of the human emotions. The mentioned techniques are used on a dataset namely Real World Worry dataset (RWWD) that was collected during COVID-19. The dataset, which covers eight emotions on a 9-point scale, grading their anxiety levels about the COVID-19 situation, was collected by using 2500 participants. The performance evaluation of the ML techniques on emotion prediction is carried out by using the accuracy score. Five-fold cross validation technique is also adopted in experiments. The experiment works show that the ML approaches are promising in determining the emotion in COVID-19 RWWD. More specifically, the NN method produced the highest average accuracy scores for both emotion and gender classification where a 75.7% and 72.1% average scores were obtained.

Keywords

COVID-19 worry dataset, emotion classification, machine learning

References

• Agrawal, A., & An, A. (2012). Unsupervised emotion detection from text using semantic and syntactic relations. Proceedings - 2012 IEEE/WIC/ACM International Conference on Web Intelligence, WI 2012, 346–353. https://doi.org/10.1109/WI-IAT.2012.170
• Aha, D. W., Kibler, D., & Albert, M. K. (1991). Instance-based learning algorithms. Machine Learning, 6(1), 37–66. https://doi.org/10.1007/BF00153759
• Akbulut, Y., Sengur, A., Guo, Y., & Smarandache, F. (2017). NS-k-NN: Neutrosophic set-based k-nearest neighbors classifier. Symmetry, 9(9). https://doi.org/10.3390/sym9090179
• Alm, C. O., Roth, D., & Sproat, R. (2005). Emotions from text. October, 579–586. https://doi.org/10.3115/1220575.1220648
• Altuntaş, Y., Kocamaz, A. F., Cömert, Z., Cengiz, R., & Esmeray, M. (2019). Identification of Haploid Maize Seeds using Gray Level Co-occurrence Matrix and Machine Learning Techniques. 2018 International Conference on Artificial Intelligence and Data Processing, IDAP 2018, 8–12. https://doi.org/10.1109/IDAP.2018.8620740
• Bandhakavi, A., Wiratunga, N., Deepak, P., & Massie, S. (2014). Generating a word-emotion lexicon from #emotional tweets. Proceedings of the 3rd Joint Conference on Lexical and Computational Semantics, *SEM 2014, 12–21. https://doi.org/10.3115/v1/s14-1002
• Bandhakavi, A., Wiratunga, N., Massie, S., & Padmanabhan, D. (2017). Lexicon Generation for Emotion Detection from Text. IEEE Intelligent Systems, 32(1), 102–108. https://doi.org/10.1109/MIS.2017.22
• Bandhakavi, A., Wiratunga, N., Padmanabhan, D., & Massie, S. (2017). Lexicon based feature extraction for emotion text classification. Pattern Recognition Letters, 93, 133–142. https://doi.org/10.1016/j.patrec.2016.12.009
• Boynukalin, Z. (2012). Emotion Analysis of Turkish Texts by Using Machine Learning Methods. Middle East Technical University.
• Canales, L., & Martínez-Barco, P. (2015). Emotion Detection from text: A Survey. 37–43. https://doi.org/10.3115/v1/w14-6905
• Chudacek, V., Spilka, J., Rubackova, B., Koucky, M., Georgoulas, G., Lhotska, L., & Stylios, C. (2008). Evaluation of feature subsets for classification of cardiotocographic recordings. Computers in Cardiology, 35(21), 845–848. https://doi.org/10.1109/CIC.2008.4749174
• Cömert, Z., & Kocamaz, A. F. (2017). Comparison of machine learning techniques for fetal heart rate classification. Acta Physica Polonica A, 132(3), 451–454. https://doi.org/10.12693/APhysPolA.132.451
• Fakhri, A., Nasir, A., Conf, I. O. P., Mater, S., Eng, S., Fakhri, A., Nasir, A., Nee, E. S., Choong, C. S., Shahrizan, A., Ghani, A., Majeed, A. P. P. A., Adam, A., & Furqan, M. (2020). Text-based emotion prediction system using machine learning approach. https://doi.org/10.1088/1757-899X/769/1/012022
• Hasan, M., Agu, E., & Rundensteiner, E. (2014). Using Hashtags as Labels for Supervised Learning of Emotions in Twitter Messages. Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, 187–193.
• Huang, M.-L., & Hsu, Y.-Y. (2012). Fetal distress prediction using discriminant analysis, decision tree, and artificial neural network. Journal of Biomedical Science and Engineering, 05(09), 526–533. https://doi.org/10.4236/jbise.2012.59065
• Kleinberg, B., van der Vegt, I., & Mozes, M. (2020). Measuring Emotions in the COVID-19 Real World Worry Dataset. 1. http://arxiv.org/abs/2004.04225
• Mohri, M., Rostamizadeh, A., & Talwalkar, A. (2012). Foundations of Machine Learning. MIT Press.
• Rey-Villamizar, N., Shrestha, P., Sadeque, F., Bethard, S., Pedersen, T., Mukherjee, A., & Solorio, T. (2016). Analysis of Anxious Word Usage on Online Health Forums. May 2017, 37–42. https://doi.org/10.18653/v1/w16-6105
• Roberts, K., Roach, M. A., Johnson, J., Guthrie, J., & Harabagiu, S. M. (2012). EmpaTweet: Annotating and detecting emotions on twitter. Proceedings of the 8th International Conference on Language Resources and Evaluation, LREC 2012, 3806–3813.
• Safavian, S. R., & Landgrebe, D. (1991). A survey of decision tree classifier methodology. IEEE Transactions on Systems, Man, and Cybernetics, 21(3), 660–674. https://doi.org/10.1109/21.97458
• Sahin, H., & Subasi, A. (2015). Classification of the cardiotocogram data for anticipation of fetal risks using machine learning techniques. Applied Soft Computing Journal, 33, 231–238. https://doi.org/10.1016/j.asoc.2015.04.038
• Seyeditabari, A., Tabari, N., & Zadrozny, W. (2018). Emotion Detection in Text: a Review. http://arxiv.org/abs/1806.00674
• Suhasini, M., & Badugu, S. (2018). Two Step Approach for Emotion Detection on Twitter Data. International Journal of Computer Applications, 179(53), 12–19. https://doi.org/10.5120/ijca2018917350
• Suttles, J., & Ide, N. (2013). Distant supervision for emotion classification with discrete binary values. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 7817 LNCS(PART 2), 121–136. https://doi.org/10.1007/978-3-642-37256-8_11
• Taran, S., & Bajaj, V. (2019). Emotion recognition from single-channel EEG signals using a two-stage correlation and instantaneous frequency-based filtering method. Computer Methods and Programs in Biomedicine, 173, 157–165. https://doi.org/10.1016/j.cmpb.2019.03.015
• van der Vegt, I., & Kleinberg, B. (2020). Women worry about family, men about the economy: Gender differences in emotional responses to COVID-19. 1–12. http://arxiv.org/abs/2004.08202
• Wang, W., Chen, L., Thirunarayan, K., & Sheth, A. P. (2012). Harnessing twitter “big data” for automatic emotion identification. Proceedings - 2012 ASE/IEEE International Conference on Privacy, Security, Risk and Trust and 2012 ASE/IEEE International Conference on Social Computing, SocialCom/PASSAT 2012, 587–592. https://doi.org/10.1109/SocialCom-PASSAT.2012.119