Multiclass Emotion Analysis Of Public Perception Of Mpox Disease In X (Twitter) Data

Kübra Çetin Yildiz; Emine Cengil

doi:10.17798/bitlisfen.1863465

Multiclass Emotion Analysis Of Public Perception Of Mpox Disease In X (Twitter) Data

Abstract

This study aims to conduct sentiment analysis based on opinions expressed about monkeypox (mpox) in tweets shared on the X platform. To this end, 51,894 tweets containing the keywords “mpox” and “monkeypox” were collected from the X platform between January and June 2025. During the data collecting procedure, a keyword-based scanning strategy and API-based data retrieval techniques were used to retrieve English tweets. The raw data underwent extensive preprocessing to guarantee data quality and enhance the model's classification performance. These actions included changing the content to lowercase, eliminating URLs, user tags, and retweet phrases, and converting emojis to their textual equivalents. After these processes, a consistent dataset of 47,464 tweets suitable for analysis was obtained. The collected tweets were labeled with seven different categories describing the emotions they contained: fear, surprise, joy, sadness, anger, disgust, and neutrality. In the labeling process, automatic pre-labeling was performed using a zero-shot classification method, followed by manual verification by three independent experts. Inter-rater agreement was measured using Fleiss Kappa, and a high reliability level of 0.78 was achieved. The classification process was performed using five different Transformer-based models. The XLNet model demonstrated the highest performance, achieving an average accuracy of 99.05%, an F1 score of 95.64%, and a weighted F1 score of 99.05%. In conclusion, this study examined the public's emotional perception of global health threats such as mpox through social media using multi-class sentiment analysis. These findings will contribute to a better understanding of risk perception in the context of public health communication.

Keywords

References

R. Sah, B. K. Padhi, A. Siddiq, A. Abdelaal, A. Reda, B. Ismail Lashin, et al., “Public health emergency of international concern declared by the World Health Organization for monkeypox,” Global Security: Health, Science and Policy, vol. 7, no. 1, pp. 51–56, 2022, doi: 10.1080/23779497.2022.2124185.
C. Orkin, R. Torgler, R. Dawson, I. W. Holloway, and C. Hoffmann, “mpox: Exploring Epidemiology, Disease Outcomes, and Preventive Vaccination Among People Living with HIV During Ongoing Outbreaks,” Viruses, vol. 17, no. 12, p. 1526, 2025, doi: 10.3390/v17121526.
E. M. Bunge, B. Hoet, L. Chen, F. Lienert, H. Weidenthaler, L. R. Baer, and R. Steffen, “The changing epidemiology of human monkeypox—A potential threat? A systematic review,” PLoS Neglected Tropical Diseases, vol. 16, no. 2, p. e0010141, 2022, doi: 10.1371/journal.pntd.0010141.
S. Kanchan and A. Gaidhane, “Social Media Role and Its Impact on Public Health: A Narrative Review,” Cureus, vol. 15, no. 1, 2023, doi: 10.7759/cureus.33737.
K. Aggarwal, S. K. Singh, M. Chopra, and S. Kumar, “Role of social media in the COVID-19 pandemic: A literature review,” in Data Mining Approaches for Big Data and Sentiment Analysis in Social Media, pp. 91–115, 2022, doi: 10.4018/978-1-7998-8413-2.ch004.
L. Fan, L. Li, and L. Hemphill, “Toxicity on social media during the 2022 Mpox public health emergency: quantitative study of topical and network dynamics,” Journal of Medical Internet Research, vol. 26, p. e52997, 2024, doi:10.2196/52997.
Y. Ortiz-Martínez, J. Sarmiento, D. K. Bonilla-Aldana, and A. J. Rodríguez-Morales, “Monkeypox goes viral: measuring the misinformation outbreak on Twitter” Journal of Infection in Developing Countries, vol. 16, no. 7, pp. 1218–1220, 2022, doi:10.3855/jidc.16907.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proc. 2019 Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), vol. 1 (Long and Short Papers), pp. 4171–4186, 2019, doi: 10.18653/v1/N19-1423.

I. Y. Kusuma, Á. Visnyovszki, and M. A. Bahar, “Mapping the mpox discourse: A network and sentiment analysis,” Exploratory Research in Clinical and Social Pharmacy, vol. 16, p. 100521, 2024, doi: 10.1016/j.rcsop.2024.100521.
M. E. Dalton, R. Duffy, E. Quinn, K. Larsen, C. Peters, D. Brenner, et al., “A qualitative examination of social media sharing and the 2022 monkeypox outbreak: Did early labeling help reduce misinformation or fuel the fire?,” Sexual Health, vol. 21, no. 1, 2024, doi: 10.1071/SH23158.
S. Lai, J. Dong, Z. Lin, Y. Peng, X. Long, L. Hao, et al., “Machine learning-based sentiment analysis for infodemic during the mpox outbreak,” Humanities and Social Sciences Communications, vol. 12, p. 6324, 2025, doi: 10.1057/s41599-025-06324-3.
D. Luo, J. Xu, Y. Jiang, M. Tan, Y. Yao, L. He, et al., “Public interest in mpox in China during the pandemic: A qualitative analysis of TikTok data using latent Dirichlet allocation topic modeling,” Journal of Medical Internet Research, vol. 25, 2023, doi: 10.2196/77424.
H. Gharaibeh, R. E. Al Mamlook, G. Samara, A. Nasayreh, S. Smadi, K. M. O. Nahar, et al., “Arabic sentiment analysis of monkeypox using deep neural networks and optimized hyperparameters,” Social Network Analysis and Mining, vol. 14, p. 30, 2024, doi: 10.1007/s13278-023-01188-4.
V. S. Dsouza, P. Rajkhowa, B. R. Mallya, D. S. Raksha, V. Mrinalini, K. Cauvery, et al., “A sentiment and content analysis of tweets on monkeypox stigma among the LGBTQ+ community: A cue to risk communication plan,” Dialogues in Health, vol. 2, p. 100095, 2023, doi: 10.1016/j.dialog.2022.100095.
S. Bengesi, T. Oladunni, R. Olusegun, and H. Audu, “A machine learning-sentiment analysis on monkeypox outbreak: An extensive dataset to show the polarity of public opinion from Twitter tweets,” IEEE Access, vol. 11, pp. 19743–19759, 2023, doi: 10.1109/ACCESS.2023.3242290.
P. Sv and R. Ittamalla, “What concerns the general public the most about monkeypox virus? - A text analytics study based on Natural Language Processing (NLP),” Travel Medicine and Infectious Disease, vol. 49, p. 102404, 2022, doi: 10.1016/j.tmaid.2022.102404.
L. N. Cooper, A. P. Radunsky, J. J. Hanna, Z. M. Most, T. M. Perl, C. U. Lehmann, and R. J. Medford, “Analyzing an Emerging Pandemic on Twitter: Monkeypox” Open Forum Infectious Diseases, vol. 9, no. 7, 2022, doi: 10.1093/ofid/ofad142.
R. A. Farahat, M. A. Yassin, J. A. Al-Tawfiq, C. A. Bejan, and B. Abdelazeem, “Public perspectives of monkeypox in Twitter: A social media analysis using machine learning,” Travel Medicine and Infectious Disease, vol. 50, p. 102461, 2022, doi: 10.1016/j.nmni.2022.101053.
Y. Wang, K. O’Connor, I. Flores, C. T. Berdahl, R. J. Urbanowicz, R. Stevens, et al., “Mpox Discourse on Twitter by Sexual Minority Men and Gender-Diverse Individuals: Infodemiological Study Using BERTopic,” JMIR Public Health and Surveillance, vol. 9, 2023, doi: 10.2196/59193.
B. Knudsen, T. B. Høeg, and V. Prasad, “Analysis of tweets discussing the risk of Mpox among children and young people in school (May-October 2022): a retrospective observational study,” BMJ Paediatrics Open, vol. 8, p. e002236, 2024, doi: 10.1136/bmjpo-2023-002236.
C. Calabrese, S. Kumble, and X. Yu, “Stigmatization of mpox and the MSM community on Twitter: A computational approach to exploring the negative emotions expressed through stigmatizing language,” Telematics and Informatics, vol. 102, p. 103236, 2025, doi: 10.1016/j.tele.2025.102326.
S. Chakroborty, “A hybrid deep learning framework for early detection of mpox using image data,” Healthcare Analytics, vol. 7, p. 100396, 2025, doi: 10.1016/j.health.2025.100396.
S. Vuran, M. Ucan, M. Akin, and M. Kaya, “Multi-class classification of skin lesion images including mpox disease using transformer-based deep learning architectures,” Diagnostics, vol. 14, no. 3, p. 345, 2024, doi: 10.3390/diagnostics15030374.
D. Q. Nguyen, T. Vu, and a. T. Nguyen, “BERTweet: A pre-trained language model for English Tweets.” In Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pp. 9-14, 2020, doi: 10.48550/arXiv.2005.10200
Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, ... and V. Stoyanov, “Roberta: A robustly optimized bert pretraining approach.” arXiv preprint arXiv:1907.11692, 2019, doi: 10.48550/arXiv.1907.11692.
P. He, J. Gao, and W. Chen, “Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing.” arXiv preprint arXiv:2111.09543, 2021, doi: 10.48550/arXiv.2111.09543.
V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter.” arXiv preprint arXiv:1910.01108, 2019, doi: 10.48550/arXiv.1910.01108.
Z. Yang, Z. Dai, Y. Yang, J. G. Carbonell, R. Salakhutdinov, and Q. V. Le, “XLNet: Generalized autoregressive pretraining for language understanding,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 32, 2019, doi: 10.48550/arXiv.1906.08237.
S. Henning, W. Beluch, A. Fraser, and A. Friedrich, “A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processing,” in Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL), Dubrovnik, Croatia, 2023, pp. 523–540, doi: 10.18653/v1/2023.eacl-main.38.
Z. Dai, Z. Yang, Y. Yang, J. Carbonell, Q. Le, and R. Salakhutdinov, “Transformer-XL: Attentive language models beyond a fixed-length context,” in Proc. 57th Annu. Meeting of the Association for Computational Linguistics (ACL), Florence, Italy, pp. 2978–2988, 2019, doi: 10.48550/arXiv.1901.02860.
HFL (Harbin Institute of Technology & iFLYTEK Research), “Chinese XLNet pre-trained models,” GitHub repository, 2019. [Online]. Available: https://github.com/ymcui/Chinese-XLNet.
A. Alduailej, and A. Alothaim, “AraXLNet: pre-trained language model for sentiment analysis of Arabic,” Journal of Big Data, vol. 9, no. 1, 2022, doi: 10.1186/s40537-022-00625-z.
G. Naidu, T. Zuva, and E. M. Sibanda, “A review of evaluation metrics in machine learning algorithms,” in Proc. Computer Science Online Conference, Cham, Switzerland: Springer, pp. 15–25, 2023, doi: 10.1007/978-3-031-35314-7_2.
K. Blagec, G. Dorffner, M. Moradi, S. Ott, and M. Samwald, “A global analysis of metrics used to measure performance in natural language processing,” arXiv preprint arXiv:2204.11574, 2022J.

Details

Primary Language

English

Subjects

Natural Language Processing

Journal Section

Research Article

Authors

Kübra Çetin Yildiz
0000-0002-4259-2660
Türkiye

Emine Cengil ^*
0000-0003-4313-8694
Türkiye

Publication Date

June 30, 2026

Submission Date

January 14, 2026

Acceptance Date

April 30, 2026

Published in Issue

Year 2026 Volume: 15 Number: 2

DOI

https://doi.org/10.17798/bitlisfen.1863465

IZ

https://izlik.org/JA94FP76EU

Cite

RIS / Bibtex

APA

Çetin Yildiz, K., & Cengil, E. (2026). Multiclass Emotion Analysis Of Public Perception Of Mpox Disease In X (Twitter) Data. Bitlis Eren Üniversitesi Fen Bilimleri Dergisi, 15(2), 977-992. https://doi.org/10.17798/bitlisfen.1863465

AMA

1.Çetin Yildiz K, Cengil E. Multiclass Emotion Analysis Of Public Perception Of Mpox Disease In X (Twitter) Data. Bitlis Eren Üniversitesi Fen Bilimleri Dergisi. 2026;15(2):977-992. doi:10.17798/bitlisfen.1863465

Chicago

Çetin Yildiz, Kübra, and Emine Cengil. 2026. “Multiclass Emotion Analysis Of Public Perception Of Mpox Disease In X (Twitter) Data”. Bitlis Eren Üniversitesi Fen Bilimleri Dergisi 15 (2): 977-92. https://doi.org/10.17798/bitlisfen.1863465.

EndNote

Çetin Yildiz K, Cengil E (June 1, 2026) Multiclass Emotion Analysis Of Public Perception Of Mpox Disease In X (Twitter) Data. Bitlis Eren Üniversitesi Fen Bilimleri Dergisi 15 2 977–992.

IEEE

[1]K. Çetin Yildiz and E. Cengil, “Multiclass Emotion Analysis Of Public Perception Of Mpox Disease In X (Twitter) Data”, Bitlis Eren Üniversitesi Fen Bilimleri Dergisi, vol. 15, no. 2, pp. 977–992, June 2026, doi: 10.17798/bitlisfen.1863465.

ISNAD

Çetin Yildiz, Kübra - Cengil, Emine. “Multiclass Emotion Analysis Of Public Perception Of Mpox Disease In X (Twitter) Data”. Bitlis Eren Üniversitesi Fen Bilimleri Dergisi 15/2 (June 1, 2026): 977-992. https://doi.org/10.17798/bitlisfen.1863465.

JAMA

1.Çetin Yildiz K, Cengil E. Multiclass Emotion Analysis Of Public Perception Of Mpox Disease In X (Twitter) Data. Bitlis Eren Üniversitesi Fen Bilimleri Dergisi. 2026;15:977–992.

MLA

Çetin Yildiz, Kübra, and Emine Cengil. “Multiclass Emotion Analysis Of Public Perception Of Mpox Disease In X (Twitter) Data”. Bitlis Eren Üniversitesi Fen Bilimleri Dergisi, vol. 15, no. 2, June 2026, pp. 977-92, doi:10.17798/bitlisfen.1863465.

Vancouver

1.Kübra Çetin Yildiz, Emine Cengil. Multiclass Emotion Analysis Of Public Perception Of Mpox Disease In X (Twitter) Data. Bitlis Eren Üniversitesi Fen Bilimleri Dergisi. 2026 Jun. 1;15(2):977-92. doi:10.17798/bitlisfen.1863465