Multi-Task Transformer Architectures for Simultaneous Sentiment and Urgency Classification of Turkish Social Media Crisis Communications

Abdulrahman Alsamaraie; Can Yüzkollar

doi:10.35377/saucis...1939701

Multi-Task Transformer Architectures for Simultaneous Sentiment and Urgency Classification of Turkish Social Media Crisis Communications

Abstract

After the Kahramanmaras¸ earthquakes on 6 February 2023, approximately one million online messages related to the crisis were generated in a short period of time. Although these online messages contain abundant information, they also include a mixture of urgent requests and noise. Furthermore, the short length of such messages and the difficulty of interpreting them during crisis situations make it challenging to extract critical information. Instead of modeling emotion and urgency as separate components, an integrated model may reveal deeper underlying patterns in Turkish crisis discourse. In the revised experimental design, 25,000 tweets were manually annotated. Sentiment analysis was formulated as a binary task with two labels, negative and neutral, while urgency was defined as a multi-class task with five escalating labels from U1 to U5. Within this revised setting, classical TF–IDF methods, sequence-based neural models, and transformer-based architectures were implemented as baseline approaches. Subsequently, a shared-encoder multi-task framework with separate heads for sentiment and urgency was developed. In the single-task setting, BERTurk (cased) achieved the highest Macro-F1 and AUC scores for sentiment classification, while XLM-RoBERTa-large (thresholded) obtained a slightly higher accuracy. BERTurk (cased) achieved 92.44% accuracy, 87.17% Macro-F1, and 95.39% AUC for sentiment classification. For urgency classification, BERTurk (cased) achieved the strongest overall performance, reaching 77.20% accuracy, 77.11% Macro-F1, and 94.35% Macro-AUC. In the multi-task setting, XLM-RoBERTa-large achieved the strongest joint performance among the evaluated models. It reached 90.62% accuracy, 85.26% Macro-F1, and 95.66% AUC for sentiment classification, together with 79.18% accuracy, 78.50% Macro-F1, and 95.30% Macro-AUC for urgency classification. The manually labeled training and evaluation dataset used in this study is publicly available at https://github.com/AbdalrahmanM/turkish-crisis-sentiment-urgency-dataset.

Keywords

Ethical Statement

The study used a publicly available dataset and did not involve direct interaction with human participants. The authors declare that scientific and ethical principles were followed throughout the preparation of this study, and all sources used are properly cited.

References

B. Ilyas and A. Sharifi, “A systematic review of social media-based sentiment analysis in disaster risk management,” Int. J. Disaster Risk Reduct., vol. 123, Jun. 2025, Art. no. 105487, doi: 10.1016/j.ijdrr.2025.105487.
E. Laurenti, N. Bourgon, F. Benamara, A. Mari, V. Moriceau, and C. Courgeon, “Give me your intentions, I’ll predict our actions: A two-level classification of speech acts for crisis management in social media,” in Proc. 13th Lang. Resources Evaluation Conf., Marseille, France, 2022, pp. 4333–4343. [Online]. Available: https://aclanthology.org/2022.lrec-1.462/
J. Kim, M. Ki, J. Yang, C. Na, J. Kim, and C. Han, “Feasibility of big data analytics in disaster psychiatry: Impact of Seoul Itaewon tragedy on sentiment distribution on Twitter,” Soc. Sci. Med., vol. 359, Oct. 2024, Art. no. 117276, doi: 10.1016/j.socscimed.2024.117276.
R. McCreadie, C. Buntain, and I. Soboroff, “TREC incident streams: Finding actionable information on social media,” in Proc. 16th Int. Conf. Inf. Syst. Crisis Response Manage. (ISCRAM), Valencia, Spain, 2019, pp. 691–705.
I. Ullah, S. Khan, M. Imran, and Y.-K. Lee, “RweetMiner: Automatic identification and categorization of help requests on Twitter during disasters,” Expert Syst. Appl., vol. 176, Aug. 2021, Art. no. 114787, doi: 10.1016/j.eswa.2021.114787.
A. L. Hughes and L. Palen, “Twitter adoption and use in mass convergence and emergency events,” Int. J. Emergency Manage., vol. 6, no. 3–4, pp. 248–260, 2009, doi: 10.1504/IJEM.2009.031564.
S. Vieweg, A. L. Hughes, K. Starbird, and L. Palen, “Microblogging during two natural hazards events: What Twitter may contribute to situational awareness,” in Proc. SIGCHI Conf. Human Factors Comput. Syst. (CHI ’10), Atlanta, GA, USA, Apr. 10–15, 2010, pp. 1079–1088, doi: 10.1145/1753326.1753486.
M. Mendoza, B. Poblete, and C. Castillo, “Twitter under crisis: Can we trust what we RT?,” in Proc. 1st Workshop Social Media Anal. (SOMA ’10), Washington, DC, USA, Jul. 25, 2010, pp. 71–79, doi: 10.1145/1964858.1964869.

K. Starbird and L. Palen, “‘Voluntweeters’: Self-organizing by digital volunteers in times of crisis,” in Proc. SIGCHI Conf. Human Factors Comput. Syst. (CHI ’11), Vancouver, BC, Canada, May 7–12, 2011, pp. 1071–1080, doi: 10.1145/1978942.1979102.
L. Palen and K. M. Anderson, “Crisis informatics—New data for extraordinary times,” Science, vol. 353, no. 6296, pp. 224–225, Jul. 2016, doi: 10.1126/science.aag2579.
C. Reuter and M.-A. Kaufhold, “Fifteen years of social media in emergencies: A retrospective review and future directions for crisis informatics,” J. Contingencies Crisis Manage., vol. 26, no. 1, pp. 41–57, Mar. 2018, doi: 10.1111/1468-5973.12196.
d. boyd and K. Crawford, “Critical questions for big data,” Inf. Commun. Soc., vol. 15, no. 5, pp. 662–679, Jun. 2012, doi: 10.1080/1369118X.2012.678878.
D. E. Alexander, “Social media in disaster risk reduction and crisis management,” Sci. Eng. Ethics, vol. 20, no. 3, pp. 717–733, Sep. 2014, doi: 10.1007/s11948-013-9502-z.
W. D. Lewis, H. Zhu, K. Strawn, and F. Xia, “Tapping into social media in crisis: A survey,” in Proc. 4th Workshop NLP Positive Impact (NLP4PI), Vienna, Austria, 2025, pp. 306–331, doi: 10.18653/v1/2025.nlp4pi-1.27.
N. Aljedani, R. Alotaibi, and A. Cherif, “AI–social media integration for crisis management: A systematic review of data and learning aspects,” Appl. Sci., vol. 15, no. 22, Nov. 2025, Art. no. 12283, doi: 10.3390/app152212283.
J. Abdul Wahid et al., “AI-driven social media text analysis during crisis: A review for natural disasters and pandemics,” Appl. Soft Comput., vol. 171, Mar. 2025, Art. no. 112774, doi: 10.1016/j.asoc.2025.112774.
F. G¨und¨uz Zeybeko˘glu, E. Atalay, K. Torpus¸, E. Bekircan, and G. Usta, “Evaluation of Kahramanmaras¸ centered earthquake in T¨urkiye in terms of disaster crisis management: The case of AFAD X (Twitter),” Karamano˘glu Mehmetbey ¨Universitesi Sosyal ve Ekonomik Aras¸tırmalar Dergisi, vol. 27, no. 49, pp. 1053–1076, Aug. 2025, doi: 10.18493/kmusekad.1569072.
A. Olteanu, C. Castillo, F. Diaz, and S. Vieweg, “CrisisLex: A lexicon for collecting and filtering microblogged communications in crises,” Proc. Int. AAAI Conf. Web Social Media (ICWSM), vol. 8, no. 1, pp. 376–385, May 2014, doi: 10.1609/icwsm.v8i1.14538.
A. Olteanu, S. Vieweg, and C. Castillo, “What to expect when the unexpected happens: Social media communications across crises,” in Proc. ACM Conf. Comput.-Supported Cooperative Work Social Comput. (CSCW), Vancouver, BC, Canada, Mar. 14–18, 2015, pp. 994–1009, doi: 10.1145/2675133.2675242.
M. Imran, S. Elbassuoni, C. Castillo, F. Diaz, and P. Meier, “Practical extraction of disaster-relevant information from social media,” in Proc. 22nd Int. Conf. World Wide Web (WWW ’13 Companion), Rio de Janeiro, Brazil, May 13–17, 2013, pp. 1021–1024, doi: 10.1145/2487788.2488109.
M. Imran, C. Castillo, F. Diaz, and S. Vieweg, “Processing social media messages in mass emergency: A survey,” ACM Comput. Surv., vol. 47, no. 4, Jul. 2015, Art. no. 67, pp. 1–38, doi: 10.1145/2771588.
F. Alam, H. Sajjad, M. Imran, and F. Ofli, “CrisisBench: Benchmarking crisis-related social media datasets for humanitarian information processing,” Proc. Int. AAAI Conf. Web Social Media (ICWSM), vol. 15, no. 1, pp. 923–932, May 2021, doi: 10.1609/icwsm.v15i1.18115.
F. Alam, F. Ofli, and M. Imran, “CrisisMMD: Multimodal Twitter datasets from natural disasters,” Proc. Int. AAAI Conf. Web Social Media (ICWSM), vol. 12, no. 1, pp. 465–473, Jun. 2018, doi: 10.1609/icwsm.v12i1.14983.
A. K. Rashid and O. Fındık, “Sentiment analysis of Twitter: Turkey earthquake 2023 case,” in Proc. SMARTINDUSTRY-2024: Int. Conf. Smart Automat. Robot. Future Ind., CEUR Workshop Proceedings, vol. 3699, 2024, pp. 198–208. [Online]. Available: https://ceur-ws.org/Vol-3699/paper14.pdf
B. Pang and L. Lee, “Opinion mining and sentiment analysis,” Found. Trends Inf. Retr., vol. 2, nos. 1–2, pp. 1–135, Jul. 2008, doi: 10.1561/1500000011.
B. Liu, Sentiment Analysis and Opinion Mining, ser. Synthesis Lectures on Human Language Technologies, vol. 5, no. 1. San Rafael, CA, USA: Morgan & Claypool Publishers, 2012, pp. 1–167, doi: 10.2200/S00416ED1V01Y201204HLT016.
Y. Kim, “Convolutional neural networks for sentence classification,” in Proc. 2014 Conf. Empirical Methods Natural Lang. Process. (EMNLP), Doha, Qatar, Oct. 25–29, 2014, pp. 1746–1751, doi: 10.3115/v1/D14-1181.
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput., vol. 9, no. 8, pp. 1735–1780, Nov. 1997, doi: 10.1162/neco.1997.9.8.1735.
M. Schuster and K. K. Paliwal, “Bidirectional recurrent neural networks,” IEEE Trans. Signal Process., vol. 45, no. 11, pp. 2673–2681, Nov. 1997, doi: 10.1109/78.650093.
A. Vaswani et al., “Attention is all you need,” in Adv. Neural Inf. Process. Syst., vol. 30, Long Beach, CA, USA, Dec. 4–9, 2017, pp. 5998–6008.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proc. 2019 Conf. North Amer. Chapter Assoc. Comput. Linguistics: Human Lang. Technol. (NAACL-HLT), vol. 1 (Long and Short Papers), Minneapolis, MN, USA, Jun. 2019, pp. 4171–4186, doi: 10.18653/v1/N19-1423.
Y. Liu et al., “RoBERTa: A robustly optimized BERT pretraining approach,” arXiv:1907.11692, 2019, doi: 10.48550/arXiv.1907.11692.
S. Aslan and M. Yildirim, “A novel attention-based deep learning model for improving sentiment classification after the case of the 2023 Kahraman- maras/Turkey earthquake on Twitter,” PeerJ Comput. Sci., vol. 11, May 2025, Art. no. e2881, doi: 10.7717/peerj-cs.2881.
K. Zahra, M. Imran, and F. O. Ostermann, “Automatic identification of eyewitness messages on Twitter during disasters,” Inf. Process. Manage., vol. 57, no. 1, Jan. 2020, Art. no. 102107, doi: 10.1016/j.ipm.2019.102107.
D. T. Nguyen, K. A. Al Mannai, S. Joty, H. Sajjad, M. Imran, and P. Mitra, “Robust classification of crisis-related data on social networks using convolutional neural networks,” Proc. Int. AAAI Conf. Web Social Media (ICWSM), vol. 11, no. 1, pp. 632–635, May 2017, doi: 10.1609/icwsm.v11i1.14950.
R. Meunier, F. Benamara, V. Moriceau, Z. Qiao, and S. Ramasamy, “CrisisTS: Coupling social media textual data and meteorological time series for urgency classification,” in Proc. 63rd Annu. Meeting Assoc. Comput. Linguistics (vol. 1: Long Papers), Vienna, Austria, 2025, pp. 16082–16099, doi: 10.18653/v1/2025.acl-long.783.
A. Conneau et al., “Unsupervised cross-lingual representation learning at scale,” in Proc. 58th Annu. Meeting Assoc. Comput. Linguistics (ACL), Online, Jul. 2020, pp. 8440–8451, doi: 10.18653/v1/2020.acl-main.747.
P. He, X. Liu, J. Gao, and W. Chen, “DeBERTa: Decoding-enhanced BERT with disentangled attention,” in Proc. Int. Conf. Learn. Representations (ICLR), Virtual Event, Austria, May 3–7, 2021. [Online]. Available: https://openreview.net/forum?id=XPZIaotutsD
R. Caruana, “Multitask learning,” Mach. Learn., vol. 28, no. 1, pp. 41–75, Jul. 1997, doi: 10.1023/A:1007379606734.
S. Ruder, “An overview of multi-task learning in deep neural networks,” arXiv:1706.05098, 2017, doi: 10.48550/arXiv.1706.05098.
X. Liu, P. He, W. Chen, and J. Gao, “Multi-task deep neural networks for natural language understanding,” in Proc. 57th Annu. Meeting Assoc. Comput. Linguistics (ACL), Florence, Italy, Jul. 2019, pp. 4487–4496, doi: 10.18653/v1/P19-1441.
J. Liu, T. Singhal, L. T. M. Blessing, K. L. Wood, and K. H. Lim, “CrisisBERT: A robust transformer for crisis classification and contextual crisis embedding,” in Proc. 32nd ACM Conf. Hypertext Social Media (HT ’21), Online, Aug. 30–Sep. 2, 2021, pp. 133–141, doi: 10.1145/3465336.3475117.
R. Lamsal, M. Rodriguez Read, and S. Karunasekera, “CrisisTransformers: Pre-trained language models and sentence encoders for crisis-related social media texts,” Knowl.-Based Syst., vol. 296, Jul. 2024, Art. no. 111916, doi: 10.1016/j.knosys.2024.111916.
S. Schweter, “BERTurk—BERT models for Turkish,” version 1.0.0, Zenodo, 2020, doi: 10.5281/zenodo.3770924.
A. B. Altınel Girgin, G. G¨um¨us¸c¸ekic¸c¸i, and N. C. Birdemir, “Turkish sentiment analysis: A comprehensive review,” Sigma J. Eng. Nat. Sci., vol. 42, no. 4, pp. 1292–1314, Aug. 2024, doi: 10.14744/sigma.2024.00033.
E. Akkas¸o˘glu, “Turkey 06-02-2023 eartquake Tweets,” Kaggle, 2023. Accessed: Oct. 12, 2026. [Online]. Available: https://www.kaggle.com/datasets/esatakkasoglu/turkey-06-02-2023-eartquake-tweets
M. Grootendorst, “BERTopic: Neural topic modeling with a class-based TF–IDF procedure,” arXiv:2203.05794, 2022, doi: 10.48550/arXiv.2203.05794.
M. T. Ribeiro, S. Singh, and C. Guestrin, “‘Why should I trust you?’: Explaining the predictions of any classifier,” in Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining (KDD ’16), San Francisco, CA, USA, Aug. 13–17, 2016, pp. 1135–1144, doi: 10.1145/2939672.2939778.
Streamlit: The Fastest Way to Build and Share Data Apps. (2019). Streamlit Inc. Accessed: Oct. 12, 2026. [Online]. Available: https://streamlit.io/

Details

Primary Language

English

Subjects

Computing Applications in Life Sciences, Natural Language Processing

Journal Section

Research Article

Authors

Abdulrahman Alsamaraie
0009-0003-2997-9398
Iraq

Can Yüzkollar ^*
0000-0001-5823-8773
Türkiye

Early Pub Date

October 27, 2026

Publication Date

October 30, 2026

Submission Date

April 28, 2026

Acceptance Date

June 3, 2026

Published in Issue

Year 2026 Volume: 9 Number: 5

DOI

https://doi.org/10.35377/saucis...1939701

IZ

https://izlik.org/JA47YC49PN

Cite

RIS / Bibtex

APA

Alsamaraie, A., & Yüzkollar, C. (2026). Multi-Task Transformer Architectures for Simultaneous Sentiment and Urgency Classification of Turkish Social Media Crisis Communications. Sakarya University Journal of Computer and Information Sciences, 9(5), 1350-1364. https://doi.org/10.35377/saucis...1939701

AMA

1.Alsamaraie A, Yüzkollar C. Multi-Task Transformer Architectures for Simultaneous Sentiment and Urgency Classification of Turkish Social Media Crisis Communications. SAUCIS. 2026;9(5):1350-1364. doi:10.35377/saucis.1939701

Chicago

Alsamaraie, Abdulrahman, and Can Yüzkollar. 2026. “Multi-Task Transformer Architectures for Simultaneous Sentiment and Urgency Classification of Turkish Social Media Crisis Communications”. Sakarya University Journal of Computer and Information Sciences 9 (5): 1350-64. https://doi.org/10.35377/saucis. 1939701.

EndNote

Alsamaraie A, Yüzkollar C (October 1, 2026) Multi-Task Transformer Architectures for Simultaneous Sentiment and Urgency Classification of Turkish Social Media Crisis Communications. Sakarya University Journal of Computer and Information Sciences 9 5 1350–1364.

IEEE

[1]A. Alsamaraie and C. Yüzkollar, “Multi-Task Transformer Architectures for Simultaneous Sentiment and Urgency Classification of Turkish Social Media Crisis Communications”, SAUCIS, vol. 9, no. 5, pp. 1350–1364, Oct. 2026, doi: 10.35377/saucis...1939701.

ISNAD

Alsamaraie, Abdulrahman - Yüzkollar, Can. “Multi-Task Transformer Architectures for Simultaneous Sentiment and Urgency Classification of Turkish Social Media Crisis Communications”. Sakarya University Journal of Computer and Information Sciences 9/5 (October 1, 2026): 1350-1364. https://doi.org/10.35377/saucis. 1939701.

JAMA

1.Alsamaraie A, Yüzkollar C. Multi-Task Transformer Architectures for Simultaneous Sentiment and Urgency Classification of Turkish Social Media Crisis Communications. SAUCIS. 2026;9:1350–1364.

MLA

Alsamaraie, Abdulrahman, and Can Yüzkollar. “Multi-Task Transformer Architectures for Simultaneous Sentiment and Urgency Classification of Turkish Social Media Crisis Communications”. Sakarya University Journal of Computer and Information Sciences, vol. 9, no. 5, Oct. 2026, pp. 1350-64, doi:10.35377/saucis. 1939701.

Vancouver

1.Abdulrahman Alsamaraie, Can Yüzkollar. Multi-Task Transformer Architectures for Simultaneous Sentiment and Urgency Classification of Turkish Social Media Crisis Communications. SAUCIS. 2026 Oct. 1;9(5):1350-64. doi:10.35377/saucis. 1939701