PIDMUS: A Pipeline for Identifying Disaster Tweets on Twitter with Multilingual Support Using TF-IDF and Ensemble Learning Model

Öz

In emergencies, X (formerly Twitter) has emerged as an essential and fast information-sharing tool. A mechanism based on TF-IDF (Term Frequency-Inverse Document Frequency), WordNet, an ensemble (stacking) learning model, and multilingual support is proposed to classify disaster-related tweets obtained from the X platform. Moreover, this proposed model is evaluated on eight popular classifiers for classifying textual data. An ensemble of MLP, MNB, CNB, and SVC yields a good overall model with 81.36% accuracy and an AUC of 0.80. The proposed model can predict whether a tweet is related to a disaster by using an unlabeled dataset of tweets. It can classify tweets in multiple languages due to its multilingual support. The model uses Turkish, a low-resource language, to classify disaster-related tweets. Current research shows the application of this model in English and Turkish. This research uses two publicly available datasets of disaster-related tweets. The proposed model achieved good performance metrics on both labeled and unlabeled datasets. The model can be utilized by disaster relief authorities to filter out actual disaster-related tweets automatically and efficiently from many other fake and “clickbait tweets”.

Anahtar Kelimeler

• Natural Language Processing
• Unsupervised Learning
• Text Classification
• Deep Learning
• Disaster Related Tweets

PIDMUS: TF-IDF ve Topluluk Öğrenme Modeli Kullanarak Çok Dilli Destekle Twitter'da Afet Tweetlerini Tanımlamaya Yönelik Bir Süreç

Öz

Acil durumlarda, X (eski adıyla Twitter), önemli ve hızlı bir bilgi paylaşım aracı olarak ortaya çıkmıştır. X platformundan elde edilen afetle ilgili tweet'leri sınıflandırmak için TF-IDF (Terim Frekansı-Ters Belge Frekansı), WordNet, bir topluluk (yığınlama) öğrenme modeli ve çok dilli desteğe dayalı bir mekanizma önerilmiştir. Ayrıca, önerilen bu model, metinsel verileri sınıflandırmak için sekiz popüler sınıflandırıcı üzerinde değerlendirilmiştir. MLP, MNB, CNB ve SVC'nin bir topluluğu, %81,36 doğruluk ve 0,80 AUC ile iyi bir genel model ortaya koymuştur. Önerilen model, etiketlenmemiş bir tweet veri kümesi kullanarak bir tweet'in afetle ilgili olup olmadığını tahmin edebilir. Çok dilli desteği sayesinde birden fazla dildeki tweet'leri sınıflandırabilir. Model, düşük kaynaklı bir dil olan Türkçe'yi afetle ilgili tweet'leri sınıflandırmak için kullanmaktadır. Mevcut araştırma, bu modelin İngilizce ve Türkçe dillerindeki uygulamasını göstermektedir. Bu araştırma, afetle ilgili tweet'lerin iki kamuya açık veri kümesini kullanmaktadır. Önerilen model, hem etiketli hem de etiketsiz veri kümelerinde iyi performans ölçütleri elde etti. Model, afet yardım yetkilileri tarafından gerçek afetle ilgili tweet'leri diğer birçok sahte ve "tıklama tuzağı" tweet'inden otomatik ve verimli bir şekilde filtrelemek için kullanılabilir.

Anahtar Kelimeler

• Doğal Dil İşleme
• Denetimsiz Öğrenme
• Metin Sınıflandırma
• Derin Öğrenme
• Afetle İlgili Tweetler

Kaynakça

1. [1] Karimiziarani, Mohammadsepehr, et al. "Hazard risk awareness and disaster management: Extracting the information content of twitter data." Sustainable Cities and Society 77 (2022): 103577. https://doi.org/10.1016/j.scs.2021.103577
2. [2] DiCarlo, M.F.; Berglund, E.Z. Connected communities improve hazard response: An agent-based model of social media behaviors during hurricanes. Sustain. Cities Soc. 2021, 69, 102836. https://doi.org/10.1016/j.scs.2021.102836
3. [3] Roy, P.K.; Kumar, A.; Singh, J.P.; Dwivedi, Y.K.; Rana, N.P.; Raman, R. Disaster related social media content processing for sustainable cities. Sustain. Cities Soc. 2021, 75, 103363. https://doi.org/10.1016/j.scs.2021.103363
4. [4] Son, J.; Lee, H.K.; Jin, S.; Lee, J. Content features of tweets for effective communication during disasters: A media synchronicity theory perspective. Int. J. Inf. Manag. 2019, 45, 56–68. https://doi.org/10.1016/j.ijinfomgt.2018.10.012
5. [5] Zhai, W.; Peng, Z.R.; Yuan, F. Examine the effects of neighborhood equity on disaster situational awareness: Harness machine learning and geotagged Twitter data. Int. J. Disaster Risk Reduct. 2021, 48, 101611. https://doi.org/10.1016/j.ijdrr.2020.101611
6. [6] Karimiziarani, M.; Jafarzadegan, K.; Abbaszadeh, P.; Shao, W.; Moradkhani, H. Hazard risk awareness and disaster management: Extracting the information content of twitter data. Sustain. Cities Soc. 2022, 77, 103577. https://doi.org/10.1016/j.scs.2021.103577
7. [7] Balakrishnan, V.; Shi, Z.; Law, C.L.; Lim, R.; Teh, L.L.; Fan, Y.; Periasamy, J. A Comprehensive Analysis of Transformer-Deep Neural Network Models in Twitter Disaster Detection. Mathematics 2022, 10, 4664. https://doi.org/10.3390/math10244664
8. [8] Robertson, B.W.; Johnson, M.; Murthy, D.; Smith, W.R.; Stephes, K.K. Using a combination of human insights and ‘deep learning’ for real-time disaster communication. Prog. Disaster Sci. 2019, 2, 100030. https://doi.org/10.1016/j.pdisas.2019.100030

9. [9] Miller, G. (1995). WordNet: A Lexical Database for English. Commun. ACM, 38, 39-41. https://doi.org/10.1145/219717.219748.
10. [10] Tian, H., Zhang, X., Wang, Y., & Zeng, D. (2022). Multi-Task Learning and Improved TextRank for Knowledge Graph Completion. Entropy, 24. https://doi.org/10.3390/e24101495.
11. [11] Li, J., Hou, J., & Zhou, C. (2021). An Improved Capsule Network-based Embedding Model for Knowledge Graph Completion. 2021 33rd Chinese Control and Decision Conference (CCDC), 2247-2251. https://doi.org/10.1109/CCDC52312.2021.9602345.
12. [12] Ma, Z., & Yan, L. (2018). Modeling fuzzy data with RDF and fuzzy relational database models. International Journal of Intelligent Systems, 33, 1534 - 1554. https://doi.org/10.1002/int.21996.
13. [13] Chantrapornchai, C., & Choksuchat, C. (2018). TripleID-Q: RDF Query Processing Framework Using GPU. IEEE Transactions on Parallel and Distributed Systems, 29, 2121-2135. https://doi.org/10.1109/TPDS.2018.2814567.
14. [14] Lan, F. (2022). Research on Text Similarity Measurement Hybrid Algorithm with Term Semantic Information and TF-IDF Method. Advances in Multimedia. https://doi.org/10.1155/2022/7923262.
15. [15] Lovera, F., Cardinale, Y., & Homsi, M. (2021). Sentiment Analysis in Twitter Based on Knowledge Graph and Deep Learning Classification. Electronics. https://doi.org/10.3390/electronics10222739.
16. [16] Sutrisno, A. (2020). The Accuracy and Shortcomings of Google Translate Translating English Sentences to Indonesian. Education Quarterly Reviews. https://doi.org/10.31235/osf.io/dz3e2.
17. [17] Mahajan, D., & Chaudhary, D. (2018). Sentiment Analysis Using Rnn and Google Translator. 2018 8th International Conference on Cloud Computing, Data Science & Engineering (Confluence), 798-802. https://doi.org/10.1109/CONFLUENCE.2018.8442924.
18. [18] Argin, Y. (2023). Doğal Afetlerde Sosyal Medya Kullanimi: 2023 Kahramanmaraş Depremi Özelinde Twitter Örneği. İnsanat Sanat Tasarım ve Mimarlık Araştırmaları Dergisi, 3(1), 140-165. https://izlik.org/JA82WA79KK
19. [19] Ulvi, O., Karamehic-Muratovic, A., Baghbanzadeh, M., Bashir, A., Smith, J., & Haque, U. (2022). Social Media Use and Mental Health: A Global Analysis. Epidemiologia, 3, 11 - 25. https://doi.org/10.3390/epidemiologia3010002.
20. [20] Lamsal, R., & Kumar, T. V. (2023). Twitter-based disaster response using recurrent nets. In Research Anthology on Managing Crisis and Risk Communications (pp. 613-632). IGI Global. https://doi.org/10.4018/978-1-6684-7145-6.ch031
21. [21] Didi, Y., Walha, A., & Wali, A. (2022). COVID-19 tweets classification based on a hybrid word embedding method. Big Data and Cognitive Computing, 6(2), 58. https://doi.org/10.3390/bdcc6020058
22. [22] Gulati, K., Kumar, S. S., Boddu, R. S. K., Sarvakar, K., Sharma, D. K., & Nomani, M. Z. M. (2022). Comparative analysis of machine learning-based classification models using sentiment classification of tweets related to COVID-19 pandemic. Materials Today: Proceedings, 51, 38-41. https://doi.org/10.1016/j.matpr.2021.04.364
23. [23] Bikku, T., Chandrika, P., Kanyadari, A., Prathima, V., & Sai, B. B. (2022). Analysis of Disaster Tweets Using Natural Language Processing. In Intelligent Computing and Applications: Proceedings of ICDIC 2020 (pp. 491-501). Singapore: Springer Nature Singapore. https://doi.org/10.1007/978-981-19-4162-7_46
24. [24] Dilrukshi, I., De Zoysa, K., & Caldera, A. (2013, April). Twitter news classification using SVM. In 2013 8th International Conference on Computer Science & Education (pp. 287-291). IEEE. https://doi.org/10.1109/ICCSE.2013.6553926
25. [25] Chy, A. N., Seddiqui, M. H., & Das, S. (2014, March). Bangla news classification using naive Bayes classifier. In 16th Int'l Conf. Computer and Information Technology (pp. 366-371). IEEE. https://doi.org/10.1109/ICCITechn.2014.6997369
26. [26] Khaleq, A. A., & Ra, I. (2019). Twitter analytics for disaster relevance and disaster phase discovery. In Proceedings of the Future Technologies Conference (FTC) 2018: Volume 1 (pp. 401-417). Springer International Publishing. https://doi.org/10.1007/978-3-030-02686-8_31
27. [27] Nugent, T., Petroni, F., Raman, N., Carstens, L., & Leidner, J. L. (2017, December). A comparison of classification models for natural disaster and critical event detection from news. In 2017 IEEE international conference on big data (Big Data) (pp. 3750-3759). IEEE. https://doi.org/10.1109/BigData.2017.8258374
28. [28] Pratama, R. A., & Pardede, H. F. (2023). Disaster tweet classifications using hybrid convolutional layers and gated recurrent unit. International Journal of Computing and Digital Systems, 14(1), 1-1. http://dx.doi.org/10.12785/ijcds/140174
29. [29] Suleymanov, U., Rustamov, S., Zulfugarov, M., Orujov, O., Musayev, N., & Alizade, A. (2018, October). Empirical study of online news classification using machine learning approaches. In 2018 IEEE 12th International Conference on Application of Information and Communication Technologies (AICT) (pp. 1-6). IEEE. https://doi.org/10.1109/ICAICT.2018.8747012
30. [30] Alhammadi, Humaid, "Using Machine Learning in Disaster Tweets Classification" (2022). Thesis. Rochester Institute of Technology. Accessed from https://repository.rit.edu/theses/11161
31. [31] Rathod, J., Rathod, G., Upadhyay, P., & Vakhare, P. (2022, May). Disaster Tweet Classification using ML. In 2022 International Conference on Applied Artificial Intelligence and Computing (ICAAIC) (pp. 523-527). IEEE. https://doi.org/10.1109/ICAAIC53929.2022.9792836
32. [32] Chen, Z.; Lim, S. Social media data-based typhoon disaster assessment. Int. J. Disaster Risk Reduct. 2021, 64, 102482. https://doi.org/10.1016/j.ijdrr.2021.102482
33. [33] Resch, B.; Usländer, F.; Havas, C. Combining machine-learning topic models and spatiotemporal analysis of social media data for disaster footprint and damage assessment. Cartogr. Geogr. Inf. Sci. 2018, 45, 362–376. https://doi.org/10.1080/15230406.2017.1356242
34. [34] Magueresse, A.; Carles, V.; Heetderks, E. Low-resource languages: A review of past work and future challenges. arXiv 2020, arXiv:2006.07264. https://doi.org/10.48550/arXiv.2006.07264
35. [35] Cieri, C.; Maxwell, M.; Strassel, S.; Tracey, J. Selection criteria for low resource language programs. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), Portoroz, Slovenia, 23–28 May 2016; pp. 4543–4549.
36. [36] Guellil, I.; Saâdane, H.; Azouaou, F.; Gueni, B.; Nouvel, D. Arabic natural language processing: An overview. J. King Saud Univ. Comput. Inf. Sci. 2021, 33, 497–507. https://doi.org/10.1016/j.jksuci.2019.02.006
37. [37] Mohamed, S.A.; Elsayed, A.A.; Hassan, Y.F.; Abdou, M.A. Neural machine translation: Past, present, and future. Neural Comput. Appl. 2021, 33, 15919–15931. https://doi.org/10.1007/s00521-021-06268-0
38. [38] Sutskever, I.; Vinyals, O.; Le, Q. Sequence to Sequence Learning with Neural Networks. Adv. Neural Inf. Process. Syst. 2014, 4, 1–9.
39. [39] Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. In Advances in Neural Information Processing Systems; Curran Associates: Red Hook, NY, USA, 2017; pp. 6000–6010.
40. [40] Haddow, B., Bawden, R., Miceli Barone, A. V., Helcl, J., & Birch, A. (2022). Survey of Low-Resource Machine Translation. Computational Linguistics, 48(3), 673-732. https://doi.org/10.1162/coli_a_00446
41. [41] Addison Howard, devrishi, Phil Culliton, Yufeng Guo. (2019). Natural Language Processing with Disaster Tweets. Kaggle. https://kaggle.com/competitions/nlp-getting-started (accessed on 26 January 2026 ).
42. [42] Turkish Unlabelled Natural Disaster Tweet Texts. Available online: https://www.kaggle.com/datasets/kubilayayturan1/turkish-unlabelled-natural-disaster-tweet-texts/data (accessed on 26 January 2026).
43. [43] Rudra, Koustav, et al. "Extracting situational information from microblogs during disaster events: a classification-summarization approach." Proceedings of the 24th ACM international on conference on information and knowledge management. 2015. https://doi.org/10.1145/2806416.2806485
44. [44] Ayturan, K. (2024). Data querying from heterogeneous data sources on the internet using artificial intelligence and deep learning algorithms [Doctoral thesis, Gazi University].