Performance Analysis of NLP-Based Machine Learning Algorithms in Cyberbullying Detection

Year 2024, Volume: 17 Issue: 2, 445 - 459, 31.08.2024

Funda Akar

https://doi.org/10.18185/erzifbed.1474112

Abstract

In today's pervasive online landscape, the escalating threat of cyberbullying demands advanced detection and mitigation tools. This study utilizes Natural Language Processing (NLP) techniques to confront this imperative challenge, particularly in the dynamic realm of social media, focusing on tweets. A comprehensive NLP-based classification methods is deployed to uncover instances of cyberbullying. Nine prominent machine learning algorithms are meticulously evaluated: Logistic Regression, Decision Tree, Random Forest, Naive Bayes, K-Nearest Neighbor, Support Vector Machine, XGBoost, AdaBoost, and Gradient Boosting. Through the analysis, encompassing accuracy, precision, recall, and F1 score metrics, the study offers insights into the strengths and limitations of each approach. The findings carry profound implications for online user safeguarding and cyberbullying prevalence reduction. Notably, Random Forest and XGBoost classifiers emerge as pioneers with accuracy rates of 93.34% and 93.32%, respectively. This comparative research underscores the pivotal role of expert algorithmic choices in addressing the urgency of cyberbullying and has the potential to be a valuable resource for academics and practitioners engaged in combatting this pressing societal issue.

Keywords

Cyberbullying, Machine Learning, Multi-Class Classification, Natural Language Processing (NLP).

Thanks

Makalenin revizyonunu geliştiren değerli öneri ve yorumları için hakemlere ve editörlere teşekkür ederim.

References

• [1] A. Saravanaraj, J. I. Sheeba, and S. P. Devaneyan, “Automatic Detection of Cyberbullying From Twitter,” IRACST-International Journal of Computer Science and Information Technology & Security (IJCSITS), vol. 6, no. 6, pp. 2249–9555, 2019, [Online]. Available: https://www.researchgate.net/publication/333320174.
• [2] W. N. H. W. Ali, M. Mohd, and F. Fauzi, “Cyberbullying detection: an overview,” in 2018 Cyber Resilience Conference (CRC), 2018, pp. 1–3.
• [3] J.-M. Xu, K.-S. Jun, X. Zhu, and A. Bellmore, “Learning from bullying traces in social media,” in Proceedings of the 2012 conference of the North American chapter of the association for computational linguistics: Human language technologies, 2012, pp. 656–666.
• [4] M. Dadvar, F. M. G. de Jong, R. Ordelman, and D. Trieschnigg, “Improved cyberbullying detection using gender information,” in Proceedings of the Twelfth Dutch-Belgian Information Retrieval Workshop (DIR 2012), 2012, pp. 23–25.
• [5] D. Jurafsky, Speech \& language processing. Pearson Education India, 2000.
• [6] T. P. Nagarhalli, V. Vaze, and N. K. Rana, “Impact of machine learning in natural language processing: A review,” in 2021 third international conference on intelligent communication technologies and virtual mobile networks (ICICV), 2021, pp. 1529–1534.
• [7] J. Cheng, C. Danescu-Niculescu-Mizil, and J. Leskovec, “Antisocial behavior in online discussion communities,” in Proceedings of the international aaai conference on web and social media, 2015, vol. 9, no. 1, pp. 61–70.
• [8] Z. Ghasem, I. Frommholz, and C. Maple, “Machine learning solutions for controlling cyberbullying and cyberstalking,” J Inf Secur Res, vol. 6, no. 2, pp. 55–64, 2015.
• [9] S. Murnion, W. J. Buchanan, A. Smales, and G. Russell, “Machine learning and semantic analysis of in-game chat for cyberbullying,” Computers \& Security, vol. 76, pp. 197–213, 2018.
• [10] K. Reynolds, A. Kontostathis, and L. Edwards, “Using machine learning to detect cyberbullying,” in 2011 10th International Conference on Machine learning and applications and workshops, 2011, vol. 2, pp. 241–244.
• [11] D. Van Bruwaene, Q. Huang, and D. Inkpen, “A multi-platform dataset for detecting cyberbullying in social media,” Language Resources and Evaluation, vol. 54, pp. 851–874, 2020.
• [12] J. Wang, R. J. Iannotti, and T. R. Nansel, “School bullying among adolescents in the United States: Physical, verbal, relational, and cyber,” Journal of Adolescent health, vol. 45, no. 4, pp. 368–375, 2009.
• [13] V. Balakrishnan, S. Khan, and H. R. Arabnia, “Improving cyberbullying detection using Twitter users’ psychological features and machine learning,” Computers \& Security, vol. 90, p. 101710, 2020.
• [14] J. Hani, N. Mohamed, M. Ahmed, Z. Emad, E. Amer, and M. Ammar, “Social media cyberbullying detection using machine learning,” International Journal of Advanced Computer Science and Applications, vol. 10, no. 5, 2019.
• [15] M. O. Raza, M. Memon, S. Bhatti, and R. Bux, “Detecting cyberbullying in social commentary using supervised machine learning,” in Advances in Information and Communication: Proceedings of the 2020 Future of Information and Communication Conference (FICC), Volume 2, 2020, pp. 621–630.
• [16] M. Sintaha and M. Mostakim, “An empirical study and analysis of the machine learning algorithms used in detecting cyberbullying in social media,” in 2018 21st International Conference of Computer and Information Technology (ICCIT), 2018, pp. 1–6.
• [17] B. R. Chakravarthi, “Hope speech detection in YouTube comments,” Social Network Analysis and Mining, vol. 12, no. 1, p. 75, 2022.
• [18] C. Iwendi, G. Srivastava, S. Khan, and P. K. R. Maddikunta, “Cyberbullying detection solutions based on deep learning architectures,” Multimedia Systems, vol. 29, no. 3, pp. 1839–1852, 2023, doi: 10.1007/s00530-020-00701-5.
• [19] Kaggle, “Cyberbullying Classification.” https://www.kaggle.com/datasets/andrewmvd/cyberbullying-classification (accessed Apr. 17, 2023).
• [20] J. Wang, K. Fu, and C.-T. Lu, “Sosnet: A graph convolutional network approach to fine-grained cyberbullying detection,” in 2020 IEEE International Conference on Big Data (Big Data), 2020, pp. 1699–1708.
• [21] S. Bird, “NLTK: the natural language toolkit,” in Proceedings of the COLING/ACL 2006 Interactive Presentation Sessions, 2006, pp. 69–72.
• [22] G. Grefenstette, “Tokenization,” in Syntactic wordclass tagging, Springer, 1999, pp. 117–133.
• [23] S. Sperandei, “Understanding logistic regression analysis,” Biochemia medica, vol. 24, no. 1, pp. 12–18, 2014.
• [24] J. Chen et al., “A comparison of linear regression, regularization, and machine learning algorithms to develop Europe-wide spatial models of fine particles and nitrogen dioxide,” Environment international, vol. 130, p. 104934, 2019.
• [25] D. Maulud and A. M. Abdulazeez, “A review on linear regression comprehensive in machine learning,” Journal of Applied Science and Technology Trends, vol. 1, no. 4, pp. 140–147, 2020.
• [26] S. R. Safavian and D. Landgrebe, “A survey of decision tree classifier methodology,” IEEE transactions on systems, man, and cybernetics, vol. 21, no. 3, pp. 660–674, 1991.
• [27] Y. K. Qawqzeh, M. M. Otoom, and F. Al-Fayez, “A Proposed Decision Tree Classifier for Atherosclerosis Prediction and Classification,” International Journal of Computer Science and Network Security (IJCSNS), vol. 19, no. 12, pp. 197–202, 2019.
• [28] L. Breiman, J. Friedman, C. Stone, and R. Olshen, “Classification and regression trees (crc, boca raton, fl),” 1984.
• [29] L. Breiman, “Random forests; uc berkeley tr567,” University of California: Berkeley, CA, USA, 1999.
• [30] L. Breiman, “Random Forests for Scientific Discovery,” Presentation, pp. 1–167, 2013, [Online]. Available: http://www.math.usu.edu/adele/RandomForests/ENAR.pdf.
• [31] J. R. Quinlan, C4. 5: programs for machine learning. Elsevier, 2014.
• [32] I. Rish and others, “An empirical study of the naive Bayes classifier,” in IJCAI 2001 workshop on empirical methods in artificial intelligence, 2001, vol. 3, no. 22, pp. 41–46.
• [33] E. Frank and R. R. Bouckaert, “Naive bayes for text classification with unbalanced classes,” in Knowledge Discovery in Databases: PKDD 2006: 10th European Conference on Principles and Practice of Knowledge Discovery in Databases Berlin, Germany, September 18-22, 2006 Proceedings 10, 2006, pp. 503–510.
• [34] Ö. Şahinaslan, H. Dalyan, and E. Şahinaslan, “Naive bayes s{\i}n{\i}fland{\i}r{\i}c{\i}s{\i} kullan{\i}larak youtube verileri üzerinden çok dilli duygu analizi,” Bili{\c{s}}im Teknolojileri Dergisi, vol. 15, no. 2, pp. 221–229, 2022.
• [35] Y. Wu, K. Ianakiev, and V. Govindaraju, “Improved k-nearest neighbor classification,” Pattern recognition, vol. 35, no. 10, pp. 2311–2318, 2002.
• [36] G. Guo, H. Wang, D. Bell, Y. Bi, and K. Greer, “KNN Model-Based Approach in Classification,” in On The Move to Meaningful Internet Systems 2003: CoopIS, DOA, and ODBASE, 2003, pp. 986–996.
• [37] T. Chen et al., “Xgboost: extreme gradient boosting,” R package version 0.4-2, vol. 1, no. 4, pp. 1–4, 2015.
• [38] T. Chen, T. He, M. Benesty, and V. Khotilovich, “Package ‘xgboost,’” R version, vol. 90, pp. 1–66, 2019.
• [39] W. S. Noble, “What is a support vector machine?,” Nature biotechnology, vol. 24, no. 12, pp. 1565–1567, 2006.
• [40] A. Vezhnevets and V. Vezhnevets, “Modest AdaBoost-teaching AdaBoost to generalize better,” in Graphicon, 2005, vol. 12, no. 5, pp. 987–997.
• [41] X. Li, L. Wang, and E. Sung, “AdaBoost with SVM-based component classifiers,” Engineering Applications of Artificial Intelligence, vol. 21, no. 5, pp. 785–795, 2008.
• [42] T.-K. An and M.-H. Kim, “A new diverse AdaBoost classifier,” in 2010 International conference on artificial intelligence and computational intelligence, 2010, vol. 1, pp. 359–363.
• [43] J. Son, I. Jung, K. Park, and B. Han, “Tracking-by-segmentation with online gradient boosting decision tree,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 3056–3064.
• [44] S. Peter, F. Diego, F. A. Hamprecht, and B. Nadler, “Cost efficient gradient boosting,” Advances in neural information processing systems, vol. 30, 2017.
• [45] M. Hossin and M. N. Sulaiman, “A review on evaluation metrics for data classification evaluations,” International journal of data mining \& knowledge management process, vol. 5, no. 2, p. 1, 2015.