TRANSFORMATÖR-TABANLI EVRİŞİMLİ SİNİR AĞI MODELİ KULLANARAK TWITTER VERİSİNDE SALDIRGANLIK TESPİTİ

Abstract

Çevrimiçi ortamlar, insanların sosyal etkileşimlerinde anti-sosyal davranışların artmasını kolaylaştırmaktadır. Sosyal medya kullanımının yaygınlaşmasıyla özellikle son yıllarda nefret söylemleri, siber zorbalık ve trolleme gibi davranışlar önemli ölçüde artmıştır. Saldırgan ve nefret içerikli söylemlerin tespiti siber zorbalıkların azaltılması ve engellenmesinde önemli bir adımdır. Siber zorbalık, sosyal medya üzerinden nefret dolu, saldırgan, kaba, aşağılayıcı ve alaycı ifadeler kullanarak diğer bireylere zarar vermek adına yapılan yorumlar olarak adlandırılmaktadır. Hızla büyüyen verilerin varlığı, bunun insan denetimiyle gerçekleştirilmeye çalışılması yavaş ve pahalı olduğundan saldırganlığın otomatik tespitiyle siber zorbalığın durdurulması sağlanabilir. Bu çalışmada Twitter veri seti olan Cyber-Trolls üzerinden saldırganlık tespitini otomatik olarak belirlenmesi ele alınmaktadır. LMTweets adında bir kodlayıcı, veri kümesinin özelliklerinin çıkarılması için 20001 adet tweet üzerinden eğitilmiştir. Çıkarılan öznitelikler, metni saldırgan / saldırgan olmayan olarak sınıflandırmak üzere evrişim sinir ağı modeline girdi olarak verilir. Ayrıca Naïve Bayes, Destek Vektör Makinesi, K-En Yakın Komşu, olmak üzere üç sınıflandırma algoritması uygulanmıştır. Bunun yanında, Evrişimli Sinir Ağı, Uzun Kısa-Süreli Bellek ve Kapılı Tekrarlayan Birim üç öğrenme algoritması ile birlikte BERT, XLNet ve ULMFIT olmak üzere üç transformatör modeli uygulanmıştır. Önerilen modelde Python, Keras API ve Tensorflow birlikte kullanılmıştır. Deneysel sonuçlarda elde edilen performans parametreleri doğruluk, kesinlik, duyarlılık, F1-ölçütü ve AUC olarak belirlenmiş ve LMTweets + CNN modelinin kullanılan tüm modeller arasında daha iyi performans gösterdiği ortaya konmuştur.

Keywords

• Saldırganlık
• CNN
• Derin öğrenme
• Twitter
• Transformatör modeller

Aggression Detection in Twitter Data Using Transformer‑Based Convolutional Neural Network Model

Abstract

Online environments facilitate the increase of anti-social behaviors in people's social interactions. Behaviors such as hate speech, cyberbullying, and trolling have increased significantly, especially in recent years, with the widespread use of social media. Detection of aggression and hateful speech is an important step in reducing and preventing cyberbullying. Cyberbullying is defined as comments made on social media to harm other individuals by using hateful, offensive, rude, humiliating, and sarcastic expressions. It is slow and expensive to try to achieve this with human control with the existence of rapidly growing data, so cyberbullying can be stopped by automatic detection of aggression. In this study, the automatic determination of aggression detection via Cyber-Trolls, which is the Twitter dataset, is discussed. A coder named LMTweets was trained on 20001 tweets to extract the features of the dataset. The extracted features are given as input to the convolutional neural network model to classify the text as aggressive / non-aggressive. In addition, three classification algorithms, namely Naïve Bayes, Support Vector Machine, K-Nearest Neighbors, were applied. In addition, three transformer models, BERT, XLNet, and ULMFIT were applied along with the Convolutional Neural Network, Long Short-Term Memory, and Gated Recurrent Unit three learning algorithms. Python, Keras API, and Tensorflow are used together in the proposed model. The performance parameters obtained in the experimental results were determined as accuracy, precision, recall, F1-score, and AUC, and it was revealed that the LMTweets + CNN model performed better among all the models used.

Keywords

• Aggression
• CNN
• Deep learning
• Twitter
• Transformer models

References

1. Abulaish, M., Kamal, A., Zaki, M., 2020, "A survey of figurative language and its computational detection in online social networks." 14(1): 1-52.
2. Al-Garadi, M. A., Varathan, K. D., Ravana, S. D. J. C. i. H. B., 2016, "Cybercrime detection in online communications: The experimental case of cyberbullying detection in the Twitter network." 63: 433-443.
3. Aroyehun, S. T., & Gelbukh, A., 2018. "Aggression detection in social media: Using deep neural networks, data augmentation, and pseudo labeling. In Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying" (TRAC-2018) (pp. 90-97).
4. Avvaru, A., Vobilisetty, S., & Mamidi, R., 2020, "Detecting sarcasm in conversation context using transformer-based models. In Proceedings of the second workshop on figurative language processing" (pp. 98-103).
5. Balakrishnan, V., Khan, S., Fernandez, T., Arabnia, H. R. J. P., 2019, "Cyberbullying detection on twitter using Big Five and Dark Triad features." 141: 252-257.
6. Bansal, A., Sharma, S. M., Kumar, K., Aggarwal, A., Goyal, S., Choudhary, K., 2012, "Classification of flames in computer mediated communications."
7. Basile, V., Bosco, C., Fersini, E., Debora, N., Patti, V., Pardo, F. M. R., 2019, “Semeval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter.” 13th International Workshop on Semantic Evaluation, Association for Computational Linguistics.
8. Chatzakou, D., Kourtellis, N., Blackburn, J., De Cristofaro, E., Stringhini, G., & Vakali, A., 2017, “Mean birds: Detecting aggression and bullying on twitter.” Proceedings of the 2017 ACM on web science conference.

9. Chavan, V. S., & Shylaja, S., 2015, “Machine learning approach for detection of cyber-aggressive comments by peers on social media network.” 2015 International Conference on Advances in Computing, Communications and Informatics (ICACCI), IEEE.
10. Chia, Z. L., Ptaszynski, M., & Masui, F., 2019, “Exploring machine learning techniques for irony detection.” Proceedings of the Annual Conference of JSAI 33rd Annual Conference, 2019, The Japanese Society for Artificial Intelligence.
11. Davidson, T., Warmsley, D., Macy, M., & Weber, I., 2017, “Automated hate speech detection and the problem of offensive language.” Proceedings of the International AAAI Conference on Web and Social Media.
12. Dinakar, K., Jones, B., Havasi, C., Lieberman, H., & Picard, R. J. A. T., 2012, "Common sense reasoning for detection, prevention, and mitigation of cyberbullying." 2(3): 1-30.
13. Djuric, N., Zhou, J., Morris, R., Grbovic, M., Radosavljevic, V., & Bhamidipati, N., 2015, “Hate speech detection with comment embeddings.” Proceedings of the 24th international conference on world wide web.
14. Farías, D. I. H., Montes-y-Gómez, M., Escalante, H. J., Rosso, P., & Patti, V., 2018, “A knowledge-based weighted KNN for detecting Irony in Twitter.” Mexican International Conference on Artificial Intelligence, Springer.
15. Gambäck, B., & Sikdar, U. K., 2017, “Using convolutional neural networks to classify hate-speech.” Proceedings of the first workshop on abusive language online.
16. Greevy, E., & Smeaton, A. F., 2004, “Classifying racist texts using a support vector machine.” Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval.
17. Gregory, H., Li, S., Mohammadi, P., Tarn, N., Draelos, R., & Rudin, C., 2020, “A Transformer approach to contextual Sarcasm detection in Twitter.” Proceedings of the Second Workshop on Figurative Language Processing.
18. Grigg, D. W., 2010, “Cyber-aggression: Definition and concept of cyberbullying.” Journal of Psychologists and Counsellors in Schools, 20(2), 143-156.
19. Hepburn, A. D., 1875, Manual of English Rhetoric, American Book Company.
20. Jianqiang, Z., & Xiaolin, G. J. I. A., 2017, "Comparison research on text pre-processing methods on twitter sentiment analysis." 5: 2870-2879.
21. Joachims, T., 1998, “Text categorization with support vector machines: Learning with many relevant features.” European conference on machine learning, Springer.
22. John, T. N., 2000, “Hate Speech.” In Encyclopedia of the American Constitution (2nd ed.,edited by Leonard, W. L., Kenneth, L. K. et al., New York: Macmillan), pp. 1277-1279.
23. Joshi, A., Bhattacharyya, P., & Carman, M. J. J. A. C. S., 2017, "Automatic sarcasm detection: A survey." 50(5): 1-22.
24. Khan, U., Khan, S., Rizwan, A., Atteia, G., Jamjoom, M. M., & Samee, N. A. 2022. “Aggression Detection in Social Media from Textual Data Using Deep Learning Models.” Applied Sciences, 12(10), 5083.
25. Kumar, A., Sangwan, S. R., Arora, A., Nayyar, A., & Abdel-Basset, M. J. I. a., 2019, "Sarcasm detection using soft attention-based bidirectional long short-term memory model with convolution network." 7: 23319-23328.
26. Kumar, R., Ojha, A. K., Malmasi, S., & Zampieri, M., 2018, “Benchmarking aggression identification in social media.” Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC-2018).
27. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., 2019, "Roberta: A robustly optimized bert pretraining approach."
28. Madisetty, S., & Desarkar, M. S., 2018, “Aggression detection in social media using deep neural networks.” Proceedings of the first workshop on trolling, aggression and cyberbullying (TRAC-2018).
29. Maslej-Krešňáková, V., Sarnovský, M., Butka, P., & Machová, K. J. A. S., 2020, "Comparison of deep learning models and various text pre-processing techniques for the toxic comments classification." 10(23): 8631.
30. Mihaylov, T., Georgiev, G., & Nakov, P., 2015, “Finding opinion manipulation trolls in news community forums.” Proceedings of the nineteenth conference on computational natural language learning.
31. Mubarak, H., Darwish, K., & Magdy, W., 2017, “Abusive language detection on Arabic social media.” Proceedings of the first workshop on abusive language online.
32. Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., & Chang, Y., 2016, “Abusive language detection in online user content.” Proceedings of the 25th international conference on world wide web.
33. Pareek, K., Choudhary, A., Tripathi, A., Mishra, K. K., & Mittal, N. 2022. “Hate and Aggression Detection in Social Media Over Hindi English Language.” International Journal of Software Science and Computational Intelligence (IJSSCI), 14(1), 1-20.
34. Potamias, R.-A., Siolas, G., & Stafylopatis, A., 2019, “A robust deep ensemble classifier for figurative language detection.” International Conference on Engineering Applications of Neural Networks, Springer.
35. Potamias, R. A., Siolas, G., Stafylopatis, A.-G. J. N. C., 2020, "A transformer-based approach to irony and sarcasm detection." 32(23): 17309-17320.
36. Prentice, S., Taylor, P. J., Rayson, P., Hoskins, A., & O’Loughlin, B. J. I. S. F., 2011, "Analyzing the semantic content and persuasive composition of extremist media: A case study of texts produced during the Gaza conflict." 13(1): 61-73.
37. Risch, J., & Krestel, R., 2018, “Aggression identification using deep learning and data augmentation.” Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC-2018).
38. Sadiq, S., Mehmood, A., Ullah, S., Ahmad, M., Choi, G. S., & On, B.-W. J. F. G. C. S., 2021, "Aggression detection through deep neural model on twitter." 114: 120-129.
39. Salawu, S., He, Y., & Lumsden, J. J. I. T. o. A. C., 2017, "Approaches to automated detection of cyberbullying: A survey." 11(1): 3-24.
40. Saravanaraj, A., Sheeba, J., Devaneyan, S. P. J. I. J. o. C. S., 2016, "Automatic detection of cyberbullying from twitter."
41. Sarsam, S. M., Al-Samarraie, H., Alzahrani, A. I., & Wright, B. J. I. J. o. M. R., 2020, "Sarcasm detection using machine learning algorithms in Twitter: A systematic review." 62(5): 578-598.
42. Schmidt, A., & Wiegand, M., 2017, “A survey on hate speech detection using natural language processing.” Proceedings of the fifth international workshop on natural language processing for social media.
43. Sharif, O., & Hoque, M. M. 2022. “Tackling cyber-aggression: Identification and fine-grained categorization of aggressive texts on social media using weighted ensemble of transformers.” Neurocomputing, 490, 462-481.
44. Shen, Y., He, X., Gao, J., Deng, L., & Mesnil, G., 2014, “Learning semantic representations using convolutional neural networks for web search.” Proceedings of the 23rd international conference on world wide web.
45. Singh, V., Varshney, A., Akhtar, S. S., Vijay, D., & Shrivastava, M., 2018, “Aggression detection on social media text using deep neural networks.” Proceedings of the 2nd Workshop on Abusive Language Online (ALW2).
46. Smit, D. J. S. A. J. o. E., 2015, "Cyberbullying in South African and American schools: A legal comparative study." 35(2): 1-11.
47. Su, H.-P., Huang, Z.-J., Chang, H.-T., & Lin, C.-J., 2017, “Rephrasing profanity in chinese text.” Proceedings of the First Workshop on Abusive Language Online.
48. Tai, K. S., Socher, R., & Manning, C. D. J. a. p. a., 2015, "Improved semantic representations from tree-structured long short-term memory networks."
49. Tulkens, S., Hilte, L., Lodewyckx, E., Verhoeven, B., & Daelemans, W. J. a. p. a., 2016, "A dictionary-based approach to racism detection in dutch social media."
50. Van der Walt, E., Eloff, J. H., Grobler, J. J. C., 2018, "Cyber-security: Identity deception detection on social media platforms." 78: 76-89.
51. Xiao, Y. and Cho, K. J. a. p. a., 2016, "Efficient character-level document classification by combining convolution and recurrent layers."