Twitter Platformundan Elde Edilen Türkçe Saldırgan Dil Derlemi

Öz

Sosyal medya platformlarında kullanıcıların paylaşımlar arasında saldırgan dil barındıran içeriklerin önemli oranda arttığı gözlemlenmiştir. Çalışma Türkçe dilinde bu sorunun çözümüne katkı sağlamayı amaçlamaktadır. Bu çalışmada Twitter platformundan elde edilen bir veri seti oluşturulmuştur. 14752 Türkçe tweet metninden oluşan bu veri seti etiketleyiciler tarafından manuel olarak etiketlenmiştir. Buna ek olarak oluşturulan veri seti kullanılarak LSTM (Long ShortTerm Memory) ve GRU (Gated Recurrent Units) modellerinin sınıflandırma performansları karşılaştırılmıştır. Çalışmada ikili ve çoklu sınıflandırma yapılmıştır. Saldırgan dil ile ilgili Türkçe için çoklu sınıflandırma yapılan ilk çalışmadır. Bunlara ek olarak Twitter platformundan 1 milyon 860 bin tweet metninden oluşan genişletilmiş derlem elde edilmiştir. Burada word2vec yöntemi ile kelime temsilleri elde edilmiştir. Böylelikle genişletilmiş derlem kullanımının sınıflandırma performanslarına katkısı karşılaştırılmıştır. Çalışmada yapılan ikili sınıflandırma da genişletilmiş derlem kullanımıyla en yüksek performans GRU modeli F1-skor değeri %94,49’dur. Bu sebeple çoklu sınıflandırma yapılırken GRU modeli kullanılmıştır. Çoklu sınıflandırmada elde edilen sınıflandırma performans değerleri genişletilmiş derlemin katkısıyla GRU F1-makro değeri %71,97 ve %54,10’dur. Bu alanda Türk dili literatürüne katkı sağlamak amacıyla mevcut çalışmanın veri setleri ve genişletilmiş derlem kelime vektörleri paylaşılacaktır.

Anahtar Kelimeler

• Duygu Analizi
• Derin Öğrenme
• Saldırgan Dil
• Sosyal Medya Uygulaması
• Twitter

Kaynakça

1. [1] Cambria, E., Poria, S., Gelbukh, A. and Thelwall, M., “Sentiment Analysis Is a Big Suitcase”. IEEE Intelligent Systems, vol. 32, no. 6, pp. 74–80, 2017.
2. [2] Liu, B., “Sentiment analysis and opinion mining”. Synthesis lectures on human language technologies, vol. 5, no. 1, pp. 160-167, 2012.
3. [3] Craig, W., Boniel-Nissim, M., King, N., Walsh, S. D., Boer, M., Donnelly, P. D., ... and Van den Eijnden, R. “Social media use and cyber-bullying: a cross-national analysis of young people in 42 countries”, Journal of Adolescent Health, vol. 66 no. 6, pp. 100-108, 2020.
4. [4] Hinduja, S. and Patchin, J. W., “Bullying, cyberbullying and suicide”, Archiands of suicide research, vol. 14, no. 3, pp. 206-221, 2010. [5] Newberry, C. (2022). 36 Twitter Stats All Marketers Need to Know in 2021. https://blog.hootsuite.com/twitter-statistics/ (Access Date: April 12, 2022).
5. [6] Twitter, Rules Enforcement, https://transparency.twitter.com/en/reports/rules-enforcement.html#2020-jul-dec (Accces date: June 12, 2021).
6. [7] Oflazer, K., “Türkçe ve Doğal Dil İşleme”, Türkiye Bilişim Vakfı Bilgisayar Bilimleri ve Mühendisliği Dergisi, vol. 5, no. 2, 2016.
7. [8] Dehkharghani, R., Saygin, Y., Yanikoglu, B., & Oflazer, K., “SentiTurkNet: a Turkish polarity lexicon for sentiment analysis”, Language Resources and Evaluation, vol. 50, no. 3, pp. 667-685, 2016.
8. [9] Dehkharghani, R., Yanikoglu, B., Tapucu, D., & Saygin, Y., “Adaptation and use of subjectivity lexicons for domain dependent sentiment classification”, In 2012 IEEE 12th International Conference on Data Mining Workshops, pp. 669-673, 2012.

9. [10] Stamou, S., Ntoulas, A., Hoppenbrouwers, J., Saiz-Noeda, M., & Christodoulakis, D.,” Euroterm: Extending the eurowordnet with domain-specific terminology using an expand model approach”, In Proceedings of the 1st International Global Wordnet Conference,2002.
10. [11] Oflazer, K., “Two-level description of Turkish morphology”. Literary and linguistic computing”, vol. 9, no. 2, pp. 137-148, 1994.
11. [12] Hakkani-Tür, D. Z., Oflazer, K., & Tür, G., “Statistical morphological disambiguation for agglutinative languages”, Computers and the Humanities, vol. 36, no. 4, pp. 381-410, 2002.
12. [13] Oflazer, K., & Kuruoz, I., “Tagging and morphological disambiguation of Turkish text”, arXiv preprint cmp-lg/9407026, 1994. [14] Oflazer, K., Say, B., Hakkani-Tür, D. Z., & Tür, G., “Building a Turkish treebank”, In Treebanks, pp. 261-277, 2003.
13. [15] Özer Z., “The effect of normalization on the classification of traffic comments”, Doctorate Thesis, Karabük University, Computer Science Enstitute, Karabük, 15-23., 2019.
14. [16] Safaya, A., Kurtuluş, E., Göktoğan, A., & Yuret, D., “Mukayese: Turkish NLP Strikes Back”, Association for Computational Linguistics, pp. 846-863, 2022.
15. [17] Yilmaz, S. and Toklu, S., “A deep learning analysis on question classification task using Word2vec representations”, Neural Computing and Applications, vol. 32, no. 7, pp. 2909-2928, 2020.
16. [18] Zampieri M, Malmasi S, Nakov P., Rosenthal S., Farra N and Kumar R., “SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)”, In Proceedings of the 13th International Workshop on Semantic Evaluation, pp. 75–86, 2019.
17. [19] Wiegand, M., Siegel, M. and Ruppenhofer, J., “Oandrview of the GermEval 2018 shared task on the identification of offensiand language”, In Proceedings of the GermEval 2018 Workshop at Konandns 2018, pp. 1– 10, 2018.
18. [20] Sarmaşık, Ş. (2009). İşyerinde cinsel taciz algılaması ve yönetim ilişkilerine etkisi hakkında bir araştırma (Master's thesis), 2009.
19. [21] Reinsel, D., J. Gantz and J. Rydning (2018). Data Age 2025: The Evolution of Data to Life-Critical, www-content/our-story/trends/files/Seagate-WPDataAge2025-March-2017 (Access Date: August 12,2020)
20. [22] Mehl, M. R. and Pennebaker, J. W. (2003). The sounds of social life: A psychometric analysis of students’ daily social environments and natural conandrsations. Journal of personality and social psychology, 84 (4),857
21. [23] Wang, W., Chen, L., Thirunarayan, K. and Sheth, A. P., “Cursing in english on twitter”, In Proceedings of the 17th ACM conference on Computer supported cooperatiand work and social computing, pp. 415-425, 2014.
22. [24] Xu, J. M., Jun, K. S., Zhu, X., & Bellmore, A. “Learning from bullying traces in social media”, In Proceedings of the 2012 conference of the North American chapter of the association for computational linguistics: Human language technologies pp. 656-666, 2012.
23. [25] Çöltekin, Ç., “A corpus of Turkish offensiand language on social media”, In Proceedings of the 12th Language Resources and Evaluation Conference, 6174-6184, 2020.
24. [26] Basile, V., Bosco, C., Fersini, E., Debora, N., Patti, V., Pardo, F. M. R., ... and Sanguinetti, M., “Semeval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter”, In 13th International Workshop on Semantic Evaluation, pp. 54-63, 2019.
25. [27] Kwok, I., & Wang, Y., “Locate the hate: detecting tweets against blacks”, Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence, pp. 1621-1622, 2013.
26. [28] Ross, M. Rist, G. Carbonell, B. Cabrera, N. Kurowsky and M. Wojatzki. “Measuring the reliability of hate speech annotations: The case of the european refugee crisis”, In Proceedings of NLP4CMC III: 3rd Workshop on Natural Language Processing for Computer-Mediated Communication, vol. 17, no. 1, pp. 6–9, 2016.
27. [29] Jaki, S. and De Smedt, T., “Right-wing German hate speech on Twitter: Analysis and automatic detection”, arXiv:1910.07518, 2019.
28. [30] Burnap, P., Williams, M. L., Sloan, L., Rana, O., Housley, W., Edwards, A., ... and Voss, A., “Tweeting the terror: modelling the social media reaction to the Woolwich terrorist attack”, Social Network Analysis and Mining, vol. 4, no. 1, pp. 1-14, 2014.
29. [31] Dadvar, M., Trieschnigg, D., Ordelman, R. and de Jong, F., “Improving cyberbullying detection with user context” In European Conference on Information Retrieval, pp. 693–696., 2013.
30. [32] Dadvar, M., Trieschnigg, D. and de Jong, F., “Experts and machines against bullies: A hybrid approach to detect cyberbullies”, In Canadian Conference on Artificial Intelligence, pp. 275–281, 2014.
31. [33] Dinakar, K., Jones, B., Havasi, C., Lieberman, H. and Picard, R., “Common sense reasoning for detection, preandntion and mitigation of cyberbullying”, ACM Transactions on Interactiand Intelligent Systems (TiiS), vol. 2, no. 3, pp. 18, 2012.
32. [34] Nitta, T., Masui, F., Ptaszynski, M., Kimura, Y., Rzepka, R., & Araki, K., “Detecting cyberbullying entries on informal school websites based on category relevance maximization”, In Proceedings of the Sixth International Joint Conference on Natural Language Processing, pp. 579-586, 2013.
33. [35] American Psychological Association. (2004). “APA resolution on bullying among children and youth”, http://www.apa.org/about/goandrnance/ council/policy/bullying.pdf. (Access Date: June 20, 2020).
34. [36] Chen, Y., Zhou, Y., Zhu, S. and Xu, H., “Detecting offensiand language in social media to protect adolescent online safety”, In 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing, pp. 71–80, 2012
35. [37] Struß, J. M., Siegel, M., Ruppenhofer, J., Wiegand, M. and Klenner, M., “Oandrview of GermEval task 2, 2019 shared task on the identification of offensiand language”, In Preliminary proceedings of the 15th Conference on Natural Language Processing (KONANDNS 2019), 352– 363,2019.
36. [38] Waseem, Z., Davidson, T., Warmsley, D. and Weber, I.. Understanding abuse: A typology of Römer, U. (2009). The inseparability of lexis and grammar: Corpus linguistic perspectives. Annual Review of Cognitive Linguistics, 7(1), 140-162.abusiand language detection subtasks. In Proceedings of the First Workshop on Abusiand Language Online, 78–84, 2017.
37. [39] Ruppenhofer, J., Siegel, M. and Wiegand, M. “Guidelines for IGGSA shared task on the identification of offensiand language.” https://github.com/uds-lsv/GermEval-2018-Data/ blob/master/guidelines-iggsa-shared.pdf, 2018 (Access Date November 20, 2020).
38. [40] Zampieri M., Malmasi S., Nakov P., Rosenthal S., Atanasova P., Karadzhov G., Mubarak H., Derczynski L., Pitenis Z. and Çöltekin Ç., “SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)” In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pp. 1425–1447, 2020.
39. [41] Yılmaz, Ş. Ş., Özer, İ. ve Gökçen, H., “Türkçe Metinlerde Derin Öğrenme Yöntemleri Kullanılarak Duygu Analizi” In International Symposium of Scientific Research and Innovative Studies, 22, 971-982, 2021.
40. [42] Filatova, E., “Irony and Sarcasm: Corpus Generation and Analysis Using Crowdsourcing”, In Lrec, 392-398, 2012.
41. [43] Sharma, M., Kandasamy, I., & Kandasamy, V., “Deep Learning for predicting neutralities in Offensive Language Identification Dataset”, Expert Systems with Applications, pp. 185, 115458, 2021.
42. [44] Qiu, H., Zeng, Y., Zhang, T., Jiang, Y., & Qiu, M., “FenceBox: A Platform for Defeating Adversarial Examples with Data Augmentation Techniques”, ArXiv: 2012.01701, 2020.
43. [45] Bojanowski, P., Graand, E., Joulin, A. and Mikolov, T., “Enriching word andctors with subword information”, Transactions of the Association for Computational Linguistics, vol. 5, pp. 135-146, 2017.
44. [46] Ozer, I., Ozer, Z., & Findik, O. “Noise robust sound event classification with convolutional neural network”, Neurocomputing, 272, 505-512, 2018.
45. [47] Ozer, Z., Ozer, I., & Findik, O., “Diacritic restoration of Turkish tweets with word2vec”. Engineering Science and Technology, an International Journal, vol. 21, no. 6, pp. 1120-1127, 2018.
46. [48] Ozer, I., Ozer, Z., & Findik, O., “Lanczos kernel based spectrogram image features for sound classification”, Procedia computer science, vol. 111, pp.137-144,2017.
47. [49] Ligthart, A., Catal, C. and Tekinerdogan, B., “Systematic reviews in sentiment analysis: a tertiary study”, Artifitical Intelligence Revolution, vol. 54, pp. 4997–5053, 2021.
48. [50] Bowker, L. and Pearson, J. “Working with specialized language: A practical guide to using corpora”, New York: Routledge, pp. 15-29, 2002.
49. [51] Römer, U. “The inseparability of lexis and grammar: Corpus linguistic perspectives”, Annual Review of Cognitive Linguistics, vol. 7, no. 1, pp. 140-162, 2009.