Türkçe Hakaret ve Nefret Söylemi Otomatik Tespit Modeli

Year 2023, Volume: 6 Issue: 1, 61 - 73, 30.06.2023

Mehmet Salih Kurt , Eylem Yücel Demirel

Abstract

İnsanların çevrimiçi dünyada, özellikle sosyal medya platformlarında iletişim kurmasıyla birlikte, kullanıcılar tarafından oluşturulan içeriklerin internet üzerindeki miktarı artmıştır. Bu platformların anonim yapısı nedeniyle, kullanıcılar hakaret ve nefret içeren düşünceleri paylaşabilmektedir. Bu istenmeyen içerikler, hem bireyler hem de toplumlar üzerinde olumsuz etkilere neden olabilir. Bu nedenle, hakaret ve nefret içeren içeriklerin tespit edilmesi ve filtrelenmesi önemlidir. Bu tür içeriklerin manuel olarak tespit edilmesi zordur, bu yüzden otomatik yöntemlere ihtiyaç duyulmaktadır. Son yıllarda, çevrimiçi hakaret ve nefret söylemlerinin tespitiyle ilgili akademik araştırmalarda artış görülmektedir. BERT gibi transfer öğrenme modelleriyle İngilizce hakaret ve nefret söylemlerinin otomatik tespiti konusunda umut verici sonuçlar elde edilmiştir. Ancak, Türkçe gibi sınırlı kaynaklara sahip dillerde hakaret ve nefret söyleminin otomatik tespiti üzerine yapılan araştırma sayısı oldukça azdır.
Bu çalışmada, Türkçe dili için hakaret ve nefret söylemi otomatik tespit sistemi geliştirme çabalarının sonuçları paylaşılmıştır. İlk olarak, Türkçe veri seti oluşturmak için otomatik etiketleme yöntemi önerilmiş ve bu yöntemle Türkçe hakaret ve nefret söylemi veri seti oluşturulmuştur. Doğal dil işleme alanında en iyi sonuçlar veren BERT modelinin farklı varyantları ve çeşitli Türkçe hakaret ve nefret söylemi veri setleri kullanılarak deneyler gerçekleştirilmiştir. Yapılan deneyler sonucunda, en iyi performansa sahip olan XLM-RoBERTa modeli için hiperparametre optimizasyonu yapılmış ve en kapsamlı veri setleri kullanılarak nihai Türkçe hakaret ve nefret söylemi otomatik tespit sistemi oluşturulmuştur. Oluşturulan Türkçe hakaret ve nefret söylemi otomatik tespit modeli, diğer çalışmalarla aynı test veri setini kullanarak karşılaştırılmıştır.

Keywords

Hakaret Söylemi, Nefret Söylemi, BERT, Doğal Dil İşleme

Offensıve Language And Hate Speech Detectıon In Turkısh

Abstract

As a result of people communicating online, especially on social media platforms, the amount of user-generated content on the internet has increased. Due to the anonymous nature of these platforms, users can share content containing offensive language and hate speech. Such undesirable content can have negative effects on both individuals and societies. Therefore, it is important to detect and filter content that contains offensive language and hate speech. Detecting such content manually is challenging, which is why there is a need for automated methods. In recent years, there has been an increase in academic research on the detection of online offensive language and hate speech. Promising results have been achieved in the automatic detection of offensive language and hate speech in English using transfer learning models such as BERT. However, the number of studies on automatic detection of offensive language and hate speech in languages with limited resources such as Turkish is quite limited.
This study presents the results of efforts to develop an automatic detection system for offensive language and hate speech in the Turkish language. Firstly, an automatic labeling method was proposed to create a Turkish dataset, and using this method, a Turkish dataset for hate speech and offensive language was created. Experiments were conducted using various variants of the BERT model, which is considered state-of-the-art in natural language processing, along with various Turkish datasets related to offensive language and hate speech. Through these experiments, the XLM-RoBERTa model, which achieved the best performance, underwent hyperparameter optimization. Subsequently, using the most comprehensive datasets available, the final Turkish automatic detection system for offensive language and hate speech was developed. The developed Turkish automatic detection model for offensive language and hate speech was compared with other studies using the same test dataset.

Keywords

Offensive Language, Hate Speech, BERT, Natural Language Processing

References

• Sap, M., Card, D., Gabriel, S., Choi, Y., A, N., 2019. The risk of racial bias in hate speech detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , 1668– 1678.
• Mathew, B., Dutt, R., Goyal, P., Mukherjee, A., 2019. Spread of hate speech in online social media. In Proceedings of WebSci. ACM.
• Das, M., Mathew, B., Saha, P., Goyal, P., Mukherjee, A., 2020. Hate speech in online social media. ACM SIGWEB Newsletter, (Autumn) , 1–8.
• Rizwan, H., Shakeel, M.H., Karim, A., 2020. Hatespeech and offensive language detection in roman urdu. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP) , 2512–2522.
• https://www.theguardian.com/world/2018/mar/14/facebook-accused-by-sri-lanka-of-failing-tocontrol-hate-speech. (15.07.2022)
• https://www.reuters.com/investigates/specialreport/myanmar-facebook-hate. (15.07.2022)
• https://money.cnn.com/2017/06/01/technology/twitter-facebook-hate-speech-europe/index.html (15.07.2022)
• https://help.twitter.com/tr/rules-andpolicies/hateful-conduct-policy. (17.07.2022)
• https://www.theverge.com/2019/12/16/21021005/google-youtube-moderators-ptsd-accentureviolent-disturbing-content-interviews-video(17.07.2022)
• Wiedemann, G., Ruppert, E., Jindal, R., Biemann, C., 2018. Transfer learning from lda to bilstm-cnn for offensive language detection in twitter. Proceedings of GermEval 2018, 14th Conference on Natural Language Processing (KONVENS 2018) , 85–94.
• Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., Chang, Y., 2016. Abusive language detection in online user content. In Proceedings of the 25th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee , 145–153.
• Davidson, T., Warmsley, D., Macy, M., Weber, I., 2017. Automated hate speech detection and the problem of offensive language. The 11th International AAAI Conference on Web and Social Media , 6–7.
• Waseem, Z., Hovy, D., 2016. Hateful symbols or hateful people? predictive features for hate speech detection on twitter. In Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , 88–93.
• Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., Kumar, R., 2019a. Predicting the type and target of offensive posts in social media. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.
• Çağrı Çöltekin, 2020. A corpus of turkish offensive language on social media. Proceedings of the 12th Language Resources and Evaluation Conference, 6174–6184.
• Zampieri, M., Nakov, P., Rosenthal, S., Atanasova, P., Karadzhov, G., Mubarak, H., Derczynski, L., Pitenis, Z., Çağrı Çöltekin, 2020. Semeval-2020 task 12: Multilingual offensive language identification in social media (offenseval 2020). In Proceedings of SemEval
• Şahi, H., Kılıç, Y., Sağlam, R.B., 2018. Automated detection of hate speech towards woman on twitter. 3rd International Conference
• Mayda, I., Diri, B., Dalyan, T., 2021. Türkçe tweetler üzerinde makine öğrenmesi ile nefret söylemi tespiti. Avrupa Bilim ve Teknoloji Dergisi , 328–334.
• Sezer, T., 2020. Twitter derlemi – ts tweets corpus. URL: https://tanersezer.com/?p=155. (14.09.2020)
• Kemik, 2020. Kemik doğal dil İşleme grubu. URL: http://www.kemik.yildiz.edu.tr/verikumelerimiz.html. (10.09.2020)
• https://github.com/drsalihkurt/HateSpeechandOffensiveLanguageDetectionInTurkish. (15.05.2023)
• Devlin, J., Chang, M.W., Lee, K., Toutanova, K., 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXivpreprint arXiv:1810.04805.
• Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., Zettlemoyer, L., 2018. Deep contextualized word representations. arxiv preprintarxiv:1802.05365. Structure.
• Howard, J., Ruder, S., 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics5 , 328–339.
• Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., 2018. Improving language understanding by generative pre-training. Technical Report. OpenAI.
• Sanh, V., Debut, L., Chaumond, J., Wolf, T., 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108.
• Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V., 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692
• Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzman, F., Grave, E., Ott, M., Zettlemoyer, L., Stoyanov, V., 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL) .