Research Article
BibTex RIS Cite

Comparative Analysis of Transformers Based Architectures for Turkish Sentiment Classification

Year 2023, Volume: IDAP-2023 : International Artificial Intelligence and Data Processing Symposium Issue: IDAP-2023, 1 - 6, 18.10.2023
https://doi.org/10.53070/bbd.1350405

Abstract

Sentiment classification based on Transformers is a topic that has recently been widely studied in natural language processing and machine learning. There are many areas where it can be used, such as the interpretation and classification of emotional expressions in texts, social media analysis, market research, user experiences, etc. For this reason, this study aims to realize sentiment classification using Transformers-based architectures. In this study, 8 different BERTurk and 2 different ELECTRA variations were used for sentiment classification on the TRSAv1 dataset consisting of 150000 data. These models are pre-trained models to be used in sentiment classification studies on Turkish texts. The models were trained on the dataset using 3 different methods and the results were evaluated comparatively. As a result of the experiments, the sentiment classification performance of the models was measured using accuracy and F1-score metrics. The results of the experiments revealed the effectiveness of Transformers models in sentiment classification and the performance evaluations of the models used.

References

  • Kaynar, O., Görmez, Y., Yıldız, M., & Albayrak, A. (2016, September). Makine öğrenmesi yöntemleri ile Duygu Analizi. In International Artificial Intelligence and Data Processing Symposium (IDAP'16) (Vol. 17, No. 18, pp. 17-18).
  • Köksal, Ö. (2021, June). Enhancing Turkish sentiment analysis using pre-trained language models. In 2021 29th Signal Processing and Communications Applications Conference (SIU) (pp. 1-4). IEEE.
  • Adoma, A. F., Henry, N. M., & Chen, W. (2020, December). Comparative analyses of bert, roberta, distilbert, and xlnet for text-based emotion recognition. In 2020 17th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP) (pp. 117-121). IEEE.
  • Guven, Z. A. (2021, September). Comparison of BERT models and machine learning methods for sentiment analysis on Turkish tweets. In 2021 6th International Conference on Computer Science and Engineering (UBMK) (pp. 98-101). IEEE.
  • Acikalin, U. U., Bardak, B., & Kutlu, M. (2021). BERT modeli ile türkçe duygu analizi.
  • Aydoğan, (2022, February). TRSAv1: A new benchmark dataset for classifying user reviews on Turkish e-commerce websites. https://journals.sagepub.com/doi/abs/10.1177/01655515221074328 . Erişim Tarihi: 10 Haziran 2023
  • Çoban, Ö., Özyer, B., & Özyer, G. T. (2015, May). Sentiment analysis for Turkish Twitter feeds. In 2015 23nd Signal Processing and Communications Applications Conference (SIU) (pp. 2388-2391). IEEE.
  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30. DistilBERT . https://huggingface.co/docs/transformers/model_doc/distilbert . Erişim Tarihi: 17 Temmuz 2023
  • Savci, P., & Das, B. (2023). Comparison of pre-trained language models in terms of carbon emissions, time and accuracy in multi-label text classification using AutoML. Heliyon, 9(5).

Türkçe Duygu Sınıflandırma İçin Transformers Tabanlı Mimarilerin Karşılaştırılmalı Analizi

Year 2023, Volume: IDAP-2023 : International Artificial Intelligence and Data Processing Symposium Issue: IDAP-2023, 1 - 6, 18.10.2023
https://doi.org/10.53070/bbd.1350405

Abstract

Transformers tabanlı ile duygu sınıflandırma, son zamanlarda doğal dil işleme ve makine öğrenmesi alanında yaygın olarak çalışılan bir konudur. Metinler içerisinde karşılaşılan duygusal ifadelerin anlamlandırılması ve sınıflandırılması, sosyal medya analizi, piyasa araştırması, kullanıcı deneyimleri vb. gibi kullanılabileceği birçok alan mevcuttur. Bu sebeple, bu çalışmada Transformers tabanlı mimariler kullanılarak duygu sınıflandırmasının gerçekleştirilmesi hedeflenmiştir. Bu çalışmada, 150000 veriden oluşan TRSAv1 veriseti üzerinde, 8 farklı BERTurk ve 2 farklı ELECTRA varyasyonu üzerinde duygu sınıflandırma işlemi için kullanılmıştır. Bu modeller, Türkçe metinler üzerinde duygu sınıflandırılması çalışmalarında kullanılmak için önceden eğitilmiş modellerdir. Veri seti üzerinde 3 farklı metot kullanılarak modeller eğitilmiş ve sonuçlar karşılaştırmalı olarak değerlendirilmiştir. Yapılan deneyler sonucunda, modellerin duygu sınıflandırma performansları doğruluk ve F1-skor metrikleri kullanılarak ölçülmüştür. Deney sonuçları, Transformers modellerinin duygu sınıflandırması konusundaki etkinliğini ve kullanılan modellerin performans değerlendirmelerini ortaya koymuştur.

References

  • Kaynar, O., Görmez, Y., Yıldız, M., & Albayrak, A. (2016, September). Makine öğrenmesi yöntemleri ile Duygu Analizi. In International Artificial Intelligence and Data Processing Symposium (IDAP'16) (Vol. 17, No. 18, pp. 17-18).
  • Köksal, Ö. (2021, June). Enhancing Turkish sentiment analysis using pre-trained language models. In 2021 29th Signal Processing and Communications Applications Conference (SIU) (pp. 1-4). IEEE.
  • Adoma, A. F., Henry, N. M., & Chen, W. (2020, December). Comparative analyses of bert, roberta, distilbert, and xlnet for text-based emotion recognition. In 2020 17th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP) (pp. 117-121). IEEE.
  • Guven, Z. A. (2021, September). Comparison of BERT models and machine learning methods for sentiment analysis on Turkish tweets. In 2021 6th International Conference on Computer Science and Engineering (UBMK) (pp. 98-101). IEEE.
  • Acikalin, U. U., Bardak, B., & Kutlu, M. (2021). BERT modeli ile türkçe duygu analizi.
  • Aydoğan, (2022, February). TRSAv1: A new benchmark dataset for classifying user reviews on Turkish e-commerce websites. https://journals.sagepub.com/doi/abs/10.1177/01655515221074328 . Erişim Tarihi: 10 Haziran 2023
  • Çoban, Ö., Özyer, B., & Özyer, G. T. (2015, May). Sentiment analysis for Turkish Twitter feeds. In 2015 23nd Signal Processing and Communications Applications Conference (SIU) (pp. 2388-2391). IEEE.
  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30. DistilBERT . https://huggingface.co/docs/transformers/model_doc/distilbert . Erişim Tarihi: 17 Temmuz 2023
  • Savci, P., & Das, B. (2023). Comparison of pre-trained language models in terms of carbon emissions, time and accuracy in multi-label text classification using AutoML. Heliyon, 9(5).
There are 9 citations in total.

Details

Primary Language Turkish
Subjects Deep Learning, Natural Language Processing
Journal Section PAPERS
Authors

Mehmet Arzu 0000-0001-6610-2788

Murat Aydoğan 0000-0002-6876-6454

Publication Date October 18, 2023
Submission Date August 26, 2023
Acceptance Date August 26, 2023
Published in Issue Year 2023 Volume: IDAP-2023 : International Artificial Intelligence and Data Processing Symposium Issue: IDAP-2023

Cite

APA Arzu, M., & Aydoğan, M. (2023). Türkçe Duygu Sınıflandırma İçin Transformers Tabanlı Mimarilerin Karşılaştırılmalı Analizi. Computer Science, IDAP-2023 : International Artificial Intelligence and Data Processing Symposium(IDAP-2023), 1-6. https://doi.org/10.53070/bbd.1350405

The Creative Commons Attribution 4.0 International License 88x31.png is applied to all research papers published by JCS and

A Digital Object Identifier (DOI) Logo_TM.png is assigned for each published paper