BERT Modeli'nin Sınıflandırma Doğruluğunun Sıfır-Atış Öğrenmesi ile Artırılması

Year 2021, Volume: 14 Issue: 2, 99 - 108, 22.12.2021

Şükrü OZAN Umut ÖZDİL Davut Emre TAŞAR Büşra ARSLAN Gökçe POLAT

https://doi.org/10.54525/tbbmd.1004781

Abstract

Bu çalışmada, çevrimiçi reklam platformlarında oluşturulan reklam metinlerinin sektöre göre otomatik olarak sınıflandırılması için Çift Yönlü Kodlayıcı Gösterimleri (BERT) modeli kullanılmıştır. Eğitim veri setimiz 44 farklı sektöre ait yaklaşık 101.000 adet, ilgili sektör isimleri ile etiketlenmiş reklam metninden oluşmaktadır. Sınıflandırma başarımının belirli sektörler için düşük olduğu gözlemlenmiştir. Bunun, ilgili eğitim veri setindeki aykırı bazı reklam metinlerinden kaynaklandığı gözlemlenmiştir. Bu sebeple sektör ve reklam metni uygunluğunu otomatik olarak tespit etmek ve veri seti için bir ön işleme gerçekleştirmek amacıyla doğal dil işleme (NLP) alanında güncel bir yöntem olan Sıfır-Atış Öğrenmesi yöntemi kullanılmıştır. Bu ön işleme çalışmasından sonra temizlenen veri seti ile tekrardan eğitilen BERT modelinin sınıflandırma başarımının önemli ölçüde arttığı gözlemlenmiştir.

Keywords

Dijital Pazarlama, Reklam Metni, NLP, Metin Sınıflandırma, BERT, Sıfır-Atış Öğrenmesi

References

• Emarketer,, “Advertisers will spend nearly $600 billion worldwide in 2015,” 2014.
• Google., “1.1 Çevrimici reklamcılık ve Google Ads’in avantajları - Google Ads Yardım,” https://support.google.com/google-ads/answer/6123875?hl=tr, May 2021.
• Z, A. and Adali, E., “Opinion mining and sentiment analysis for contex-tual online-advertisement,” in 2016 IEEE 10th International Conferenceon Application of Information and Communication Technologies (AICT). IEEE, 2016, pp. 1–3.
• Google., “Reklam kalitesi hakkında - Google Ads Yardım,” https://support.google.com/google-ads/answer/156066?hl=tr&reftopic=10549746, May2021.
• Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K., “Bert: Pre-trainingof deep bidirectional transformers for language understanding,” arXivpreprint arXiv:1810.04805, 2018.
• Ozan, S ̧ . and Tas ̧ar, D. E.,“Auto-tagging of short conversational sentences using natural language processing methods,” in 2021 29th Signal Processing and Communications Applications Conference (SIU), 2021, pp. 1–4
• Zulfat Miftahutdinov, Ilseyar Alimova, E. T., “Kfu nlp team at smm4h2019 tasks: Want to extract adverse drugs reactions from tweets? bertto the rescue.,” ACL 2019, pp. 52–57, 2019
• Rønningstad, E., “Targeted sentiment analysis for norwegian text,” 2020.
• Hochreiter, S. and Schmidhuber, J., “Long short-term memory,”Neuralcomputation, vol. 9, no. 8, pp. 1735–1780, 1997.
• [Chen, P., Sun, Z., Bing, L., and Yang, W., “Recurrent attention networkon memory for aspect sentiment analysis,” in Proceedings of the 2017conference on empirical methods in natural language processing, 2017,pp. 452–461.
• Dong, L., Wei, F., Tan, C., Tang, D., Zhou, M., and Xu, K., “Adaptiverecursive neural network for target-dependent twitter sentiment classifi-cation,” in Proceedings of the 52nd annual meeting of the associationfor computational linguistics (volume 2: Short papers), 2014, pp. 49–54.
• Xue, W. and Li, T.,“Aspect based sentiment analysis with gatedconvolutional networks,” arXiv preprint arXiv:1805.07043, 2018.
• Huang, B. and Carley, K. M.,“Parameterized convolutional neural networks for aspect level sentiment classification,” arXiv preprintarXiv:1909.06276, 2019.
• Nergız, G., Safali, Y., Avaroğlu, E., and Erdoğan, S., “Classification ofturkish news content by deep learning based lstm using fasttext model,” in 2019 International Artificial Intelligence and Data Processing Sym-posium (IDAP). IEEE, 2019, pp. 1–6.
• Dogru, H. B., Tilki, S., Jamil, A., and Hameed, A. A., “Deep learning-based classification of news texts using doc2vec model,” in2021 1st International Conference on Artificial Intelligence and Data Analytics(CAIDA). IEEE, 2021, pp. 91–96.
• Gonz ́alez-Carvajal, S. and Garrido-Merch ́an, E. C., “Comparing bertagainst traditional machine learning text classification,” arXiv preprintarXiv:2005.13012, 2020.
• Gao, Z., Feng, A., Song, X., and Wu, X., “Target-dependent sentimentclassification with bert,” IEEE Access, vol. 7, pp. 154290–154299, 2019.
• Ozdil, U., Arslan, B., Taşar, D. E., Polat, G., and Ozan, Ş., “Adtext classification with bidirectional encoder representations,” in 2021 6th International Conference on Computer Science and Engineering (UBMK), 2021, pp. 1–6.
• Chang, M., Ratinov, L., Roth, D., and Srikumar, V., “Importance of semantic representation: Dataless classification,” in AAAI-08/IAAI-08 Proceedings - 23rd AAAI Conference on Artificial Intelligence and the 20th Innovative Applications of Artificial Intelligence Conference, 2008, vol. 2, pp. 830–835, 23rd AAAI Conference on Artificial Intelligence and the 20th Innovative Applications of Artificial Intelligence Conference, AAAI-08/IAAI-08 ; Conference date: 13-07-2008 Through 17-07-2008.
• Pushp, P. K. and Srivastava, M. M., “Train once, test anywhere: Zero-shot learning for text classification,” CoRR, vol. Abs/1712.05972, 2017.
• Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., and Dean, J., “Distributed representations of words and phrases and their compositionality,” in Advances in Neural Information Processing Systems, Burges, C. J. C., Bottou, L., Welling, M., Ghahramani, Z., and Weinberger, K. Q., Eds. 2013, vol. 26, Curran Associates, Inc.
• Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez,A. N., Kaiser, L., and Polosukhin, I., “Attention is all you need,” arXivpreprint arXiv:1706.03762, 2017.
• Gupta, Shashij, et al. "Machine translation testing via pathological invariance." Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2020.
• Do, Quang-Minh, Kungan Zeng, and Incheon Paik. "Resolving Lexical Ambiguity in English-Japanese Neural Machine Translation." 2020 3rd Artificial Intelligence and Cloud Computing Conference. 2020.
• Loodos., “loodos/bert-base-turkish-uncased · hugging face,” https://github.com/Loodos/turkish-language-models, Aug. 2020.
• Taşar, D. Emre, et al. "Auto-tagging of Short Conversational Sentences using Transformer Methods." arXiv preprint arXiv:2106.01735 (2021).
• Adresgezgini. (n.d.). Adresgezgini/ICABMWZSL. GitHub. Retrieved November 11, 2021, from https://github.com/adresgezgini/ICABMwZSL.