Research Article
BibTex RIS Cite

BERT Modeli'nin Sınıflandırma Doğruluğunun Sıfır-Atış Öğrenmesi ile Artırılması

Year 2021, Volume: 14 Issue: 2, 99 - 108, 22.12.2021

Şükrü Ozan Umut Özdil Davut Emre Taşar Büşra Arslan Gökçe Polat

https://doi.org/10.54525/tbbmd.1004781
Cited By: 1

Abstract

Bu çalışmada, çevrimiçi reklam platformlarında oluşturulan reklam metinlerinin sektöre göre otomatik olarak sınıflandırılması için Çift Yönlü Kodlayıcı Gösterimleri (BERT) modeli kullanılmıştır. Eğitim veri setimiz 44 farklı sektöre ait yaklaşık 101.000 adet, ilgili sektör isimleri ile etiketlenmiş reklam metninden oluşmaktadır. Sınıflandırma başarımının belirli sektörler için düşük olduğu gözlemlenmiştir. Bunun, ilgili eğitim veri setindeki aykırı bazı reklam metinlerinden kaynaklandığı gözlemlenmiştir. Bu sebeple sektör ve reklam metni uygunluğunu otomatik olarak tespit etmek ve veri seti için bir ön işleme gerçekleştirmek amacıyla doğal dil işleme (NLP) alanında güncel bir yöntem olan Sıfır-Atış Öğrenmesi yöntemi kullanılmıştır. Bu ön işleme çalışmasından sonra temizlenen veri seti ile tekrardan eğitilen BERT modelinin sınıflandırma başarımının önemli ölçüde arttığı gözlemlenmiştir.

Keywords

Dijital Pazarlama Reklam Metni NLP Metin Sınıflandırma BERT Sıfır-Atış Öğrenmesi

References

  • Emarketer,, “Advertisers will spend nearly $600 billion worldwide in 2015,” 2014.
  • Google., “1.1 Çevrimici reklamcılık ve Google Ads’in avantajları - Google Ads Yardım,” https://support.google.com/google-ads/answer/6123875?hl=tr, May 2021.
  • Z, A. and Adali, E., “Opinion mining and sentiment analysis for contex-tual online-advertisement,” in 2016 IEEE 10th International Conferenceon Application of Information and Communication Technologies (AICT). IEEE, 2016, pp. 1–3.
  • Google., “Reklam kalitesi hakkında - Google Ads Yardım,” https://support.google.com/google-ads/answer/156066?hl=tr&reftopic=10549746, May2021.
  • Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K., “Bert: Pre-trainingof deep bidirectional transformers for language understanding,” arXivpreprint arXiv:1810.04805, 2018.
  • Ozan, S ̧ . and Tas ̧ar, D. E.,“Auto-tagging of short conversational sentences using natural language processing methods,” in 2021 29th Signal Processing and Communications Applications Conference (SIU), 2021, pp. 1–4
  • Zulfat Miftahutdinov, Ilseyar Alimova, E. T., “Kfu nlp team at smm4h2019 tasks: Want to extract adverse drugs reactions from tweets? bertto the rescue.,” ACL 2019, pp. 52–57, 2019
  • Rønningstad, E., “Targeted sentiment analysis for norwegian text,” 2020.
  • Hochreiter, S. and Schmidhuber, J., “Long short-term memory,”Neuralcomputation, vol. 9, no. 8, pp. 1735–1780, 1997.
  • [Chen, P., Sun, Z., Bing, L., and Yang, W., “Recurrent attention networkon memory for aspect sentiment analysis,” in Proceedings of the 2017conference on empirical methods in natural language processing, 2017,pp. 452–461.
  • Dong, L., Wei, F., Tan, C., Tang, D., Zhou, M., and Xu, K., “Adaptiverecursive neural network for target-dependent twitter sentiment classifi-cation,” in Proceedings of the 52nd annual meeting of the associationfor computational linguistics (volume 2: Short papers), 2014, pp. 49–54.
  • Xue, W. and Li, T.,“Aspect based sentiment analysis with gatedconvolutional networks,” arXiv preprint arXiv:1805.07043, 2018.
  • Huang, B. and Carley, K. M.,“Parameterized convolutional neural networks for aspect level sentiment classification,” arXiv preprintarXiv:1909.06276, 2019.
  • Nergız, G., Safali, Y., Avaroğlu, E., and Erdoğan, S., “Classification ofturkish news content by deep learning based lstm using fasttext model,” in 2019 International Artificial Intelligence and Data Processing Sym-posium (IDAP). IEEE, 2019, pp. 1–6.
  • Dogru, H. B., Tilki, S., Jamil, A., and Hameed, A. A., “Deep learning-based classification of news texts using doc2vec model,” in2021 1st International Conference on Artificial Intelligence and Data Analytics(CAIDA). IEEE, 2021, pp. 91–96.
  • Gonz ́alez-Carvajal, S. and Garrido-Merch ́an, E. C., “Comparing bertagainst traditional machine learning text classification,” arXiv preprintarXiv:2005.13012, 2020.
  • Gao, Z., Feng, A., Song, X., and Wu, X., “Target-dependent sentimentclassification with bert,” IEEE Access, vol. 7, pp. 154290–154299, 2019.
  • Ozdil, U., Arslan, B., Taşar, D. E., Polat, G., and Ozan, Ş., “Adtext classification with bidirectional encoder representations,” in 2021 6th International Conference on Computer Science and Engineering (UBMK), 2021, pp. 1–6.
  • Chang, M., Ratinov, L., Roth, D., and Srikumar, V., “Importance of semantic representation: Dataless classification,” in AAAI-08/IAAI-08 Proceedings - 23rd AAAI Conference on Artificial Intelligence and the 20th Innovative Applications of Artificial Intelligence Conference, 2008, vol. 2, pp. 830–835, 23rd AAAI Conference on Artificial Intelligence and the 20th Innovative Applications of Artificial Intelligence Conference, AAAI-08/IAAI-08 ; Conference date: 13-07-2008 Through 17-07-2008.
  • Pushp, P. K. and Srivastava, M. M., “Train once, test anywhere: Zero-shot learning for text classification,” CoRR, vol. Abs/1712.05972, 2017.
  • Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., and Dean, J., “Distributed representations of words and phrases and their compositionality,” in Advances in Neural Information Processing Systems, Burges, C. J. C., Bottou, L., Welling, M., Ghahramani, Z., and Weinberger, K. Q., Eds. 2013, vol. 26, Curran Associates, Inc.
  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez,A. N., Kaiser, L., and Polosukhin, I., “Attention is all you need,” arXivpreprint arXiv:1706.03762, 2017.
  • Gupta, Shashij, et al. "Machine translation testing via pathological invariance." Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2020.
  • Do, Quang-Minh, Kungan Zeng, and Incheon Paik. "Resolving Lexical Ambiguity in English-Japanese Neural Machine Translation." 2020 3rd Artificial Intelligence and Cloud Computing Conference. 2020.
  • Loodos., “loodos/bert-base-turkish-uncased · hugging face,” https://github.com/Loodos/turkish-language-models, Aug. 2020.
  • Taşar, D. Emre, et al. "Auto-tagging of Short Conversational Sentences using Transformer Methods." arXiv preprint arXiv:2106.01735 (2021).
  • Adresgezgini. (n.d.). Adresgezgini/ICABMWZSL. GitHub. Retrieved November 11, 2021, from https://github.com/adresgezgini/ICABMwZSL.

Year 2021, Volume: 14 Issue: 2, 99 - 108, 22.12.2021

Şükrü Ozan Umut Özdil Davut Emre Taşar Büşra Arslan Gökçe Polat

https://doi.org/10.54525/tbbmd.1004781
Cited By: 1

Abstract

References

  • Emarketer,, “Advertisers will spend nearly $600 billion worldwide in 2015,” 2014.
  • Google., “1.1 Çevrimici reklamcılık ve Google Ads’in avantajları - Google Ads Yardım,” https://support.google.com/google-ads/answer/6123875?hl=tr, May 2021.
  • Z, A. and Adali, E., “Opinion mining and sentiment analysis for contex-tual online-advertisement,” in 2016 IEEE 10th International Conferenceon Application of Information and Communication Technologies (AICT). IEEE, 2016, pp. 1–3.
  • Google., “Reklam kalitesi hakkında - Google Ads Yardım,” https://support.google.com/google-ads/answer/156066?hl=tr&reftopic=10549746, May2021.
  • Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K., “Bert: Pre-trainingof deep bidirectional transformers for language understanding,” arXivpreprint arXiv:1810.04805, 2018.
  • Ozan, S ̧ . and Tas ̧ar, D. E.,“Auto-tagging of short conversational sentences using natural language processing methods,” in 2021 29th Signal Processing and Communications Applications Conference (SIU), 2021, pp. 1–4
  • Zulfat Miftahutdinov, Ilseyar Alimova, E. T., “Kfu nlp team at smm4h2019 tasks: Want to extract adverse drugs reactions from tweets? bertto the rescue.,” ACL 2019, pp. 52–57, 2019
  • Rønningstad, E., “Targeted sentiment analysis for norwegian text,” 2020.
  • Hochreiter, S. and Schmidhuber, J., “Long short-term memory,”Neuralcomputation, vol. 9, no. 8, pp. 1735–1780, 1997.
  • [Chen, P., Sun, Z., Bing, L., and Yang, W., “Recurrent attention networkon memory for aspect sentiment analysis,” in Proceedings of the 2017conference on empirical methods in natural language processing, 2017,pp. 452–461.
  • Dong, L., Wei, F., Tan, C., Tang, D., Zhou, M., and Xu, K., “Adaptiverecursive neural network for target-dependent twitter sentiment classifi-cation,” in Proceedings of the 52nd annual meeting of the associationfor computational linguistics (volume 2: Short papers), 2014, pp. 49–54.
  • Xue, W. and Li, T.,“Aspect based sentiment analysis with gatedconvolutional networks,” arXiv preprint arXiv:1805.07043, 2018.
  • Huang, B. and Carley, K. M.,“Parameterized convolutional neural networks for aspect level sentiment classification,” arXiv preprintarXiv:1909.06276, 2019.
  • Nergız, G., Safali, Y., Avaroğlu, E., and Erdoğan, S., “Classification ofturkish news content by deep learning based lstm using fasttext model,” in 2019 International Artificial Intelligence and Data Processing Sym-posium (IDAP). IEEE, 2019, pp. 1–6.
  • Dogru, H. B., Tilki, S., Jamil, A., and Hameed, A. A., “Deep learning-based classification of news texts using doc2vec model,” in2021 1st International Conference on Artificial Intelligence and Data Analytics(CAIDA). IEEE, 2021, pp. 91–96.
  • Gonz ́alez-Carvajal, S. and Garrido-Merch ́an, E. C., “Comparing bertagainst traditional machine learning text classification,” arXiv preprintarXiv:2005.13012, 2020.
  • Gao, Z., Feng, A., Song, X., and Wu, X., “Target-dependent sentimentclassification with bert,” IEEE Access, vol. 7, pp. 154290–154299, 2019.
  • Ozdil, U., Arslan, B., Taşar, D. E., Polat, G., and Ozan, Ş., “Adtext classification with bidirectional encoder representations,” in 2021 6th International Conference on Computer Science and Engineering (UBMK), 2021, pp. 1–6.
  • Chang, M., Ratinov, L., Roth, D., and Srikumar, V., “Importance of semantic representation: Dataless classification,” in AAAI-08/IAAI-08 Proceedings - 23rd AAAI Conference on Artificial Intelligence and the 20th Innovative Applications of Artificial Intelligence Conference, 2008, vol. 2, pp. 830–835, 23rd AAAI Conference on Artificial Intelligence and the 20th Innovative Applications of Artificial Intelligence Conference, AAAI-08/IAAI-08 ; Conference date: 13-07-2008 Through 17-07-2008.
  • Pushp, P. K. and Srivastava, M. M., “Train once, test anywhere: Zero-shot learning for text classification,” CoRR, vol. Abs/1712.05972, 2017.
  • Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., and Dean, J., “Distributed representations of words and phrases and their compositionality,” in Advances in Neural Information Processing Systems, Burges, C. J. C., Bottou, L., Welling, M., Ghahramani, Z., and Weinberger, K. Q., Eds. 2013, vol. 26, Curran Associates, Inc.
  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez,A. N., Kaiser, L., and Polosukhin, I., “Attention is all you need,” arXivpreprint arXiv:1706.03762, 2017.
  • Gupta, Shashij, et al. "Machine translation testing via pathological invariance." Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2020.
  • Do, Quang-Minh, Kungan Zeng, and Incheon Paik. "Resolving Lexical Ambiguity in English-Japanese Neural Machine Translation." 2020 3rd Artificial Intelligence and Cloud Computing Conference. 2020.
  • Loodos., “loodos/bert-base-turkish-uncased · hugging face,” https://github.com/Loodos/turkish-language-models, Aug. 2020.
  • Taşar, D. Emre, et al. "Auto-tagging of Short Conversational Sentences using Transformer Methods." arXiv preprint arXiv:2106.01735 (2021).
  • Adresgezgini. (n.d.). Adresgezgini/ICABMWZSL. GitHub. Retrieved November 11, 2021, from https://github.com/adresgezgini/ICABMwZSL.
There are 27 citations in total.

Details

Primary Language Turkish
Subjects Engineering
Journal Section Makaleler(Araştırma)
Authors

Şükrü Ozan 0000-0002-3227-348X

Umut Özdil This is me 0000-0002-6909-1727

Davut Emre Taşar 0000-0002-7788-0478

Büşra Arslan This is me 0000-0002-6155-7967

Gökçe Polat This is me 0000-0003-1657-6824

Publication Date December 22, 2021
Published in Issue Year 2021 Volume: 14 Issue: 2

Cite

APA Ozan, Ş., Özdil, U., Taşar, D. E., Arslan, B., et al. (2021). BERT Modeli’nin Sınıflandırma Doğruluğunun Sıfır-Atış Öğrenmesi ile Artırılması. Türkiye Bilişim Vakfı Bilgisayar Bilimleri Ve Mühendisliği Dergisi, 14(2), 99-108. https://doi.org/10.54525/tbbmd.1004781
AMA Ozan Ş, Özdil U, Taşar DE, Arslan B, Polat G. BERT Modeli’nin Sınıflandırma Doğruluğunun Sıfır-Atış Öğrenmesi ile Artırılması. TBV-BBMD. December 2021;14(2):99-108. doi:10.54525/tbbmd.1004781
Chicago Ozan, Şükrü, Umut Özdil, Davut Emre Taşar, Büşra Arslan, and Gökçe Polat. “BERT Modeli’nin Sınıflandırma Doğruluğunun Sıfır-Atış Öğrenmesi Ile Artırılması”. Türkiye Bilişim Vakfı Bilgisayar Bilimleri Ve Mühendisliği Dergisi 14, no. 2 (December 2021): 99-108. https://doi.org/10.54525/tbbmd.1004781.
EndNote Ozan Ş, Özdil U, Taşar DE, Arslan B, Polat G (December 1, 2021) BERT Modeli’nin Sınıflandırma Doğruluğunun Sıfır-Atış Öğrenmesi ile Artırılması. Türkiye Bilişim Vakfı Bilgisayar Bilimleri ve Mühendisliği Dergisi 14 2 99–108.
IEEE Ş. Ozan, U. Özdil, D. E. Taşar, B. Arslan, and G. Polat, “BERT Modeli’nin Sınıflandırma Doğruluğunun Sıfır-Atış Öğrenmesi ile Artırılması”, TBV-BBMD, vol. 14, no. 2, pp. 99–108, 2021, doi: 10.54525/tbbmd.1004781.
ISNAD Ozan, Şükrü et al. “BERT Modeli’nin Sınıflandırma Doğruluğunun Sıfır-Atış Öğrenmesi Ile Artırılması”. Türkiye Bilişim Vakfı Bilgisayar Bilimleri ve Mühendisliği Dergisi 14/2 (December 2021), 99-108. https://doi.org/10.54525/tbbmd.1004781.
JAMA Ozan Ş, Özdil U, Taşar DE, Arslan B, Polat G. BERT Modeli’nin Sınıflandırma Doğruluğunun Sıfır-Atış Öğrenmesi ile Artırılması. TBV-BBMD. 2021;14:99–108.
MLA Ozan, Şükrü et al. “BERT Modeli’nin Sınıflandırma Doğruluğunun Sıfır-Atış Öğrenmesi Ile Artırılması”. Türkiye Bilişim Vakfı Bilgisayar Bilimleri Ve Mühendisliği Dergisi, vol. 14, no. 2, 2021, pp. 99-108, doi:10.54525/tbbmd.1004781.
Vancouver Ozan Ş, Özdil U, Taşar DE, Arslan B, Polat G. BERT Modeli’nin Sınıflandırma Doğruluğunun Sıfır-Atış Öğrenmesi ile Artırılması. TBV-BBMD. 2021;14(2):99-108.

Cited By

Ön eğitimli Bert modeli ile patent sınıflandırılması
Gazi Üniversitesi Mühendislik Mimarlık Fakültesi Dergisi
https://doi.org/10.17341/gazimmfd.1292543
 Download Cover Image

Article Acceptance

Use user registration/login to upload articles online.

The acceptance process of the articles sent to the journal consists of the following stages:

1. Each submitted article is sent to at least two referees at the first stage.

2. Referee appointments are made by the journal editors. There are approximately 200 referees in the referee pool of the journal and these referees are classified according to their areas of interest. Each referee is sent an article on the subject he is interested in. The selection of the arbitrator is done in a way that does not cause any conflict of interest.

3. In the articles sent to the referees, the names of the authors are closed.

4. Referees are explained how to evaluate an article and are asked to fill in the evaluation form shown below.

5. The articles in which two referees give positive opinion are subjected to similarity review by the editors. The similarity in the articles is expected to be less than 25%.

6. A paper that has passed all stages is reviewed by the editor in terms of language and presentation, and necessary corrections and improvements are made. If necessary, the authors are notified of the situation.

0

.   This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.