A novel bidirectional long short-term memory model with multi-head attention for accurate language detection

Yıl 2025, Cilt: 40 Sayı: 3, 1979 - 1994, 21.08.2025

Sinan Toklu , Abdullah Talha Kabakuş

https://doi.org/10.17341/gazimmfd.1543854

https://izlik.org/JA27TF65TW

Öz

Language detection, one of the most important elements used in natural language processing, is used extensively in various applications such as machine translation, sentiment analysis, and information retrieval. Thanks to language detection, communication between people in many different countries is possible. In addition, human-animal interaction can also be carried out in this area. In this paper, a novel Bidirectional Long Short-Term Memory model with Multi-Head Attention mechanism is proposed to accurately classify text into 17 languages, namely Arabic, Danish, Dutch, English, French, German, Greek, Hindi, Italian, Kannada, Malayalam, Portuguese, Russian, Spanish, Swedish, Tamil, and Turkish. A publicly available dataset consisting of 10,337 texts written in the above-mentioned languages is utilized to train and evaluate the proposed model. The proposed novel model achieved an extraordinary accuracy, precision, recall, and F1-score of 99.9%, outperforming the state-of-the-art baseline models. In particular, the proposed model demonstrated perfect precision (100%) for 15 languages, namely Arabic, Dutch, English, French, German, Greek, Hindi, Italian, Kannada, Malayalam, Portuguese, Russian, Swedish, Tamil, and Turkish. This research highlights the effectiveness of deep learning techniques in language detection, providing promising avenues for further advances in the field of multilingual text processing.

Anahtar Kelimeler

language detection , language classification , translation , deep learning , long short-term memory

Doğru dil tespiti için çok-başlı dikkat mekanizması ile yenilikçi çift yönlü uzun kısa-süreli bellek modeli

https://izlik.org/JA27TF65TW

Öz

Doğal dil işleme alanında kullanılan en önemli unsurlardan biri olan dil tespiti, makine çevirisi, duygu analizi ve bilgi erişimi gibi çeşitli uygulamalarda yaygın olarak kullanılmaktadır. Dil tespiti sayesinde, birçok farklı ülkedeki insanlar arasındaki iletişim mümkün hale gelmektedir. Ayrıca, insan-hayvan etkileşimi de bu alanda gerçekleştirilebilmektedir. Bu çalışmada, metinleri 17 farklı dile, ismen Arapça, Danca, Felemenkçe, İngilizce, Fransızca, Almanca, Yunanca, Hintçe, İtalyanca, Kannada, Malayalamca, Portekizce, Rusça, İspanyolca, İsveççe, Tamilce ve Türkçe, doğru bir şekilde sınıflandırmak için Çok-Başlı Dikkat mekanizmasına sahip özgün bir Çift Yönlü Uzun Kısa-Süreli Hafıza modeli önerilmektedir. Önerilen modelin eğitimi ve değerlendirilmesi için yukarıda belirtilen dillerde yazılmış 10.337 metinden oluşan, halka açık bir veriseti kullanılmıştır. Önerilen özgün model, en gelişkin temel referans modelleri geride bırakarak %99,9 gibi yüksek bir doğruluk, kesinlik, duyarlılık ve F1-skoru elde etmiştir. Özellikle, önerilen model 15 dil, ismen Arapça, Kannada, Tamilce, İsveççe, Rusça, Portekizce, Malayalamca, İtalyanca, Hintçe, Yunanca, Almanca, Fransızca, İngilizce, Felemenkçe ve Türkçe için mükemmel bir kesinlik (100%) elde etmiştir. Bu araştırma, dil tespitinde derin öğrenme tekniklerinin etkinliğini vurgulayarak, çok dilli metin işleme alanında daha fazla ilerleme için umut verici yollar sunmaktadır.

Anahtar Kelimeler

dil tespiti , dil sınıflandırma , tercüme , derin öğrenme , uzun kısa-süreli hafıza

Kaynakça

• 1. Brockman G. Greg Brockman on X: ‘ChatGPT just crossed 1 million users; it’s been 5 days since launch. https://x.com/gdb/status/1599683104142430208. Yayın tarihi Aralık 05, 2022. Erişim tarihi Haziran 19, 2025.
• 2. Cook J. 6 Giveaway Signs of ChatGPT-Generated Content. https://www.forbes.com/sites/jodiecook/2023/12/06/6-giveaway-signs-of-chatgpt-generated-content/. Yayın tarihi Aralık 06, 2023. Erişim tarihi Haziran 19, 2025.
• 3. Kaya F., Ertuğrul Ö.F., A novel feature extraction approach for text-based language identification: Binary patterns, Journal of the Faculty of Engineering and Architecture of Gazi University, 31 (4), 1085-1094, 2016.
• 4. Işık G., Artuner H., Turkish dialect recognition in terms of prosodic by long short-term memory neural networks, Journal of the Faculty of Engineering and Architecture of Gazi University, 35 (1), 213-224, 2020.
• 5. Işık G., Artuner H., Turkish Dialect Recognition Using Acoustic and Phonotactic Features in Deep Learning Architectures, Journal of Information Technologies, 13 (3), 207-216, 2020.
• 6. Jauhiainen T., Lui M., Zampieri M., Baldwin T., Lindén K., Automatic language identification in texts: A survey, Journal of Artificial Intelligence Research, 65 (1), 675-682, 2019.
• 7. Habic V., Semenov A., ve Pasiliao E.L., Multitask deep learning for native language identification, Knowl Based Syst, 209, 106440, 2020.
• 8. Baştürk F., Şahin H., Comparison of Machine Learning Classification Algorithms: Example of Language Identification from Text, Electronic Letters on Science and Engineering, 18 (2), 68-78, 2022.
• 9. Fateh A., Birgani R.T, Fateh M., Abolghasemi V., Advancing Multilingual Handwritten Numeral Recognition With Attention-Driven Transfer Learning, IEEE Access, 12, 41381-41395, 2024.
• 10. Mahmud T., Ptaszynski M., Masui F., Exhaustive Study into Machine Learning and Deep Learning Methods for Multilingual Cyberbullying Detection in Bangla and Chittagonian Texts, Electronics (Basel), 13 (9), 1-36, 2024.
• 11. Ergin İ., İnan T., Encoder character based using decoder and attention algorithms word production, Journal of the Faculty of Engineering and Architecture of Gazi University, 39 (3), 1999-2009, 2024.
• 12. Karaca A., Aydın Ö., Generating headlines for Turkish news texts with transformer architecture based deep learning method, Journal of the Faculty of Engineering and Architecture of Gazi University, 39 (1), 485-495, 2024.
• 13. Ghoshal A., Swietojanski P., Renals S., Multilingual training of deep neural networks, Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP ’13), Vancouver, BC, Kanada, 7319-7323, 26-31 Mayıs, 2013.
• 14. Fateh A., Fateh M., Abolghasemi V., Multilingual handwritten numeral recognition using a robust deep network joint with transfer learning, Inf Sci (N Y), 581, 479-494, 2021.
• 15. Liu P., Zhang L., Gulla J.A., Multilingual Review-Aware Deep Recommender System via Aspect-based Sentiment Analysis, ACM Trans Inf Syst, 39 (2), 1-33, 2021.
• 16. Omran T.M., Sharef B.T., Grosan C., Li Y., Transfer learning and sentiment analysis of Bahraini dialects sequential text data using multilingual deep learning approach, Data Knowl Eng, 143, 1-19, 2023.
• 17. Licht H., Cross-Lingual Classification of Political Texts Using Multilingual Sentence Embeddings, Political Analysis, 31 (3), 366-379, 2023.
• 18. Wadud M.A.H., Mridha M.F., Shin J., Nur K., Saha A.K., Deep-BERT: Transfer Learning for Classifying Multilingual Offensive Texts on Social Media, Computer Systems Science and Engineering, 44 (2), 1775-1791, 2023.
• 19. Alshanqiti A.M., Albouq S., Alkhodre A.B., Namoun A., Nabil E., Employing a Multilingual Transformer Model for Segmenting Unpunctuated Arabic Text, Applied Sciences (Switzerland), 12 (20), 1-15, 2022.
• 20. Saumya S., Kumar A., Singh J.P., Filtering offensive language from multilingual social media contents: A deep learning approach, Eng Appl Artif Intell, 133, 1-15, 2024.
• 21. Karayigit H., Akdagli A., Aci C.I., Homophobic and Hate Speech Detection Using Multilingual-BERT Model on Turkish Social Media, Information Technology and Control, 51 (2), 356-375, 2022.
• 22. Guven Z.A., Lamurias, A., Multilingual bi-encoder models for biomedical entity linking, Expert Syst, c. 40 (9), 1-14, 2023.
• 23. Alcantara T.H.M., Krütli D., Ravada R., Hanne T., Multilingual Text Summarization for German Texts Using Transformer Models, Information (Switzerland), 14 (6), 1-13, 2023.
• 24. Yang Z.G., Laki L.J., Solving Hungarian natural language processing tasks with multilingual generative models, Annales Mathematicae et Informaticae, 57, 92-106, 2023.
• 25. Nasir J.A. ve Din Z.U., Syntactic Structured Framework for Resolving Reflexive Anaphora in Urdu Discourse Using Multilingual NLP, KSII Transactions on Internet and Information Systems, 15 (4), 1409-1425, 2021.
• 26. Unanue I.J., Haffari G., Piccardi M., T3L: Translate-and-Test Transfer Learning for Cross-Lingual Text Classification, Trans Assoc Comput Linguist, 11, 1147-1161, 2023.
• 27. Pedregosa F. vd., Scikit-learn: Machine Learning in Python, Journal of Machine Learning Research, 12, 2825-2830, 2011.
• 28. Abadi M. vd., TensorFlow: A System for Large-Scale Machine Learning, Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2016), Savannah, GA, A.B.D., 265-283, 2-4 Kasım, 2016.
• 29. Chollet F., Deep Learning with Python, Manning Publications, Shelter Island, New York, A.B.D., 2017.
• 30. Harris C.R. vd., Array Programming with NumPy, Nature, 585, 357-362, 2020.
• 31. McKinney W., Data Structures for Statistical Computing in Python, Proceedings of the 9th Python in Science Conference (SCIPY 2010), Austin, Texas, 56-61, 28 Haziran-3 Temmuz, 2010.
• 32. Hunter J.D., Matplotlib: A 2D Graphics Environment, Comput Sci Eng, 9 (3), 90-95, 2007.
• 33. Waskom M.L., seaborn: statistical data visualization, J Open Source Softw, 6 (60), 1-4, 2021.
• 34. Saji B., Language Detection. https://www.kaggle.com/datasets/basilb2s/language-detection/. Yayın tarihi Şubat 10, 2021. Erişim tarihi Haziran 19, 2025.
• 35. Liu S., Himel G.M.S., Wang J., Breast Cancer Classification With Enhanced Interpretability: DALAResNet50 and DT Grad-CAM, IEEE Access, 12, 196647-196659, 2024.
• 36. Wang L., Zhang L., Qi X., Yi Z., Deep Attention-Based Imbalanced Image Classification, IEEE Trans Neural Netw Learn Syst, 33 (8), 3320-3330, 2022.
• 37. Nguyen V.D., Bui N.D., Do H.K., Skin Lesion Classification on Imbalanced Data Using Deep Learning with Soft Attention, Sensors, 22, 1-24, 2022.
• 38. Zubair M., Woo S., Lim S., Kim D., Deep Representation Learning With Sample Generation and Augmented Attention Module for Imbalanced ECG Classification, IEEE J Biomed Health Inform, 28 (5), 2461-2472, 2024.
• 39. Bojanowski P., Grave E, Joulin A., Mikolov T., Enriching Word Vectors with Subword Information, arXiv preprint, 1607.04606, 1-12, 2016.
• 40. Vaswani A. vd., Attention Is All You Need, Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, California, A.B.D., 1-11, 4-9 Aralık, 2017.
• 41. He K., Zhang X., Ren S., Sun J., Deep Residual Learning for Image Recognition, Proceedings of the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, A.B.D., 770-778, 27-30 Haziran, 2016.
• 42. Kingma D.P., Ba J.L., Adam: A Method for Stochastic Optimization, Proceedings of the 3rd International Conference on Learning Representations (ICLR 2015), San Diego, California, A.B.D., 1-15, 7-9 Mayıs, 2015.
• 43. Akalin F., Survival Classification in Heart Failure Patients by Neural Network-Based Crocodile and Egyptian Plover (CEP) Optimization Algorithm, Arab J Sci Eng, 49 (3), 3897-3914, 2024.
• 44. O’Malley T., Bursztein E., Long J., Chollet F. KerasTuner. https://github.com/keras-team/keras-tuner. Yayın tarihi: Mart 4, 2024. Erişim tarihi Haziran 19, 2025.
• 45. Li L., Jamieson K., DeSalvo G., Rostamizadeh A., Talwalkar A., Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization, Journal of Machine Learning Research, 18 (1), 6765-6816, 2018.
• 46. van Veen R., Biehl M., ve de Vries G.J., sklvq: Scikit Learning Vector Quantization, Journal of Machine Learning Research, 22 (231), 1-6, 2021.
• 47. Ke G. vd., LightGBM: A highly efficient gradient boosting decision tree, Adv Neural Inf Process Syst, 30 (NIPS 2017), 3149-3157, 2017.
• 48. Chen T., Guestrin C., XGBoost: A scalable tree boosting system, Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’16), San Francisco, CA, A.B.D., 785-794, 13-17 Ağustos, 2016.
• 49. Devlin J., Chang M.W., Lee K., Toutanova K., BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2019), Minneapolis, Minnesota, A.B.D., 4171-4186, 2-7 Haziran, 2019.
• 50. Caruana R., Lawrence S., Giles L., Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping, Advances in Neural Information Processing Systems 13-Proceedings of the 2000 Conference (NIPS 2000), Denver, CO, A.B.D., 402-408, 27 Kasım-02 Aralık, 2000.