Ses Tabanlı Dönüştürücü Modeller Kullanılarak LLM ile Arapça Cümlelerdeki Telaffuz Hatalarının Tespiti

Abstract

Kur’an-ı Kerim, Müslümanlar için yalnızca bir kutsal kitap değil, aynı zamanda doğru ve güzel bir şekilde okunması ibadet sayılan bir metindir. Bu nedenle, Kur’an tilavetinin doğru telaffuzla yapılması hem dini hem de eğitsel açıdan büyük önem taşımaktadır. Özellikle dijitalleşen dünyada, sesli Kur’an verilerinin otomatik yöntemlerle değerlendirilmesi, yeni nesil eğitim araçlarının ve yapay zekâ tabanlı uygulamaların gelişimine katkı sağlayabilecek önemli bir çalışma alanı olarak öne çıkmaktadır. Bu çalışmada, Arapça Kur’an metinlerinin sesli okunması sonucunda oluşan verilerin doğruluğunu değerlendirmeye yönelik bir sistem önerilmektedir. Sistem kapsamında, farklı speech-to-text (konuşmadan metne) modelleri kullanılarak ses verileri metne dönüştürülmekte ve elde edilen metinler, referans Kur’an metni ile karşılaştırılarak çeşitli benzerlik metrikleriyle analiz edilmektedir.Kullanılan veri kümesi, Kur’an hafızları tarafından okunmuş yüksek kaliteli ses kayıtlarını içermektedir. Her biri Kur’an eğitiminde tecrübeli 10 farklı hafız, Kur’an-ı Kerim’in tamamını sesli olarak okumuş ve bu kayıtlar, sistemin değerlendirme süreçlerinde temel veri olarak kullanılmıştır.Bu çalışma, Arapça Kur’an tilavetinin yeni nesil yöntemlerle analiz edilmesi ve değerlendirilmesine yönelik yeni bir yaklaşım sunarak, hem dil teknolojileri alanına hem de dinî eğitim uygulamalarına katkı sağlamayı amaçlamaktadır.

Keywords

• Arapça ses tanıma
• derin öğrenme modelleri
• telaffuz doğruluğu
• benzerlik metrikleri
• otomatik metin karşılaştırma
• yapay zeka destekli karşılaştırma.

Ethical Statement

Çalışmada etik kurul iznine gerek yoktur

Detection of Pronunciation Errors in Arabic Sentences Using LLM with Voice-Based Transformer Models

Abstract

The Holy Qur'an is not only a holy book for Muslims, but also a text whose correct and beautiful recitation is considered worship. Therefore, the correct pronunciation of Qur'an recitation is of great importance both religiously and educationally. Especially in the digitalized world, the evaluation of audio Qur'an data with automatic methods stands out as an important field of study that can contribute to the development of new generation educational tools and artificial intelligence-based applications. In this study, we propose a system for evaluating the accuracy of the data generated as a result of reading Arabic Qur'an texts aloud. Within the scope of the system, audio data is converted into text using different speech-to-text models and the resulting texts are analyzed with various similarity metrics by comparing them with the reference Quran text. The dataset used includes high-quality audio recordings read by Quran memorizers. Ten different hafiz, each with experience in Qur'anic education, recited the entire Qur'an aloud and these recordings were used as the basic data in the evaluation processes of the system. This study aims to contribute to both the field of language technologies and religious education practices by presenting a new approach to the analysis and evaluation of Arabic Qur'anic recitation with automated methods.

Keywords

• Arabic voice recognition
• deep learning models
• pronunciation accuracy
• similarity metrics
• automatic text comparison
• artificial intelligence assisted comparison.

Ethical Statement

Ethics committee approval is not required for the study.

References

1. [1] Warschauer M, Healey D. Computers and language learning: An overview. Lang Teach. 1998;31(2):57-71.
2. [2] Stockwell G. A review of technology choice for teaching language skills and areas in the CALL literature. ReCALL. 2007;19(2):105-20.
3. [3] Franco H, Neumeyer L, Kim Y, Ronen O. Automatic pronunciation scoring for language instruction. In: Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP); 1997 Apr 21-24; Munich, Germany. IEEE; 1997. p. 1471-4.
4. [4] Tappert CC. A Markov model acoustic phonetic component for automatic speech recognition. Int J Man-Mach Stud. 1977;9(3):363-73.
5. [5] Ahmed AA, Hasan MK, Jaber MM, Al-Ghuribi SM, Abd DH, Khan W, et al. Arabic text detection using rough set theory: Designing a novel approach. IEEE Access. 2023;11:68428-38.
6. [6] Graves A, Jaitly N. Towards end-to-end speech recognition with recurrent neural networks. In: Proceedings of the 31st International Conference on Machine Learning (ICML); 2014 Jun 21-26; Beijing, China. PMLR; 2014. p. 1764-72.
7. [7] Jenkins J. The phonology of English as an international language: New models, new norms, new goals. Oxford: Oxford University Press; 2000.
8. [8] Phan H, Vo M. Pronunciation challenges for Vietnamese learners of English. J Lang Teach Res. 2019;10(2):210-7.

9. [9] Darwish K. Arabizi detection and conversion to Arabic. In: Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP); 2014 Oct 25; Doha, Qatar. Association for Computational Linguistics; 2014. p. 1-7.
10. [10] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS); 2017 Dec 4-9; Long Beach, CA. 2017. p. 5998-6008.
11. [11] Chung YA, Hsu WN, Tang H, Glass J. An unsupervised autoregressive model for speech representation learning. In: Proceedings of Interspeech; 2019 Sep 15-19; Graz, Austria. 2019. p. 146-50.
12. [12] Baevski A, Zhou H, Mohamed A, Auli M. wav2vec 2.0: A framework for self-supervised learning of speech representations. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS); 2020 Dec 6-12; Virtual Event. 2020.
13. [13] Akhtar S, Hussain F, Raja FR, Ehatisham-ul-haq M, Baloch NK, Ishmanov F, et al. Improving mispronunciation detection of Arabic words for non-native learners using deep convolutional neural network features. Electronics. 2020;9(6):963.
14. [14] Algabri M, Mathkour H, Alsulaiman M, Bencherif MA. Mispronunciation detection and diagnosis with articulatory-level feedback generation for non-native Arabic speech. Mathematics. 2022;10(11):1812.
15. [15] Alqadheeb FM, Asif A, Ahmad HF. Correct pronunciation detection for Classical Arabic phonemes using deep learning. Unpublished manuscript, King Faisal University and Lancaster University; 2021.
16. [16] Asif A, Alrashidi A, Alsulaiman M. An approach for pronunciation classification of classical Arabic phonemes using deep learning. Appl Sci. 2021;11(24):11925.
17. [17] Çalık ŞS, Küçükmanisa A, Kilimci ZH. An ensemble-based framework for mispronunciation detection of Arabic phonemes. Appl Acoust. 2023;191:108669.
18. [18] Farooq J, Imran M. Mispronunciation detection in articulation points of Arabic letters using machine learning. IEEE Access. 2021;9:165337-46.
19. [19] Harere AA, Jallad KA. Mispronunciation detection of basic quranic recitation rules using deep learning. arXiv preprint arXiv:2305.06429. 2023.
20. [20] Hsu WN, Bolte B, Tsai YHH, Lakhotia K, Salakhutdinov R, Mohamed A. HuBERT: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Trans Audio Speech Lang Process. 2021;29:3451-60.
21. [21] Çalık ŞS, Küçükmanisa A, Kilimci ZH. A novel framework for mispronunciation detection of Arabic phonemes using audio-oriented transformer models. Appl Acoust. 2024;215:109711. Available from: https://doi.org/10.1016/j.apacoust.2023.109711.
22. [22] Radford A, Kim JW, Xu T, Brockman G, McLeavey C, Sutskever I. Robust speech recognition via large-scale weak supervision. In: Proceedings of the 40th International Conference on Machine Learning (ICML); 2023 Jul 23-29; Honolulu, HI. PMLR; 2023. p. 28492-518.
23. [23] Alrashoudi N, Al-Khalifa HS, Alotaibi Y. Improving mispronunciation detection and diagnosis for non-native learners of the Arabic language. Discover Computing. 2025;28:14.