A Deep Learning based Approach for Identifying Spoken languages in Turkey

Abstract

Spoken language identification (SLID) systems are used in areas such as human–machine interaction and security. In forensic and intelligence work, identifying the language in short or degraded recordings is important for criminal, immigration, and national security purposes. To address the specific challenges of these domains, this study focuses on extracting robust acoustic features that remain discriminative even in noisy environments and with limited temporal data. In this study, a speech dataset was constructed using languages that are widely spoken in Türkiye, including those used by foreign residents and visitors. The recordings span six languages: Turkish, Kurdish, Arabic, Persian, Russian, and German. The task of recognizing the spoken language was formulated as a multi-class classification problem. To address this task, several state-of-the-art deep learning architectures were employed, including Convolutional Neural Networks (CNNs), Transformer-based encoders, Bidirectional Long Short-Term Memory (BiLSTM) networks, and Conformer encoder models, which integrate convolutional operations with self-attention mechanisms to effectively process short utterances and complex signal patterns. These models were trained and evaluated under different data configurations to assess their robustness and generalization. Model performance was evaluated using widely accepted metrics such as accuracy, recall, and macro-averaged F1-score, with a particular focus on handling unbalanced data and phonetically similar language pairs. Experimental results reveal that the Conformer encoder demonstrated superior robustness, achieving the highest accuracy of 99.89% and a macro-averaged F1-score of 0.9914 on the population-weighted unbalanced dataset. In contrast, the Transformer model showed significant sensitivity to class imbalance, where its macro F1-score dropped to 0.8179.

Keywords

• Spoken language identification
• Convolutional neural networks
• Transformer-based encoder architectures
• Deep learning
• Conformer encoder

References

1. Göç İdaresi Başkanlığı, T.C. İçişleri Bakanlığı, "T.C. İçişleri Bakanlığı Göç İdaresi Başkanlığı", https://www.goc.gov.tr/ , Access date: 18.05.2025.
2. Veri Portalı, TÜİK, "TÜİK - Veri Portalı", https://data.tuik.gov.tr/Kategori/GetKategori?p=Nufus-ve-Demografi-109 , Access date: 18.05.2025.
3. Marketing Türkiye, "Türkiye’ye sekiz ayda 40 milyon turist geldi: Rusya ve Almanya, başı çekiyor", https://www.marketingturkiye.com.tr/haberler/turkiyeye-sekiz-ayda-40-milyon-turist-geldi-rusya-ve-almanya-basi-cekiyor/ , Access date: 31.01.2026.
4. Gonzalez-Dominguez, J., Lopez-Moreno, I., Moreno, P. J., and Gonzalez-Rodriguez, J., "Frame-by-frame language identification in short utterances using deep neural networks", Neural Networks, 64: 49–58, (2015). DOI: https://doi.org/10.1016/j.neunet.2014.08.006
5. Li, H., Ma, B., and Lee, K.A., "Spoken language recognition: from fundamentals to practice", Proceedings of the IEEE, 101(5): 1136-1159, (2013). DOI: https://doi.org/10.1109/JPROC.2012.2237151
6. O’Shaughnessy, D., "Spoken language identification: An overview of past and present research trends", Speech Communication, 167: 103167, (2025). DOI: https://doi.org/10.1016/j.specom.2024.103167
7. Anidjar, O. H. and Yozevitch, R., "Enhancing neural spoken language recognition: An exploration with multilingual datasets", arXiv preprint, arXiv:2501.11065, (2025). DOI: https://doi.org/10.48550/arXiv.2501.11065
8. Çelik, Y., "Application of deep learning for voice command classification in Turkish language", Bitlis Eren Üniversitesi Fen Bilimleri Dergisi, 13(3): 701–708, (2024). DOI: https://doi.org/10.17798/bitlisfen.1477191

9. Al-Shathry, N. I., Eltahir, M. M., Asklany, S. A., Al Ghamdi, S. A., Almuhaimeed, A., Alanazi, F., Mohamed, A. A., and Rizwanullah, M., "Multi-class spoken language detection using artificial intelligence with fractal Al-Biruni earth radius optimization algorithm", Fractals, 32(9-10): 2540054, (2024). DOI: https://doi.org/10.1142/S0218348X25400547
10. Li, Z., Xu, Y., Ke, D., and Su, K., "PLDE: A lightweight pooling layer for spoken language recognition", Speech Communication, 158: 103055, (2024). DOI: https://doi.org/10.1016/j.specom.2024.103055
11. Singh, G., Sharma, S., Kumar, V., Kaur, M., Baz, M., and Masud, M., "Spoken language identification using deep learning", Computational Intelligence and Neuroscience, 2021: 5123671, (2021). DOI: https://doi.org/10.1155/2021/5123671
12. Mukherjee, S., Shivam, N., Gangwal, A., Khaitan, L., and Das, A. J., "Spoken language recognition using CNN", Proceedings of the 2019 International Conference on Information Technology, Institute of Electrical and Electronics Engineers Inc., Dec. 2019, 37-41, (2019). DOI: https://doi.org/10.1109/ICIT48102.2019.00013
13. Garai, S. and Samui, S., "Optimizing performance of spoken language identification systems for Indian languages using ensemble deep learning models", 2024 IEEE Calcutta Conference (CALCON), 1-5, (2024). DOI: https://doi.org/10.1109/CALCON63337.2024.10914168
14. Snyder, D., Garcia-Romero, D., McCree, A., Sell, G., Povey, D., and Khudanpur, S., "Spoken language recognition using x-vectors", Proceedings of the Speaker and Language Recognition Workshop, The International Society for Computers and Their Applications (ISCA), 105-111, (2018). DOI: https://doi.org/10.21437/Odyssey.2018-15
15. Ferrer, L., Lei, Y., McLaren, M., and Scheffer, N., "Study of senone-based deep neural network approaches for spoken language recognition", IEEE/ACM Transactions on Audio, Speech, and Language Processing, 24(1): 105-116, (2016). DOI: https://doi.org/10.1109/TASLP.2015.2496226
16. Bankur, F. and Kaya, M., "Deep learning based recognition of Turkish sign language letters with unique data set", Turkish Journal of Science and Technology, 17(2): 251-260, (2022). DOI: https://doi.org/10.55525/tjst.1073116
17. Çetintaş, D., and Yücel, Z., "Deepfake detection using fine-tuned CNN architectures", Celal Bayar Üniversitesi Fen Bilimleri Dergisi, 21(1): 121-128, (2025). DOI: https://doi.org/10.18466/cbayarfbe.1530209
18. Seymen, O. F., Olmez, E., Dogan, O., Er, O., and Hiziroglu, A., "Customer churn prediction using ordinary artificial neural network and convolutional neural network algorithms: A comparative performance assessment", Gazi University Journal of Science, 36(2): 720-733, (2023). DOI: https://doi.org/10.35378/gujs.992738
19. Salor, O., Pellom, B.L., Ciloglu, T., and Demirekler, M., "Turkish speech corpora and recognition tools developed by porting SONIC: towards multilingual speech recognition", Computer Speech and Language, 21(4): 580-593, (2007). DOI: https://doi.org/10.1016/j.csl.2007.01.001
20. Sert, E., "A deep learning based approach for the detection of diseases in pepper and potato leaves", Anadolu Tarım Bilimleri Dergisi, 36(2): 167-178, (2021). DOI: https://doi.org/10.7161/omuanajas.805152