İşitme Engellilere Yönelik Türk İşaret Dilinin En Çok Kullanılan Kelimelerini Yazıya Çeviren Sistem Tasarımı

Abstract

İletişim, insanlar arasında duygu, düşünce, bilgi ve haber alışverişi olarak tanımlanır ve çeşitli biçimlerde gerçekleşen, karşılıklı bilgi ve duygu paylaşımını içeren karmaşık bir süreçtir. Kelimelerin ve cümlelerin ötesinde, iletişim aynı zamanda kültürler arasında bağ kurma, duyguları ifade etme ve düşünceleri paylaşma sürecini içerir. İşaret Dili, genellikle işitme güçlüğü çeken bireyler tarafından kullanılan bir iletişim türüdür ve jest ve mimikleri içerir. İşaret dili evrensel değildir; farklı ülkelerden gelen işitme engelli bireyler farklı işaret dilleri kullanırlar. Jestler, mimikler ve semboller işaret dili için dil bilgisel olarak düzenlenir ve her bir jeste "işaret" denir. Her işaretin üç ana parçası vardır: el şekli, ellerin pozisyonu ve ellerin hareketi. Geliştireceğimiz projede, Türk işaret dilinin en çok kullanılan kelimelerini tanıyabilen bir sistem oluşturmayı planlıyoruz. Bu sistem, işaret dilinin kelimelerini algılayarak bu ifadeleri yazıya dönüştürmeyi amaçlamaktadır. Bu sayede işitme engelli bireyler, çevreleriyle daha etkili iletişim kurabilir ve günlük etkileşimlerinde daha rahat bir deneyim yaşayabilirler.

Keywords

• İletişim
• İşitme engelli
• Türk işaret dili
• Yazıya dönüştürme

Development of a System for Translating Frequently Used Turkish Sign Language Words into Text for the Hearing Impaired

Abstract

Communication involves the exchange of emotions, thoughts, information, and news among individuals and takes various forms, encompassing both verbal and non-verbal methods. Sign language, utilized by individuals who are deaf or hard of hearing, relies on gestures and facial expressions. Sign language is not a universal system; instead, it varies significantly across different countries, with each nation having its own distinct version. Each sign comprises three main components: hand shape, hand position, and hand movement. This study aims to develop a system that recognizes the most commonly used words in Turkish Sign Language (TSL) and converts these signs into text. The system utilizes an image processing algorithm to detect and translate these words, facilitating effective communication for individuals who are Deaf or Hard of Hearing. The dataset includes 20 frequently used words, collected from 12 individuals, and trained using the YOLOv8 machine learning algorithm. The model achieved an accuracy rate of 99.4%, demonstrating its effectiveness in real-world conditions. This system aims to improve the daily interactions and communication experiences of Deaf or Hard of Hearing individuals by providing a reliable tool for sign language translation.

Keywords

• Communication
• Hearing Impaired
• Turkish Sign Language
• Text Conversion

References

1. [1] H. Yüksel, Introduction to Interpersonal Communication, Eskişehir, Turkey: Anadolu Univ. Publ., 1994, 180 pages.
2. [2] E. Arık, “A study on classifiers in Turkish sign language,” Bilig, no. 67, pp. 1–24, 2013.
3. [3] H. Erten and N. Arıcı, “The historical adventure of sign language and Turkish sign language,” Afyon Kocatepe Univ. J. Soc. Sci., vol. 24, no. 1, pp. 1–14, 2022.
4. [4] T. Starner and A. Pentland, “Real-time American sign language recognition from video using hidden Markov models,” in Proc. Int. Symp. Comput. Vis., Coral Gables, FL, USA, Nov. 21–23, 1995, pp. 265–270.
5. [5] T. Tazalli, Z. A. Aunshu, S. S. Liya, M. Hossain, Z. Mehjabeen, M. S. Ahmed, and M. I. Hossain, “Computer vision based Bengali sign language to text generation,” in Proc. 5th IEEE Int. Conf. Image Process. Appl. Syst. (IPAS), Genoa, Italy, Dec. 5–7, 2022, pp. 1–6.
6. [6] F. Gökçe and H. Kekül, “Turkish sign language word translator with microcontroller systems,” European J. Sci. Technol., no. 28, pp. 972–977, 2021.
7. [7] K. Halim and E. Rakun, “Sign language system for Bahasa Indonesia (known as SIBI) recognizer using TensorFlow and long short‑term memory,” in Proc. 2018 Int. Conf. Adv. Comput. Sci. Inf. Syst. (ICACSIS), Yogyakarta, Indonesia, 2018, pp. 403–407.
8. [8] L. A. E. Jiménez, M. E. Benalcázar, and N. Sotomayor, “Gesture recognition and machine learning applied to sign language translation,” in Proc. VII Latin Am. Congr. Biomed. Eng. (CLAIB), Bucaramanga, Colombia, Oct. 26–28, 2016, pp. 233–236.

9. [9] J. L. Hernandez‑Rebollar, N. Kyriakopoulos, and R. W. Lindeman, “A new instrumented approach for translating American sign language into sound and text,” in *Proc. Sixth IEEE Int. Conf. Automatic Face Gesture Recognit. (FG 2004)*, Seoul, South Korea, May 17–19, 2004, pp. 547–552.
10. [10] M. F. Karaca, Turkish sign language simulation with three-dimensional virtual model, Ph.D. dissertation, Karabük Univ., Karabük, Turkey, 2018.
11. [11] A. Z. Oral, Turkish sign language translation, Ankara, Turkey: Siyasal Bookstore, 2016, 142 pages.
12. [12] M. F. Karaca and Ş. Bayır, “Turkish sign language analysis: communication and grammar,” J. Natl. Acad. Educ., vol. 2, no. 2, pp. 35–58, 2018.
13. [13] S. A. Demir, “The language of silence: observations on Turkish sign language,” Bilig, no. 54, pp. 1–20, 2010.
14. [14] H. Erten and N. Arıcı, “Historical adventure of sign language and Turkish sign language,” Afyon Kocatepe Univ. J. Soc. Sci., vol. 24, no. 1, pp. 1–14, 2022.
15. [15] F. Hermens, “Automatic object detection for behavioural research using YOLOv8,” Behavior Research Methods, vol. 56, no. 7, pp. 7307–7330, May 2024.
16. [16] W. Z. Taffese, R. Sharma, M. H. Afsharmovahed, G. Manogaran, and G. Chen, “Benchmarking YOLOv8 for optimal crack detection in civil infrastructure,” arXiv, Jan. 12, 2025, . [17] D. Deepa, A. Sivasangari, R. Roonwal, and R. Nayan, “Pothole detection using roboflow convolutional neural networks,” in 2023 7th Int. Conf. Intelligent Comput. Control Syst. (ICICCS), 2023, pp. 560–564.
17. [18] S. G. E. Brucal, L. C. M. de Jesus, S. R. Peruda, L. A. Samaniego, and E. D. Yong, “Development of tomato leaf disease detection using YoloV8 model via RoboFlow 2.0,” in 2023 IEEE 12th Global Conf. Consumer Electron. (GCCE), 2023, pp. 692–694.
18. [19] H. Vasudevan and A. Nazari, “Recognition of fruit grading based on deep learning technique,” Evolution Electr. Electron. Eng., vol. 5, no. 1, pp. 420–426, 2024.
19. [20] M. Pavithra, P. S. Karthikesh, B. Jahnavi, M. Navyalokesh, and K. L. Krishna, “Implementation of enhanced security system using Roboflow,” in 2024 11th Int. Conf. Reliability, Infocom Technol. Optim. (Trends Future Directions) (ICRITO), 2024, pp. 1–5.
20. [21] G. R. Matuck, A. J. A. Castro, L. E. da Silva, and E. G. Carvalho, “Reconhecimento facial com inteligência artificial utilizando a plataforma RoboFlow,” Revista Prociências, vol. 6, no. 2, pp. 114–131, 2023.
21. [22] E. Goceri, “Medical image data augmentation: techniques, comparisons and interpretations,” Artificial Intelligence Review, vol. 56, pp. 12561–12605, 2023.
22. [23] A. Çamlıbel, B. Karakaya, and Y. H. Tanç, “Automatic modulation recognition with deep learning algorithms,” in 2024 32nd Signal Process. Commun. Appl. Conf. (SIU), 2024, pp. 1–4.
23. [24] G. Karaduman, E. Akın, B. Binay, and M. Dilekli, “Detection of insulator defects in catenary systems with deep learning,” Railway Eng., vol. 16, pp. 185–195, 2022.
24. [25] S. B. Nabijonovich and G. Najmiddin, “Optimizing PyQt5 development with Qt designer,” Web of Teachers: Inderscience Res., vol. 2, no. 4, pp. 254–259, 2024.