LATİS TABANLI ANLAM ÇÖZÜMLENMESİ İLE TÜRKÇE İŞARET DİLİ TERCÜME SİSTEMİ

Yıl 2018, Cilt: 7 Sayı: 2, 490 - 503, 20.07.2018

Yelda Fırat Taşkın Uğurlu

https://doi.org/10.28948/ngumuh.443157

Cited By: 2

Öz

Günlük hayatta insanlar;
fikirlerini, düşüncelerini ve yaşadıklarını çevrelerindeki insanlara iletmek
için birbirleriyle etkileşirler. İşitme ve konuşma engelli insanlar ise
çevreleriyle bu etkileşimi sağlayamazlar. Başkalarıyla iletişim kurmak için
işaret dilini kullanırlar. İşaret dili ise, işitme
ve konuşma engellilerin kendi aralarında el hareketleri ve yüz mimikleri ile iletişim
kurmalarını sağlayan ülkeden ülkeye değişen evrensel olmayan bir dildir. Bu
anlamda yapılan çalışmanın amacı, işitme ve konuşma bozukluğu olan kişilerle
normal insanlar arasındaki iletişimi sağlayan Türkçe İşaret Dili tercüme sistemini geliştirmektir. Önerilen bu
sistemle işaret dilini gösteren el hareketleri,
Kinect aygıtı yardımıyla yakalanarak Kontur Analizinde kullanılan algoritmalarla
çözümlenmiştir. Çözümlenen görüntülerden elde edilen kelimelerin gerçek anlamları,
Biçimsel Kavram Analizi Kuramı çerçevesinde hazırlanan tematik rol latisleriyle
bulunmuştur. Gerçek anlamları bulunan bu kelimeler, içinde geçtikleri
cümlelerle birlikte bilgisayar ekranında görüntülenmiştir. Böylece bu sistemle,
engelli bir kişinin kitlesel bir kalabalıkla iletişim kurabilmesi sağlanmıştır.
Ayrıca geliştirilen bu sistemle Türkçe’nin anlamsal çözümlenmesine de katkı
sağlanması hedeflenmiştir.

Anahtar Kelimeler

Biçimsel kavram analizi, işaret dili tanımlama, kinect, kontur analizi, tematik rol latisleri

TURKISH SIGN LANGUAGE TRANSLATION SYSTEM WITH A LATTICE-BASED SEMANTIC ANALYSIS

Öz

In daily life people interact with each
other to communicate their ideas, thoughts and experiences to the people around
them. On the other hand, people with hearing and speech impairments are not
capable of communicating with their environment in the given sense. Thus, they
use sign language to communicate with others. Sign language is a non-universal
language that changes depending on the country in which it's being used and
enables people with hearing and speech impairments to communicate with other
people through hand gestures and facial expressions. In this sense, the aim of
the study is to develop Turkish Sign Language translation system that enables
communication between people with speech and hearing impairments and healthy
people. With this proposed system, the hand movements showing sign language are
captured with the help of Kinect device and analyzed with the algorithms used
in Contour Analysis. Actual meanings of the words obtained from the analyzed
images were found thematic role lattices that are prepared in the frame of
Formal Concept Analysis theory. The words of which actual meanings were found
are displayed with the sentences in which they were used. Therefore, thanks to
this system it is ensured that a disabled person can communicate with a mass
crowd. The developed system also aims to contribute to the semantic analysis of
the Turkish Language.

Anahtar Kelimeler

Formal concept analysis, sign language recognition, kinect, contour analysis, thematic role lattices

Kaynakça

• [1] AARTHI, M., VIJAYALAKSH, P., “Sign Language to Speech Conversion”, Fifth International Conference on Recent Trends in Information Technolgy, 2016, Doi: 10.1109/ICRTIT.2016.7569545.
• [2] AKMELIAWATIL, R., PO-LEEN OOI, M., KUANG, Y. C., “Real-Time Malaysian Sign Language Translation using Color Segmentation and Neural Network”, IEEE Instrumentation and Measurement Technology Conference, 1-6, Warsaw, Poland, 2007.
• [3] ARSAN, T., ÜLGEN, O., “Sign Language Converter”, International Journal of Computer Science & Engineering Survey (IJCSES), 6(4), 39-51, 2015.
• [4] BUI, T. D., NGUYEN, L. T., “Recognition of Vietnamese sign language using MEMS accelerometers” 1st International Conference on Sensing Technology, 118-122, Palmerston North, New Zealand, 2005.
• [5] ELLIS, K., BARCA, J. C., “Exploring Sensor Gloves for Teaching Children Sign Language”, Advances in Human-Computer Interaction, 2012, 1-8, 2012.
• [6] ERSHAED, H., AL-ALALI, I., KHASAWNEH, N., FRAIWAN, M., “An arabic sign language computer interface using the xbox Kinect”, In Annual Undergraduate Research Conf. on Applied Computing, May, 2011.
• [7] FIRAT, Y., UÇAR, Ö., KILIÇASLAN, Y., “Semantic Analysis with a LatticeBased FrameNet”, Journal of International Scientific Publications: Language, Individual & Society, 8, 512-518, Bulgaria, 2014.
• [8] FIRAT, Y., “The Semantic Inferences And Mappings Realized In Computer Through The Formal Concept Analysis”, Journal of the International Scientific Researches, 2(1), 86-107, 2017, Doi number:http://dx.doi.org/10.21733/ibad2099.
• [9] FIRAT, Y., “Referans Parametreleri ile Biçimlendirilmiş Kavram Latislerinin Bilgisayarlı Gerçekleştirimi”, Erzincan Üniversitesi, Fen Bilimleri Enstitüsü Dergisi, 10(1), 90-111, 2017, DOI: 10.18185/erzifbed.285800. [10] FRAIWAN, M., KHASAWNEH, N., ERSHEDAT, H., AL-ALALI, I., AL-KOFAHI, H., ”A Kinect-based system for Arabic sign language to speech translation”, International Journal of Computer Applications in Technology, 52(2), 117-126, 2015.
• [11] GANTER, B., WILLE, R., Formal Concept Analysis Mathematical Foundation, 5-23, Berlin: Springer, Verlag, 1999.
• [12] GAO, W., FANGA, G., “A Chinese sign language recognition system based on SOFM/SRN/HMM”, Journal of Pattern Recognition, 37 (12), 2389-2402, 2004.
• [13] GUNASEKARAN, K., MANIKANDAN, R., “Sign Language to Speech Translation System Using PIC Microcontroller”, International Journal of Engineering and Technology, 5(2), 1024-1028, 2013.
• [14] HERNNDEZ-REBOLLAR, J.L., LINDEMAN, R.W., KYRIAKOPOULOS, N., “A multi-class pattern recognition system for practical finger spelling translation”, 4th IEEE International Conference on Multimodal Interfaces (ICMI’02), 185-190, Pittsburg, USA, 2002.
• [15] JACKENDOFF, R., “On the Role of Conceptual Structure in Argument Selection: A Reply to Emonds”, Natural Language and Linguistic Theory 11, 11(2), 279-312, 1993.
• [16] LI, Y., CHEN, X., ZHANG, X., WANG, K., WANG, ZJ., “Sign-Component-Based Framework for Chinese Sign Language Recognition Using Accelerometer and sEMG Data”, IEEE Transactions On Biomedical Engineering, 59(10), 2695-2704, 2012.
• [17] ONG, EJ., BOWDEN, R., “A boosted classifier tree for hand shape detection”, In Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition FGR'04, IEEE Computer Society, 889-894, Washington, DC, USA, 2004.
• [18] QUAN, Y., “Chinese Sign Language Recognition Based On Video Sequence Appearance Modeling”, 5th IEEE Conference on Industrial Electronics and Applications, 1537-1542, 2010.
• [19] RAJAGANAPATHY, S., ARAVIND, B., KEERTHANA, B. , SIVAGAMI, M., “Conversation of Sign Language to Speech with Human Gestures”, 2nd International Symposium on Big Data and Cloud Computing (ISBCC’15), Procedia Computer Science 50, 10-15, 2015.
• [20] STARNER, T., WEAVER, J., PENTLAND, A., “Real-time american sign language recognition using desk and wearable computer based video”, IEEE Transactions on Pattern Analysis and Machine Intelligence. 20(12), 1371-1375. 1998.
• [21] https://www.codeproject.com/Articles/196168/Contour-Analysis-for-Image-Recognition-in-C (erişim tarihi 13.05.2018)
• [22] WILLE, R., Restructuring lattice theory: An approach based on hierarchies on concepts, 445-470, ed. (I. Rival), in Ordered Sets, Dordrecht-Boston: D. Reidel Publishing Company, 1982.