Abstract

Communication is the process of people transferring emotions, thoughts or information to the other party in various ways. One of the most effective ways of communication is language. Language is a communication tool that makes people's daily life easier and there are many hearing impaired people in our lives who cannot use this communication tool. Sign languages have been developed to facilitate the communication of hearing impaired people in society. There are specific sign languages varying according to the language of the countries. This study focuses on the Turkish sign language digits that are publicly available. Sign language is not known by all people of society. This situation causes communication disruptions in the social environments where hearing impaired people are present. A person who has not hearing impaired but cannot use sign language has the same problem. The aim of this study is to determine what people using sign language want to tell by using a deep learning architecture. For this purpose, the identification of digits in Turkish sign language has been realized by using the recently popular siamese neural network in this study. Siamese neural networks are a type of deep learning model that matches the same images in an image dataset. Using these networks, we have identified the digits used in Turkish sign language. The success rate of the matching was 98.16%. Consequently, siamese neural networks were found to be successful in identifying Turkish sign language digits with this study.

Keywords

• Siamese Neural Networks
• Turkish Sign Language
• Digit Recognition
• Deep Learning

Siyam Sinir Ağlarını Kullanarak Türk İşaret Dilindeki Rakamların Tanımlanması

Abstract

İletişim, insanların duygu, düşünce veya bilgiyi çeşitli yollar kullanarak karşı tarafa aktarma sürecidir. İletişimde en etkili yollardan birisi ise dildir. Dil, insanların günlük hayatını kolaylaştıran bir iletişim aracıdır ve bu iletişim aracını kullanamayan işitme engelli birçok insan vardır. İşitme engelli insanların, toplum içerisinde iletişimini kolaylaştırmak için işaret dilleri geliştirilmiştir. Her ülkenin kendi konuşma diline özgü işaret dili mevcuttur. Bu çalışma erişime açık Türk işaret dili rakamlarına odaklanmıştır. İşaret dili, toplumun her kesimi tarafından bilinmemektedir. Bu durum, işitme engelli insanların bulundukları sosyal ortamlarda iletişim aksaklıklarına neden olmaktadır. İşitme engelli olmayan ancak işaret dilini bilmeyen bir bireyde aynı problemi yaşamaktadır. Bu çalışmanın amacı, işaret dilini kullanan insanların ne anlatmak istediğini derin öğrenme mimarisi üzerinde tespit etmektir. Bu amaçla, işaret dilini rakamlarının, son zamanlarda popülerliği artan siyam sinir ağı ile tanımlanmasını bu çalışmada gerçekleştirilmiştir. Siyam sinir ağları, görüntü kümesinde aynı görüntüleri eşleştiren bir derin öğrenme modelidir. Bu ağları kullanarak Türk işaret dilinde kullanılan rakam görüntülerini tanımlamayı gerçekleştirdik. Elde edilen eşleştirme başarı oranı %98,16’dır. Sonuç olarak, bu çalışma ile Türk işaret dili rakamlarının tanımlanmasında siyam sinir ağlarının başarılı olduğu görülmüştür.

Keywords

• Siyam Sinir Ağları
• Türk İşaret Dili
• Rakam Tanımlama

References

1. [1] Redondo M. Investigación de la enseñanza ética de los periodistas en España . Análisis bibliométrico y prescripciones formativas aplicadas ( 2005-2015 ) Research on ethics education for journalists in Spain . 2017;235–52 doi:10.4185/RLCS.
2. [2] Thomas J, McDonagh D. Shared language:Towards more effective communication. Australas Med J [Internet]. 2013/01/31. 2013;6(1):46–54. Available from: https://www.ncbi.nlm.nih.gov/pubmed/ 23422948 doi:10.4066/AMJ.2013.1596.
3. [3] Lindquist KA, MacCormack JK, Shablack H. The role of language in emotion: predictions from psychological constructionism. Front Psychol [Internet]. 2015 Apr 14;6:444. Available from: https://www.ncbi.nlm.nih.gov/pubmed/25926809 doi:10.3389/fpsyg.2015.00444.
4. [4] Hassen R. Language as an Index of Identity, Power, Solidarity and Sentiment in the Multicultural Community of Wollo. J Soc. 2016;5(3):1–5. doi:10.4172/2471-8726.1000174.
5. [5] Alnfiai M, Sampali S. Social and Communication Apps for the Deaf and Hearing Impaired. In: 2017 International Conference on Computer and Applications (ICCA). 2017. p. 120–6. doi:10.1109/COMAPP.2017.8079756.
6. [6] Vijayalakshmi P, Aarthi M. Sign language to speech conversion. In: 2016 International Conference on Recent Trends in Information Technology (ICRTIT). 2016. p. 1–6. doi:10.1109/ICRTIT.2016.7569545.
7. [7] OKTEKIN B. DEVELOPMENT OF TURKISH SIGN LANGUAGE RECOGNITION APPLICATION. NEAR EAST UNIVERSITY; 2018.
8. [8] Yıldız Z, Yıldız S, Bozyer S. İŞİTMEEngelliTuri̇zmi̇ Sessi̇zTuri̇zm): Dünya VeTürki̇yPotansi̇yeli̇neYöneli̇kBi̇rDeğerlendi̇rme. Süleyman Demirel Üniversitesi Vizyoner Derg. 2018;103–17. doi:10.21076/vizyoner.339776.

9. [9] von Agris U, Zieren J, Canzler U, Bauer B, Kraiss KF. Recent developments in visual sign language recognition. Univers Access Inf Soc. 2008;6(4):323–62. doi:10.1007/s10209-007-0104-x.
10. [10] Toğaçar M, Ergen B. Biyomedikal Görüntülerde Derin Öğrenme ile Mevcut Yöntemlerin Kıyaslanması. Fırat Üniversitesi Mühendislik Bilim Derg. 2019;31(1):109–21.
11. [11] Cömert Z, Kocamaz AF. Fetal Hypoxia Detection Based on Deep Convolutional Neural Network with Transfer Learning Approach. In: Silhavy R, editor. Software Engineering and Algorithms in Intelligent Systems. Cham: Springer International Publishing; 2019. p. 239–48.
12. [12] Sertkaya ME, Ergen B, Togacar M. Diagnosis of Eye Retinal Diseases Based on Convolutional Neural Networks Using Optical Coherence Images. In: 2019 23rd International Conference Electronics. 2019. p. 1–5. doi:10.1109/ELECTRONICS.2019.8765579.
13. [13] Altuntaş Y, Cömert Z, Kocamaz AF. Identification of haploid and diploid maize seeds using convolutional neural networks and a transfer learning approach. Comput Electron Agric. 2019;163:104874. doi:https://doi.org/10.1016/j.compag.2019.104874.
14. [14] Pigou L, Dieleman S, Kindermans P-J, Schrauwen B. Sign Language Recognition Using Convolutional Neural Networks. Vol. 8925. 2015. 572–578 p. doi:10.1007/978-3-319-16178-5_40.
15. [15] Bheda V, Radpour ND. Using Deep Convolutional Networks for Gesture Recognition in American Sign Language. 2017;1710. 0683. Available from: https://arxiv.org/ftp/arxiv/papers/1710/1710.06836.pdf
16. [16] Demircioglu B, Bülbül G, Kose H. Leap Motion ile Türk İşaret Dili Tanıma / Turkish Sign Language Recognition With Leap Motion. 2016. doi:10.13140/RG.2.1.4923.3529.
17. [17] Abul Kalam M, Nazrul M, Mondal I, Ahmed B. Rotation Independent Digit Recognition in Sign Language. In: 2019 International Conference on Electrical, Computer and Communication Engineering (ECCE). 2019; 2019.
18. [18] Kwolek B, Sako S. Learning Siamese Features for Finger Spelling Recognition. 2017. 225–236 p. doi:10.1007/978-3-319-70353-4_20.
19. [19] Arda Mavi. Turkey Ankara Ayrancı Anadolu High School’s Sign Language Digits Dataset [Internet]. 2017 [cited 2019 Aug 21]. Available from: https://github.com/ardamavi/Sign-Language-Digits-Dataset
20. [20] Berlemont S, Lefebvre G, Duffner S, Garcia C. Class-Balanced Siamese Neural Networks. Neurocomputing. 2017 Oct 1; doi:10.1016/j.neucom.2017.07.060.
21. [21] YAZAN E, Talu MF. Comparison of the stochastic gradient descent based optimization techniques. In: 2017 International Artificial Intelligence and Data Processing Symposium (IDAP). 2017. p. 1–5. doi:10.1109/IDAP.2017.8090299.
22. [22] R. V, K.P. S. Siamese neural network architecture for homoglyph attacks detection. ICT Express [Internet]. 2019 May 31 [cited 2019 Aug 21]; Available from: https://www.sciencedirect.com/science/article/pii/S2405959519300025 doi:10.1016/J.ICTE.2019.05.002.
23. [23] Jansen H, Gallee MP, Schroder FH. Analysis of sonographic pattern in prostatic cancer: Comparison of longitudinal and transversal transrectal ultrasound with subsequent radical prostatectomy specimens. Eur Urol. 1990;18(3):174–8. doi:10.1159/000463903.
24. [24] Hariharan B, Arbeláez P, Girshick R, Malik J. Object Instance Segmentation and Fine-Grained Localization Using Hypercolumns. IEEE Trans Pattern Anal Mach Intell. 2017;39(4):627–39. doi:10.1109/TPAMI.2016.2578328.
25. [25] Toğaçar M, Ergen B, Sertkaya ME. Zatürre Hastalığının Derin Öğrenme Modeli ile Tespiti Detection of Pneumonia with Deep Learning Model. 2019;31(1):223–30. [26] Toğaçar M, Ergen B. Deep Learning Approach for Classification of Breast Cancer. In: 2018 International Conference on Artificial Intelligence and Data Processing (IDAP). 2018. p. 1–5. doi:10.1109/IDAP.2018.8620802.
26. [27] Cömert Z, Kocamaz AF. Comparison of Machine Learning Techniques for Fetal Heart Rate Classification. Acta Phys Pol A. 2017;132(3):451–4. doi:10.12693/APhysPolA.131.451.
27. [28] İni̇k Ö, ÜLKER Bilgisayar Mühendisliği Bölümü E, Üniversitesi G, Bilgisayar Mühendisliği Bölümü T, Üniversitesi S, yazar S, et al. Derin Öğrenme ve Görüntü Analizinde Kullanılan Derin Öğrenme Modelleri Deep Learning and Deep Learning Models Used in Image Analysis. GBAD) Gaziosmanpasa J Sci Res. 2017;ISSN:2146–8168.
28. [29] Cıbuk M, Budak U, Guo Y, Ince MC, Sengur A. Efficient deep features selections and classification for flower species recognition. Measurement. 2019;137:7–13. doi:https://doi.org/10.1016/j.measurement.2019.01.041.
29. [30] Ioffe S, Szegedy C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. 2015; Available from: http://arxiv.org/abs/1502.03167
30. [31] keras/mnist_siamese.py at master · keras-team/keras · GitHub [Internet]. [cited 2019 Aug 22]. Available from: https://github.com/keras-team/keras/blob/master/examples/mnist_siamese.py
31. [32] Reeskamp P. Is comparative advertising a trade mark issue ? Eur Intellect Prop Rev. 2008;30(4):130–7. doi:10.1145/2623330.2623612.
32. [33] Powers DMW, Ailab. Evaluation: From Precision, Recall and F-Measure To Roc, Informedness, Markedness & Correlation. 2011;2(1):37–63. Available from: http://www.bioinfo.in/ contents.php?id=51 doi:10.9735/2229-3981.
33. [34] Arıcan M, Cömert Z, Fatih Kocamaz A, Polat K. Analysis of Fetal Heart Rate Signal based on Neighborhood-based Variance Compression Method. 2018. doi:10.1109/IDAP.2018.8620898.