Turkish sign language digits classification with CNN using different optimizers

Abstract

Sign language is a way for hearing-impaired people to communicate among themselves and with people without hearing impairment. Communication with the sign language is difficult because few people know this language and the language does not have universal patterns. Sign language interpretation is the translation of visible signs into speech or writing. The sign language interpretation process has reached a practical solution with the help of computer vision technology. One of the models widely used for computer vision technology that mimics the work of the human eye in a computer environment is deep learning. Convolutional neural networks (CNN), which are included in deep learning technology, give successful results in sign language recognition as well as other image recognition applications. In this study, the dataset containing 2062 images consisting of Turkish sign language digits was classified with the developed CNN model. One of the important parameters used to minimize network error of the CNN model during the training is the learning rate. The learning rate is a coefficient used to update other parameters in the network depending on the network error. The optimization of the learning rate is important to achieve rapid progress without getting stuck in local minimums while reducing network error. There are several optimization techniques used for this purpose. In this study, the success of four different training and test processes performed with SGD, RMSprop, Adam and Adamax optimizers were compared. Adam optimizer, which is widely used today with its high performance, was found to be the most successful technique in this study with 98.42% training and 98.55% test accuracy.

Keywords

• Convolutional Neural Networks
• Sign Language Recognition
• Turkish Sign Language

References

1. 1. Oral, A. Z., Türk işaret dili çevirisi, 2016, Ankara.
2. 2. Van Herreweghe, M. Prelinguaal dove jongeren en nederlands: een syntactisch onderzoek. 1996. PhD Thesis. Ghent University.
3. 3. Alkoffash, M. S., Bawaneh, M. J., Muaidi, H., Alqrainy, S., and Alzghool, M. A survey of digital image processing techniques in character recognition. International Journal of Computer Science and Network Security (IJCSNS), 2014. 14(3): p. 65.
4. 4. Bheda, V., and Radpour, D., Using deep convolutional networks for gesture recognition in American sign language. arXiv preprint arXiv:1710.06836, 2017.
5. 5. Koller, O., Ney, H., and Bowden, R., Deep learning of mouth shapes for sign language. In Proceedings of the IEEE International Conference on Computer Vision Workshops, 2015. p. 85-91.
6. 6. Huang, J., Zhou, W., Li, H., and Li, W., Sign language recognition using 3d convolutional neural networks. In 2015 IEEE international conference on multimedia and expo (ICME), 2015. p. 1-6.
7. 7. Pigou, L., Dieleman, S., Kindermans, P. J., and Schrauwen, B., Sign language recognition using convolutional neural networks. In European Conference on Computer Vision, 2014. p. 572-578.
8. 8. Hasan, S. K., and Ahmad, M., A new approach of sign language recognition system for bilingual users. In 2015 International Conference on Electrical & Electronic Engineering (ICEEE), 2015. p. 33-36.

9. 9. Agarwal, A., and Thakur, M. K., Sign language recognition using Microsoft Kinect. In 2013 Sixth International Conference on Contemporary Computing (IC3), 2013. p 181-185.
10. 10. Oyewole, O. G., Nicholas, G., Oludele, A., and Samuel, O., Bridging Communication Gap Among People with Hearing Impairment:, An Application of Image Processing and Artificial Neural Network. International Journal of Information and Communication Sciences, 2018. 3(1): p. 11.
11. 11. Wang, C., Gao, W., and Xuan, Z., A real-time large vocabulary continuous recognition system for chinese sign language. In Pacific-Rim Conference on Multimedia, 2001. p. 150-157.
12. 12. Kim, J. S., Jang, W., and Bien, Z., A dynamic gesture recognition system for the Korean sign language (KSL). IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 1996. 26(2): p. 354-359.
13. 13. Gani, E., & Kika, A., Albanian Sign Language (AlbSL) Number Recognition from Both Hand's Gestures Acquired by Kinect Sensors. arXiv preprint arXiv:1608.02991, 2016.
14. 14. Assaleh, K., and Al-Rousan, M., Recognition of Arabic sign language alphabet using polynomial classifiers. EURASIP Journal on Advances in Signal Processing, 2005. 13: p. 507614.
15. 15. Assaleh, K., Shanableh, T., Fanaswala, M., Bajaj, H., and Amin, F., Vision-based system for continuous Arabic Sign Language recognition in user dependent mode. In 2008 5th International Symposium on Mechatronics and Its Applications, 2008. p. 1-5.
16. 16. Solís, F., Martínez, D., and Espinoza, O., Automatic mexican sign language recognition using normalized moments and artificial neural networks. Engineering, 2016. 8(10): p. 733-740.
17. 17. Rajam, P. S., and Balakrishnan, G., Recognition of tamil sign language alphabet using image processing to aid deaf-dumb people. Procedia Engineering, 2012. 30: p. 861-868.
18. 18. Turkey Ankara Ayrancı Anadolu High School's Sign Language Digits Dataset, https://www.kaggle.com /ardamavi/sign-language-digits-dataset. Web. 10 Jan 2020.
19. 19. Gazel, S. E. R., and batı, C. T., Derin Sinir Ağları ile En İyi Modelin Belirlenmesi: Mantar Verileri Üzerine Keras Uygulaması. Yüzüncü Yıl Üniversitesi Tarım Bilimleri Dergisi, 29(3): p. 406-417.
20. 20. Kingma, D. P., and Ba, J., Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
21. 21. Aran, O., Keskin, C., and Akarun, L., Sign language tutoring tool. In 2005 13th European Signal Processing Conference, 2005. pp. 1-4.
22. 22. Beşer, F., Kizrak, M. A., Bolat, B., and Yildirim, T., Recognition of sign language using capsule networks. In 2018 26th Signal Processing and Communications Applications Conference (SIU), 2018. p. 1-4.
23. 23. Ozcan, T., and Basturk, A., Transfer learning-based convolutional neural networks with heuristic optimization for hand gesture recognition. Neural Computing and Applications, 2019. 31(12): p. 8955-8970.