El Hareketlerinden İşaret Dilini Algılayıp Yazıya Dönüştürme

Öz

Tasarlanacak işaret dili tanıma projesinin güncel teknolojiler kullanılarak optimize biçimde gerçeklenmesi amaçlanmıştır. Projenin makine öğrenmesi bölümü Keras ve Sklearn kullanılarak TensorFlow üzerinden yapılacaktır. TensorFlow, ilerleyen aşamalarda projeyi mobil bir ortama taşıma ihtimali göz önünde bulundurularak seçilmiştir. Kullanılacak nesne tanıma yöntemi MediaPipe Holistic olarak seçilmiştir.

Anahtar Kelimeler

• Gerçek Zamanlı İşaret Dili Tanıma
• Makine Öğrenmesi
• Derin Öğrenme
• Nesne Tanıma
• Bilgisayarla Görü

Detecting Sign Language from Hand Gestures and Translating it into Text

Abstract

The sign language recognition project to be designed is aimed to be realized in an optimized way using up-to-date technologies. The machine learning part of the project will be done over TensorFlow using Keras and Sklearn. TensorFlow was chosen considering the possibility of moving the project to a mobile environment in the future. The object recognition method to be used was chosen as MediaPipe Holistic.

Keywords

• Real Time Sign Language Recognition
• Machine Learning
• Deep Learning
• Object Recognition
• Computer Vision

References

1. Ko, S.-K., Kim, C. J., Jung, H., & Cho, C. (2019). Neural Sign Language Translation Based on Human Keypoint Estimation. Applied Sciences, 9(13), 2683. doi:10.3390/app9132683
2. RWTH-PHOENIX-2014-T veri seti, https://www-i6.informatik.rwth-aachen.de/~koller/RWTH-PHOENIX-2014-T/
3. Necati Cihan Camgoz, Simon Hadfield, Oscar Koller, Hermann Ney, and Richard Bowden. Neural Sign Language Translation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
4. Konstantinidis, D., Dimitropoulos, K., & Daras, P. (2018). Sıgn Language Recognıtıon Based On Hand And Body Skeletal Data. 2018- 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON). doi:10.1109/3dtv.2018.8478467
5. Hosain, A. A., Santhalingam, P. S., Pathak, P., Rangwala, H., & Kosecka, J. (2020). FineHand: Learning Hand Shapes for American Sign Language Recognition. 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020). doi:10.1109/fg47880.2020.00062
6. Zhang, Z., Pu, J., Zhuang, L., Zhou, W., & Li, H. (2019). Continuous Sign Language Recognition via Reinforcement Learning. 2019 IEEE International Conference on Image Processing (ICIP). doi:10.1109/icip.2019.8802972
7. Pytorch vs Tensorflow 2021, https://towardsdatascience.com/pytorch-vs-tensorflow-2021-d403504d7bc3