Representing American Sign Language Letters and Numbers with Humanoid Robot Arm

Year 2021, Volume: 4 Issue: 2, 136 - 145, 30.11.2021

Mehmet Gül

https://doi.org/10.34088/kojose.882323

Abstract

The widespread use of robotic systems is an example of increasing technological developments in daily life. The robotic systems that are accustomed to widespread use in the industry show huge impact in education, health, and many other areas nowadays. Robotic systems are no longer expensive and are not difficult to access systems that can be drawn and designed with design software like CAD programs. Within the scope of the study, major changes are made in the reference design, and Python software is used to send data to the Arduino control card that is used for control of the robotic system and to provide movement. The humanoid robotic arm developed in the study is printed with the 3D printer, sign languages that can be realized by one-handed letters among sign languages were investigated. All 26 alphabetical letters of English in American Sign Language can be demonstrated with the right hand alone. It has nearly the same principle that represents the demonstration of sign language expressions with a robotic hand, and the American Sign Language is the most widely known sign language. If experience is gained with this study, it can be used for the other sign language. In this study, audio and text messages sent over Python were performed on the humanoid robotic arm.

Keywords

Humanoid Robotic Arm, Sign Language, Deep Learning, Human-Robot Interaction, Speech-Recognition

References

• [1] Republic of Turkey Ministry of Family Labor and Social Services (Web Page: https://www.ailevecalisma.gov.tr/media/31492/istatistik-bulteni-kasim2019.pdf) (Date Accessed: 01.05.2020) [2] Alemi M., Meghdari A., Ghazisaedy M., 2014. Employing humanoid robots for teaching English language in Iranian junior high-schools. Int. J. Humanoid Rob. 11(3), pp.1–25. doi: 10.1142/S0219843614500224
• [3] Alemi M., Ghanbarzadeh A., Meghdari A., Moghadam L.J., 2015. Clinical application of a humanoid robot in pediatric cancer interventions. Int. J. Soc. Rob., pp. 1–17. doi:10.1007/ s12369-015-0294-y
• [4] Meghdari A., Alemi M., Taheri A., 2013. The effects of using humanoid robots for treatment of individuals with autism in Iran. In: 6th Neuropsychology Symposium, Tehran, Iran.
• [5] Mayberry R.I., 2002. Cognitive development in deaf children: the interface of language and perception in neuropsychology. In: Handbook of Neuropsychology, 8(Part II), pp. 71–107
• [6] Besio S., et al., 2007. Critical factors involved in using interactive robots for play activities of children with disabilities. In: Proceedings of AAATE 2007 on Challenges for Assistive Technology, pp. 505–509
• [7] Janssen J.B., van der Wal C.C., Neerincx M.A., Looije R., 2011. Motivating children to learn arithmetic with an adaptive robot game. In: Mutlu, B., Bartneck, C., Ham, J., Evers, V., Kanda, T. (eds.) ICSR 2011. LNCS, vol. 7072, pp. 153–162. Springer, Heidelberg
• [8] Nalin M., et al., 2012. Children’s adaptation in multi-session interaction with a humanoid robot. In: 2012 IEEE on RO-MAN. IEEE
• [9] Schmitz A., et al., 2010. Design, realization and sensorization of the dexterous icub hand. In: 10th IEEE-RAS International Conference on Humanoid Robots (Humanoids). IEEE
• [10] Albers A., et al., 2006, Upper body of a new humanoid robot-the design of ARMAR III. In: 2006 6th IEEE-RAS International Conference on Humanoid Robots. IEEE
• [11] Diftler M.A., et al., 2011. Robonaut 2-the first humanoid robot in space. In: 2011 IEEE International Conference on Robotics and Automation (ICRA). IEEE
• [12] Kose H., Akalin N., Uluer P., 2014. Socially interactive robotic platforms as sign language tutors. International Journal of Humanoid Robotics, 11(01), 1450003.
• [13] Köse H., Uluer P., Akalın N., Yorgancı R., Özkul A., Ince G., 2015. The effect of embodiment in sign language tutoring with assistive humanoid robots. International Journal of Social Robotics, 7(4), pp.537-548.
• [14] Valli C., Lucas C., 2000. Linguistics of American Sign Language: An Introduction. Gallaudet University Press, Washington DC
• [15] Open-Source Thingiverse Design. https://www.thingiverse.com/thing:2269115 (Access Time: 01/05/2020)
• [16] Design of Humanoid Robot prepared by InMoov company. https://www.thingiverse.com/thing:17773 (Access Time: 01/05/2020)
• [17] Design of Humanoid Robot prepared by InMoov company. https://www.thingiverse.com/thing:65274 (Access Time: 01/05/2020)
• [18] Hirose S., Umetani Y., 1978. The development of soft gripper for the versatile robot hand. Mech. Mach. Theory, 13(3), pp. 351–359
• [19] Stenman M, 2015. Automatic speech recognition An evaluation of Google Speech, Corpus ID: 13963607, Published
• [20] Gruhn R.E, Minker W, Nakamura S, 2011. Statistical pronunciation modeling for non-native speech processing. Springer Science & Business Media
• [21] Jerri A.J., 1977. The shannon sampling theoremits various extensions and applications: A tutorial review. Proceedings of the IEEE, 65(11), pp. 1565–1596
• [22] Westphal M., 1997. The use of cepstral means in conversational speech recognition. In: EUROSPEECH
• [23] Rainer E. Gruhn W.M., Nakamura S., 2011. Statistical pronunciation modeling for non-native speech processing. In: Springer Science and Business Media
• [24] Tanigaki K., Yamamot, H., Sagisak, Y., 2000. A hierarchical language model incorporating class-dependent word models for oov words recognition. In: INTERSPEECH
• [25] Soudamalla S.K, “Implications of Conversational AI on Humanoid Robots”, site: https://nbn-resolving.org/urn:nbn:de:bsz:ch1-qucosa2-724268
• [26] Kim J.Y., Liu C., Calvo R.A., McCabe K., Taylor S.C.R., Schuller B.W., Wu K., 2019. A Comparison of Online Automatic Speech Recognition Systems and the Nonverbal Responses to Unintelligible Speech. arXiv.org > cs > arXiv:1904.12403
• [27] Daniels P., Iwago K., 2017. The suitability of cloudbased speech recognition engines for language learning. JALT CALL Journal, 13(3), pp.229-239
• [28] American Sign Language. https://en.wikipedia.org/wiki/American_manual_alphabet (Access time: 01/12/2020)