Representing American Sign Language Letters and Numbers with Humanoid Robot Arm

Öz

The widespread use of robotic systems is an example of increasing technological developments in daily life. The robotic systems that are accustomed to widespread use in the industry show huge impact in education, health, and many other areas nowadays. Robotic systems are no longer expensive and are not difficult to access systems that can be drawn and designed with design software like CAD programs. Within the scope of the study, major changes are made in the reference design, and Python software is used to send data to the Arduino control card that is used for control of the robotic system and to provide movement. The humanoid robotic arm developed in the study is printed with the 3D printer, sign languages that can be realized by one-handed letters among sign languages were investigated. All 26 alphabetical letters of English in American Sign Language can be demonstrated with the right hand alone. It has nearly the same principle that represents the demonstration of sign language expressions with a robotic hand, and the American Sign Language is the most widely known sign language. If experience is gained with this study, it can be used for the other sign language. In this study, audio and text messages sent over Python were performed on the humanoid robotic arm.

Anahtar Kelimeler

• Humanoid Robotic Arm
• Sign Language
• Deep Learning
• Human-Robot Interaction
• Speech-Recognition

Kaynakça

1. [1] Republic of Turkey Ministry of Family Labor and Social Services (Web Page: https://www.ailevecalisma.gov.tr/media/31492/istatistik-bulteni-kasim2019.pdf) (Date Accessed: 01.05.2020) [2] Alemi M., Meghdari A., Ghazisaedy M., 2014. Employing humanoid robots for teaching English language in Iranian junior high-schools. Int. J. Humanoid Rob. 11(3), pp.1–25. doi: 10.1142/S0219843614500224
2. [3] Alemi M., Ghanbarzadeh A., Meghdari A., Moghadam L.J., 2015. Clinical application of a humanoid robot in pediatric cancer interventions. Int. J. Soc. Rob., pp. 1–17. doi:10.1007/ s12369-015-0294-y
3. [4] Meghdari A., Alemi M., Taheri A., 2013. The effects of using humanoid robots for treatment of individuals with autism in Iran. In: 6th Neuropsychology Symposium, Tehran, Iran.
4. [5] Mayberry R.I., 2002. Cognitive development in deaf children: the interface of language and perception in neuropsychology. In: Handbook of Neuropsychology, 8(Part II), pp. 71–107
5. [6] Besio S., et al., 2007. Critical factors involved in using interactive robots for play activities of children with disabilities. In: Proceedings of AAATE 2007 on Challenges for Assistive Technology, pp. 505–509
6. [7] Janssen J.B., van der Wal C.C., Neerincx M.A., Looije R., 2011. Motivating children to learn arithmetic with an adaptive robot game. In: Mutlu, B., Bartneck, C., Ham, J., Evers, V., Kanda, T. (eds.) ICSR 2011. LNCS, vol. 7072, pp. 153–162. Springer, Heidelberg
7. [8] Nalin M., et al., 2012. Children’s adaptation in multi-session interaction with a humanoid robot. In: 2012 IEEE on RO-MAN. IEEE
8. [9] Schmitz A., et al., 2010. Design, realization and sensorization of the dexterous icub hand. In: 10th IEEE-RAS International Conference on Humanoid Robots (Humanoids). IEEE

9. [10] Albers A., et al., 2006, Upper body of a new humanoid robot-the design of ARMAR III. In: 2006 6th IEEE-RAS International Conference on Humanoid Robots. IEEE
10. [11] Diftler M.A., et al., 2011. Robonaut 2-the first humanoid robot in space. In: 2011 IEEE International Conference on Robotics and Automation (ICRA). IEEE
11. [12] Kose H., Akalin N., Uluer P., 2014. Socially interactive robotic platforms as sign language tutors. International Journal of Humanoid Robotics, 11(01), 1450003.
12. [13] Köse H., Uluer P., Akalın N., Yorgancı R., Özkul A., Ince G., 2015. The effect of embodiment in sign language tutoring with assistive humanoid robots. International Journal of Social Robotics, 7(4), pp.537-548.
13. [14] Valli C., Lucas C., 2000. Linguistics of American Sign Language: An Introduction. Gallaudet University Press, Washington DC
14. [15] Open-Source Thingiverse Design. https://www.thingiverse.com/thing:2269115 (Access Time: 01/05/2020)
15. [16] Design of Humanoid Robot prepared by InMoov company. https://www.thingiverse.com/thing:17773 (Access Time: 01/05/2020)
16. [17] Design of Humanoid Robot prepared by InMoov company. https://www.thingiverse.com/thing:65274 (Access Time: 01/05/2020)
17. [18] Hirose S., Umetani Y., 1978. The development of soft gripper for the versatile robot hand. Mech. Mach. Theory, 13(3), pp. 351–359
18. [19] Stenman M, 2015. Automatic speech recognition An evaluation of Google Speech, Corpus ID: 13963607, Published
19. [20] Gruhn R.E, Minker W, Nakamura S, 2011. Statistical pronunciation modeling for non-native speech processing. Springer Science & Business Media
20. [21] Jerri A.J., 1977. The shannon sampling theoremits various extensions and applications: A tutorial review. Proceedings of the IEEE, 65(11), pp. 1565–1596
21. [22] Westphal M., 1997. The use of cepstral means in conversational speech recognition. In: EUROSPEECH
22. [23] Rainer E. Gruhn W.M., Nakamura S., 2011. Statistical pronunciation modeling for non-native speech processing. In: Springer Science and Business Media
23. [24] Tanigaki K., Yamamot, H., Sagisak, Y., 2000. A hierarchical language model incorporating class-dependent word models for oov words recognition. In: INTERSPEECH
24. [25] Soudamalla S.K, “Implications of Conversational AI on Humanoid Robots”, site: https://nbn-resolving.org/urn:nbn:de:bsz:ch1-qucosa2-724268
25. [26] Kim J.Y., Liu C., Calvo R.A., McCabe K., Taylor S.C.R., Schuller B.W., Wu K., 2019. A Comparison of Online Automatic Speech Recognition Systems and the Nonverbal Responses to Unintelligible Speech. arXiv.org > cs > arXiv:1904.12403
26. [27] Daniels P., Iwago K., 2017. The suitability of cloudbased speech recognition engines for language learning. JALT CALL Journal, 13(3), pp.229-239
27. [28] American Sign Language. https://en.wikipedia.org/wiki/American_manual_alphabet (Access time: 01/12/2020)