Derleme

American Sign Language Recognition using YOLOv4 Method

Cilt: 6 Sayı: 1 20 Temmuz 2022
PDF İndir
EN

American Sign Language Recognition using YOLOv4 Method

Öz

Abstract – Sign language is one of the ways of communication that is used by people who are unable to speak or hear (deaf and mute), so not all people are able to understand this language. Therefore, to facilitate communication between normal people and deaf and mute people, many systems have been invented that translate gestures and signs within sign language into words to be understood. The aim of this research is to train a model to be able to detect and recognize hand gestures and signs and then translate them into letters, numbers and words using the You Only Look Once (YOLO) method through pictures or videos, even in real-time. YOLO is one of the methods used in detecting and recognizing things that depend in their work on convolutional neural networks (CNN), which are characterized by accuracy and speed in work. In this research, we have created a data set consisting of 8000 images divided into 40 classes, for each class, 200 images were taken with different backgrounds and under lighting conditions, which allows the model to be able to differentiate the signal regardless of the intensity of the lighting or the clarity of the image. And after training the model on the dataset many times, in the experiment using image data we got very good results in terms of MAP = 98.01% as accuracy and current average loss=1.3 and recall=0.96 and F1=0.96, and for video results, it has the same accuracy and 28.9 frames per second (fps).

Anahtar Kelimeler

Destekleyen Kurum

hora

Proje Numarası

60

Teşekkür

thanks

Kaynakça

  1. [1] Z. Zafrulla, H. Brashear, P. Yin, P. Presti, T. Starner, and H. Hamilton, “American sign language phrase verification in an educational game for deaf children,” in 2010 20th International Conference on Pattern Recognition, 2010, pp. 3846–3849.
  2. [2] C. Oz and M. C. Leu, “American sign language word recognition with a sensory glove using artificial neural networks,” Eng. Appl. Artif. Intell., vol. 24, no. 7, pp. 1204–1213, 2011.
  3. [3] S. Lang, M. Block, and R. Rojas, “Sign language recognition using kinect,” in International Conference on Artificial Intelligence and Soft Computing, 2012, pp. 394–402.
  4. [4] D. Aryanie and Y. Heryadi, “American sign language-based finger-spelling recognition using k-Nearest Neighbors classifier,” in 2015 3rd International Conference on Information and Communication Technology (ICoICT), 2015, pp. 533–536.
  5. [5] R. A. Kadhim and M. Khamees, “A real-time american sign language recognition system using convolutional neural network for real datasets,” Tem J., vol. 9, no. 3, p. 937, 2020.
  6. [6] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779–788.
  7. [7] J. Du, “Understanding of object detection based on CNN family and YOLO,” in Journal of Physics: Conference Series, 2018, vol. 1004, no. 1, p. 12029.
  8. [8] S. Daniels, N. Suciati, and C. Fathichah, “Indonesian Sign Language Recognition using YOLO Method,” in IOP Conference Series: Materials Science and Engineering, 2021, vol. 1077, no. 1, p. 12029.

Ayrıntılar

Birincil Dil

İngilizce

Konular

Mühendislik

Bölüm

Derleme

Yayımlanma Tarihi

20 Temmuz 2022

Gönderilme Tarihi

17 Haziran 2022

Kabul Tarihi

20 Haziran 2022

Yayımlandığı Sayı

Yıl 2022 Cilt: 6 Sayı: 1

Kaynak Göster

APA
Al-shaheen, A., Çevik, M., & Alqaraghulı, A. (2022). American Sign Language Recognition using YOLOv4 Method. International Journal of Multidisciplinary Studies and Innovative Technologies, 6(1), 61-65. https://izlik.org/JA47UA62KS
AMA
1.Al-shaheen A, Çevik M, Alqaraghulı A. American Sign Language Recognition using YOLOv4 Method. IJMSIT. 2022;6(1):61-65. https://izlik.org/JA47UA62KS
Chicago
Al-shaheen, Ali, Mesut Çevik, ve Alzubair Alqaraghulı. 2022. “American Sign Language Recognition using YOLOv4 Method”. International Journal of Multidisciplinary Studies and Innovative Technologies 6 (1): 61-65. https://izlik.org/JA47UA62KS.
EndNote
Al-shaheen A, Çevik M, Alqaraghulı A (01 Temmuz 2022) American Sign Language Recognition using YOLOv4 Method. International Journal of Multidisciplinary Studies and Innovative Technologies 6 1 61–65.
IEEE
[1]A. Al-shaheen, M. Çevik, ve A. Alqaraghulı, “American Sign Language Recognition using YOLOv4 Method”, IJMSIT, c. 6, sy 1, ss. 61–65, Tem. 2022, [çevrimiçi]. Erişim adresi: https://izlik.org/JA47UA62KS
ISNAD
Al-shaheen, Ali - Çevik, Mesut - Alqaraghulı, Alzubair. “American Sign Language Recognition using YOLOv4 Method”. International Journal of Multidisciplinary Studies and Innovative Technologies 6/1 (01 Temmuz 2022): 61-65. https://izlik.org/JA47UA62KS.
JAMA
1.Al-shaheen A, Çevik M, Alqaraghulı A. American Sign Language Recognition using YOLOv4 Method. IJMSIT. 2022;6:61–65.
MLA
Al-shaheen, Ali, vd. “American Sign Language Recognition using YOLOv4 Method”. International Journal of Multidisciplinary Studies and Innovative Technologies, c. 6, sy 1, Temmuz 2022, ss. 61-65, https://izlik.org/JA47UA62KS.
Vancouver
1.Ali Al-shaheen, Mesut Çevik, Alzubair Alqaraghulı. American Sign Language Recognition using YOLOv4 Method. IJMSIT [Internet]. 01 Temmuz 2022;6(1):61-5. Erişim adresi: https://izlik.org/JA47UA62KS