KEYPOINT DETECTOR RETRAINING TECHNIQUES FOR THE COMMUNICATION SYSTEM OF SIGN LANGUAGE SPEAKERS

Year 2020, Volume: 21 , 74 - 86, 27.11.2020

Guram Chaganava , David Kakulia

https://doi.org/10.18038/estubtda.822295

Abstract

The study described in this article examines the approaches of retraining of the deep learning model for hand palm keypoint detection in images. This is one of the studies conducted to create an innovative communication system for sign language speakers. The target of the given study is to find an optimal technique of retraining for increasing the degree of the keypoint detector generalization. So, it must be able to accurately detect keypoints in images it has not seen during training. It will make the communication system usable in real-life conditions.
In the article, there are reviewed three approaches of retraining: Retraining in series, retraining using ‘united’ dataset and retraining using mixed datasets. Experiments were conducted to test the effectiveness of each of them. The paper presents the results of the experiments and a relatively optimal method selected among them.

Keywords

Sign language, Communication system, Keypoint detection, Retraining

Supporting Institution

The Shota Rustaveli National Science Foundation of Georgia (SRNSFG)

Project Number

PHDF--18-342

Thanks

Thanks to the Shota Rustaveli National Science Foundation of Georgia

References

• Strecha C, Lindner A, Ali K and Fua P. Training for Task Specific Keypoint Detection, in Denzler J., Notni G., Süße H. (eds) Pattern Recognition. DAGM 2009. Lecture Notes in Computer Science, Berlin, Heidelberg.
• Howard A, Zhu M, Chen B, Kalenichenko D, Wang WWT, Andreetto M and H. Adam, MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications, 2017.
• Sandler M, Howard A, Zhu M, Zhmoginov A and Chen L. MobileNetV2: Inverted Residuals and Linear Bottlenecks, in IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 2018.
• Toshev A and Szegedy C. DeepPose: Human Pose Estimation via Deep Neural Networks, in IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, 2014.