Real-Time Hand Motion Recognition: A Robust Low-Cost Approach

Abstract

This study presents a robust, low-cost hand motion recognition approach designed to run on low-end computer systems. Our method detects and tracks hand region using real-time images obtained from a low-resolution camera (i.e. webcam) and is not depended on any training or calibration and is not required any special camera apparatus or selectors. The proposed system involves several image processing techniques such as background subtraction, face detection, skin colour detection and template matching. The experimental results show promising performance under various conditions. The method has a wide range of applications where more natural ways of interaction required, such as virtual reality applications, assistive technologies and simulation.

Keywords

• Background subtraction
• Face detection
• Hand movement recognition
• Motion tracking
• Template matching

References

1. Z. Merchant, E.T. Goetz, L. Cifuentes, W. Keeney-Kennicutt, T.J. Davis, “Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis”, Computers & Education, 70, 29–40, 2014.
2. D. Chambers, “‘Wii play as a family’: the rise in family-centred video gaming”, Leisure Studies, 31(1), 69–82, 2012.
3. H.H. Mousavi, M. Khademi, “A review on technical and clinical impact of microsoft kinect on physical therapy and rehabilitation”, Journal of Medical Engineering, 2014.
4. N.E. Seymour, A.G. Gallagher, S.A. Roman, M.K. O’brien, V.K. Bansal, D.K. Andersen, R.M. Satava, “Virtual reality training improves operating room performance: results of a randomized, double-blinded study”, Annals of surgery, 236(4), 458–464, 2002.
5. P.W. Lee, H.Y. Wang, Y.C. Tung, J.W. Lin, A. Valstar, “TranSection: hand-based interaction for playing a game within a virtual reality game”, 33rd ACM Conference on Human Factors in Computing Systems, Seoul, Republic of Korea, 18 – 23 April 2015.
6. O. Aran, Vision based sign language recognition: modeling and recognizing isolated signs with manual and non-manual components, PhD Thesis, Bogazici University, Istanbul, Turkey, 2008.
7. Y. Yin, Toward an intelligent multimodal interface for natural interaction. MSc Thesis, Massachusetts Institute of Technology, Cambridge, MA, United States, 2010.
8. X. Yang, A hand input-based approach to intuitive human-computer interactions in virtual reality. MPhil Thesis, The University of Hong Kong, Pokfulam, Hong Kong, 2010.

9. N. Ikizler, Understanding human motion: recognition and retrieval of human activities, PhD Thesis, Bilkent University; Ankara, Turkey, 2008.
10. M. Al-Rajab, Hand gesture recognition for multimedia applications, PhD Thesis, University of Leeds, Leeds, UK, 2008.
11. C.P. Chen, Y.T. Chen, P.H. Lee, Y.P. Tsai, S. Lei, “Real-time hand tracking on depth images”, Visual Communications and Image Processing, 1–4, 2011.
12. C. Keskin, L. Akarun, “STARS: Sign tracking and recognition system using input–output HMMs”, Pattern Recognition Letters, 30(12), 1086–1095, 2009.
13. L. Lamberti, F. Camastra, “Handy: a real-time three color glove-based gesture recognizer with learning vector quantization”, Expert Systems with Applications, 39(12), 10489–10494, 2012.
14. Y.H. Lee, S.K. Wu, Y.P. Liu, “Performance of remote target pointing hand movements in a 3D environment”, Human movement science, 32(3), 511–526, 2013.
15. X. Wang, K. Qin, “A Six-degree-of-freedom Virtual Mouse based on Hand Gestures”, International Conference on Electrical and Control Engineering, Wuhan, China, 25–27 June 2010.
16. N.H. Dardas, N.D. Georganas, “Real-time hand gesture detection and recognition using bag-of-features and support vector machine techniques”, IEEE Transactions on Instrumentation and Measurement, 60(11), 3592–3607, 2011.
17. N.H. Dardas, E.M. Petriu, “Hand gesture detection and recognition using principal component analysis”, IEEE International Conference on Computational Intelligence for Measurement Systems and Applications, Ottawa, Canada, 19–21 September 2011.
18. C.C. Hsieh, D.H. Liou, D. Lee, “A real time hand gesture recognition system using motion history image”, 2nd International Conference on Signal Processing Systems, Dalian, China, 5–7 July 2010.
19. P. Trigueiros, F. Ribeiro, L.P. Reis, “A comparison of machine learning algorithms applied to hand gesture recognition”, 7th Iberian Conference on Information Systems and Technologies, Madrid, Spain, 20–23 June 2012.
20. E. Sangineto, M. Cupelli, “Real-time viewpoint-invariant hand localization with cluttered backgrounds”, Image and Vision Computing. 30(1), 26–37, 2012.
21. B. Toni, J. Darko, “A robust hand detection and tracking algorithm with application to natural user interface”, 35th International Convention MIPRO, 1768–1774, 2012.
22. K. Jacobs, M. Ghasiazgar, I. Venter, R. Dodds, “Hand Gesture Recognition of Hand Shapes in Varied Orientations using Deep Learning”, Annual Conference of the South African Institute of Computer Scientists and Information Technologists; 2016.
23. P. Molchanov, S. Gupta, K. Kim, J. Kautz, “Hand gesture recognition with 3D convolutional neural networks”, Conference on Computer Vision and Pattern Recognition Workshops, 1–7, 2015.
24. Microsoft Kinect, “Kinect for Windows”, https://developer.microsoft.com/windows/kinect (24.12.2020).
25. Ultraleap, “Leap Motion Controller”, http://www.ultraleap.com/product/leap-motion-controller (24.12.2020).
26. C. Kanan, G.W. Cottrell, “Color-to-grayscale: does the method matter in image recognition?”, PloS one, 7(1), e29740:1–7, 2012.
27. K. Toyama, J. Krumm, B. Brumitt, B. Meyers, “Wallflower: Principles and practice of background maintenance”, International Conference on Computer Vision, Kerkyra, Greece, 20–27 September 1999.
28. P. Viola, M. Jones, “Rapid object detection using a boosted cascade of simple features”, Conference on Computer Vision and Pattern Recognition, Kauai, USA, 8–14 December 2001.
29. S.A. Dabhade, M.S. Bewoor, “Real time face detection and recognition using haar-based cascade classifier and principal component analysis”, International Journal of Computer Science and Management Research, 59–64, 2012.
30. R. Padilla, C. Costa Filho, M. Costa, “Evaluation of haar cascade classifiers designed for face detection”, International Journal of Computer, Electrical, Automation, Control and Information Engineering, 6(4),466–469, 2012.
31. S. Mattoccia, F. Tombari, L. Di Stefano, “Efficient template matching for multi-channel images”, Pattern Recognition Letters, 32(5), 694–700, 2011.
32. Emgu CV, “Emgu CV: OpenCV in .NET”, http://www.emgu.com (24.12.2020).
33. C.O. Conaire, N.E. O’Connor, A.F. Smeaton, “Detector adaptation by maximising agreement between independent data sources”, Conference on Computer Vision and Pattern Recognition, Minneapolis, USA, 17–22 June 2007.