Human posture prediction by Deep Learning

Abstract

İnterpreting the human posture in human videos and pictures constitutes the most basic structure of human posture prediction. A system is created that decides what the movement is and what purpose it is made by evaluating pictures and videos. In this way, a structure has been created that determines and classifies human movements as an automatic system. A mechanism of motional meaning contained in the created system has been recognized in such away that the pattern is expressed. It is intended to take advantage of these components by taking instant information. A result was obtained by primarily inferring instant still images and eliminating time intervals that do not contain information range. A classification was made according to their accuracy. Based on the location coordinates of the images and videos, it was tried to determine how people might react in the neck stage. Thanks to the analysis performed through the joints with optical flow calculation, motion information was obtained and classifications and analyses expressing the power of motion were created. Motion information on the region determined in the image is determined by the detection of joints, revealing the power generated by movement. The created histograms provide ease of classification of motion. Based on the reliability of the descriptions, which include the concept of the time in a sequential way with the detection of joints, it was desired to create a sliding classification mechanism within the framework of these joints. As a result of this study, it was aimed to obtain a functional structure that can recognize and understand the autonomous movement of stationary or moving beings. An efficient structure has been created in terms of providing a useful and facilitating mechanism by solving the problems in estimation.

Keywords

• Human pose estimation
• motion detection
• human body recognition and tracking

References

1. [1] Parekh, P., & Patel, A. (2021). Deep Learning-Based 2D and 3D Human Pose Estimation: A Survey. In Proceedings of Second International Conference on Computing, Communications, and Cyber-Security (pp. 541-556). Springer, Singapore.
2. [2] Souvenir, R., & Babbs, J. (2008, June). Learning the viewpoint manifold for action recognition. In 2008 IEEE Conference on Computer Vision and Pattern Recognition (pp. 1-7). IEEE.
3. [3] Yang, Y., & Ramanan, D. (2012). Articulated human detection with flexible mixtures of parts. IEEE transactions on pattern analysis and machine intelligence, 35(12), 2878-2890.
4. [4] Wang, Y., Huang, K., & Tan, T. (2007, June). Human activity recognition based on r transform. In 2007 IEEE Conference on Computer Vision and Pattern Recognition (pp. 1-8). IEEE.
5. [5] Ramanan, D. (2006, December). Learning to parse images of articulated bodies. In Nips (Vol. 1, No. 6, p. 7).
6. [6] Lee, J., & Ahn, B. (2020). Real-time human action recognition with a low-cost RGB camera and mobile robot platform. Sensors, 20(10), 2886.
7. [7] Tran, D., & Forsyth, D. (2010, September). Improved human parsing with a full relational model. In European Conference on Computer Vision (pp. 227-240). Springer, Berlin, Heidelberg.
8. [8] Weinland, D., Ronfard, R., & Boyer, E. (2006). Free viewpoint action recognition using motion history volumes. Computer vision and image understanding, 104(2-3), 249-257.

9. [9] Chang, M. C., Qi, H., Wang, X., Cheng, H., & Lyu, S. (2015). Fast Online Upper Body Pose Estimation from Video. In BMVC (pp. 104-1).
10. [10] Eichner, M., Ferrari, V., & Zurich, S. (2009, September). Better appearance models for pictorial structures. In Bmvc (Vol. 2, p. 5).
11. [11] Yang, Y., & Ramanan, D. (2012). Articulated human detection with flexible mixtures of parts. IEEE transactions on pattern analysis and machine intelligence, 35(12), 2878-2890.
12. [12] https://mobidev.biz/blog/human-pose-estimation-ai-personal-fitness-coach
13. [13] Toshev, A., & Szegedy, C. (2014). Deeppose: Human pose estimation via deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1653-1660).
14. [14] https://nanonets.com/imagerecognition?&utm_source=nanonets.com%2Fblog%2F&utm_medium=blog&utm_content=How%20to%20Classify%20Fashion%20Images%20easily%20using%20ConvNets
15. [15] Sethi, S., Kathuria, M., & Kaushik, T. (2021). Face mask detection using deep learning: An approach to reduce risk of Coronavirus spread. Journal of Biomedical Informatics, 120, 103848.
16. [16] Rosebrock, A. (2020). Covid-19: Face mask detector with opencv, keras/tensorflow, and deep learning. Link: https://www. pyimagesearch. com/2020/05/04/covid-19-face-mask-detector-withopencv-keras-tensorflow-and-deeplearning.
17. [17] https://github.com/prajnasb/observations
18. [18] Newell, A., Yang, K., & Deng, J. (2016, October). Stacked hourglass networks for human pose estimation. In European conference on computer vision (pp. 483-499). Springer, Cham.
19. [19] Quan Hua, “Human Pose Estimation in OpenCv” Link: human-pose-estimation-opencv/LICENSE at master · quanhua92/human-pose-estimation-opencv · GitHub
20. [20] Pons-Moll12, G., Taylor13, J., Shotton, J., Hertzmann14, A., & Fitzgibbon, A. (2013). Metric regression forests for human pose estimation. BMVC.
21. [21] Andriluka, M., Iqbal, U., Insafutdinov, E., Pishchulin, L., Milan, A., Gall, J., & Schiele, B. (2018). Posetrack: A benchmark for human pose estimation and tracking. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5167-5176).
22. [22] Andriluka, M., Iqbal, U., Insafutdinov, E., Pishchulin, L., Milan, A., Gall, J., & Schiele, B. (2018). Posetrack: A benchmark for human pose estimation and tracking. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5167-5176).
23. [23] Wang, J., & Payandeh, S. (2017). Hand motion and posture recognition in a network of calibrated cameras. Advances in Multimedia, 2017.
24. [24] Remelli, E., Han, S., Honari, S., Fua, P., & Wang, R. (2020). Lightweight multi-view 3d pose estimation through camera-disentangled representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 6040-6049).
25. [25] Zhao, M., Li, T., Abu Alsheikh, M., Tian, Y., Zhao, H., Torralba, A., & Katabi, D. (2018). Through-wall human pose estimation using radio signals. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 7356-7365).
26. [26] Sarafianos, N., Boteanu, B., Ionescu, B., & Kakadiaris, I. A. (2016). 3d human pose estimation: A review of the literature and analysis of covariates. Computer Vision and Image Understanding, 152, 1-20.
27. [27] Rogez, G., & Schmid, C. (2016). Mocap-guided data augmentation for 3d pose estimation in the wild. arXiv preprint arXiv:1607.02046.
28. [28] Chen, W., Wang, H., Li, Y., Su, H., Wang, Z., Tu, C., ... & Chen, B. (2016, October). Synthesizing training images for boosting human 3d pose estimation. In 2016 Fourth International Conference on 3D Vision (3DV) (pp. 479-488). IEEE.
29. [29] Fabbri, M., Lanzi, F., Calderara, S., Alletto, S., & Cucchiara, R. (2020). Compressed volumetric heatmaps for multi-person 3d pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 7204-7213).
30. [30] Liu, R. (2019). Attention Based Temporal Convolutional Neural Network for Real-Time 3D Human Pose Reconstruction. University of Dayton.
31. [31] Cao, Z., Hidalgo, G., Simon, T., Wei, S. E., & Sheikh, Y. (2019). OpenPose: realtime multi-person 2D pose estimation using Part Affinity Fields. IEEE transactions on pattern analysis and machine intelligence, 43(1), 172-186.
32. [32] Kang, X., Song, B., & Sun, F. (2019). A deep similarity metric method based on incomplete data for traffic anomaly detection in IoT. Applied Sciences, 9(1), 135.
33. [33] Sengupta, A., Budvytis, I., & Cipolla, R. (2020). Synthetic training for accurate 3d human pose and shape estimation in the wild. arXiv preprint arXiv:2009.10013.
34. [34] Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., & Tian, Q. (2019). Centernet: Keypoint triplets for object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 6569-6578).
35. [35] Sovit Ranjan Rath, et, al. “Human Pose Detection using PyTorch Keypoint RCNN.” Machine Learning and Deep Learning , 2020
36. [36] Sarafianos, N., Boteanu, B., Ionescu, B., & Kakadiaris, I. A. (2016). 3d human pose estimation: A review of the literature and analysis of covariates. Computer Vision and Image Understanding, 152, 1-20.