Comparison of Deep Learning Based 2D Human Pose Estimation Models

Year 2024, Volume: 7 Issue: 2, 185 - 196, 14.12.2024

Cumhur Torun , Abdulkadir Karacı

https://doi.org/10.51764/smutgd.1573626

Abstract

Human motion analysis is a field of study of great importance in computer vision and artificial intelligence. In particular, the detection of joint points plays a critical role in digitally modeling human movements and postures. This field has many applications in many disciplines such as medicine, sports, rehabilitation, security, and human-computer interaction. By accurately and efficiently identifying joint points, it is possible to evaluate athletes' performance, monitor patients' rehabilitation process, and recognize complex gestures such as sign language in a digital environment. Various software libraries developed to detect joint points offer advantages in terms of speed, accuracy, and ease of use by using different algorithms. Popular libraries such as MediaPipe, MoveNet, OpenPose, AlphaPose, Detectron2, and HRNet are widely used in this field, and each has certain advantages and disadvantages. In this study, libraries are compared and compared for analyzing human movement. evaluations of the methods used and the areas where they can be used It was done. MediaPipe Holistic and MoveNet libraries were found to be successful in real-time applications, whereas AlphaPose, ViTPose, and HRNet were found to be more effective for applications requiring high accuracy.

Keywords

Human motion analysis, joint point detection, computer vision, pose estimation models

Derin Öğrenmeye Dayalı 2 Boyutlu İnsan Poz Tahmin Modellerinin Karşılaştırılması

Abstract

İnsan hareketlerinin analizi, bilgisayarlı görü ve yapay zekâ alanlarında büyük öneme sahip bir çalışma alanıdır. Özellikle eklem noktalarının tespiti, insan hareketlerinin ve duruşlarının dijital ortamda modellenmesi açısından kritik rol oynar. Bu alan, tıp, spor, rehabilitasyon, güvenlik, insan-bilgisayar etkileşimi gibi birçok disiplinde geniş bir kullanım alanına sahiptir. Eklem noktalarının doğru ve etkin bir şekilde belirlenmesi sayesinde, sporcuların performans değerlendirilmesi, hastaların rehabilitasyon süreçlerinin izlenmesi ve işaret dili gibi karmaşık hareketlerin dijital ortamda tanınması sağlanabilmektedir. Eklem noktalarını tespit etmek için geliştirilen çeşitli yazılım kütüphaneleri, farklı algoritmalar kullanarak hız, doğruluk ve kullanım kolaylığı açısından avantajlar sunmaktadır. MediaPipe, MoveNet, OpenPose, AlphaPose, Detectron2 ve HRNet gibi popüler kütüphaneler, bu alanda yaygın olarak kullanılmakta ve her birinin belirli avantaj ve dezavantajları bulunmaktadır. Bu çalışmada, kütüphaneler karşılaştırılarak insan hareketlerinin analizinde kullanılan yöntemler ve kullanılabilecekleri alanlara yönelik değerlendirmeler yapılmıştır. MediaPipe Holistic ve MoveNet kütüphaneleri gerçek zamanlı uygulamalarda başarılı bulunurken, AlphaPose, ViTPose ve HRNet'in yüksek doğruluk gerektiren uygulamalar için daha etkili olduğu görülmüştür.

Keywords

İnsan hareket analizi, eklem noktası tespiti, bilgisayarlı görü, poz tahmin modelleri

Ethical Statement

Bu çalışmanın, özgün bir çalışma olduğunu; çalışmanın hazırlık, veri toplama, analiz ve bilgilerin sunumu olmak üzere tüm aşamalarından bilimsel etik ilke ve kurallarına uygun davrandığımı; bu çalışma kapsamında elde edilmeyen tüm veri ve bilgiler için kaynak gösterdiğimi ve bu kaynaklara kaynakçada yer verdiğimi; kullanılan verilerde herhangi bir değişiklik yapmadığımı, çalışmanın Committee on Publication Ethics (COPE)' in tüm şartlarını ve koşullarını kabul ederek etik görev ve sorumluluklara riayet ettiğimi beyan ederim. Herhangi bir zamanda, çalışmayla ilgili yaptığım bu beyana aykırı bir durumun saptanması durumunda, ortaya çıkacak tüm ahlaki ve hukuki sonuçlara razı olduğumu bildiririm.

Supporting Institution

TÜBİTAK

Thanks

Bu çalışma, Türkiye Bilimsel ve Teknolojik Araştırma Kurumu (TÜBİTAK) tarafından 124E379 numaralı hibe kapsamında desteklenmiştir. Yazarlar destekleri için TÜBİTAK'a teşekkürlerini sunarlar.

References

• Abu Awwad, Y., Rana, O., & Perera, C. (2024). Anomaly detection on the edge using smart cameras under low-light conditions. Sensors, 24(3), 772.
• Areerob, P., Matangkasombut, T., Monnikhof, K. O., & Kumwilaisak, W. (2024, May). Crowded Scene PPE Detection Using Attention Based YOLOv7 and Alpha Pose. In 2024 21st International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON) (pp. 1-6). IEEE.
• Arkushin, R. S., Moryossef, A., & Fried, O. (2023). Ham2pose: Animating sign language notation into pose sequences. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 21046-21056).
• Anuj Shah. (2024, Eylül 10). Posetrack data set summary. https://medium.com/@anuj_shah/posetrack-data-set-summary-9cf61fc6f44e
• Badiola-Bengoa, A., & Mendez-Zorrilla, A. (2021). A systematic review of the application of camera-based human pose estimation in the field of sport and physical exercise. Sensors, 21(18), 5996.
• Bao, W., Niu, T., Wang, N., & Yang, X. (2023). Pose estimation and motion analysis of ski jumpers based on ECA-HRNet. Scientific Reports, 13(1), 6132.
• Bibin Sebastian. (2024, Eylül 10). Human Action Recognition using Detectron2 and LSTM. https://learnopencv.com/human-action-recognition-using-detectron2-and-lstm/
• Bora, J., Dehingia, S., Boruah, A., Chetia, A. A., & Gogoi, D. (2023). Real-time assamese sign language recognition using mediapipe and deep learning. Procedia Computer Science, 218, 1384-1393.
• Cao, Z., Simon, T., Wei, S. E., & Sheikh, Y. (2017). Realtime multi-person 2d pose estimation using part affinity fields. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7291-7299).
• Chatterjee, R., Roy, S., Islam, S. H., & Samanta, D. (2021, August). An AI Approach to Pose-based Sports Activity Classification. In 2021 8th International Conference on Signal Processing and Integrated Networks (SPIN) (pp. 156-161). IEEE.
• Chen, L., & Fisher, R. B. (2024). Miso: Monitoring inactivity of single older adults at home using rgb-d technology. ACM Transactions on Computing for Healthcare, 5(3), 1-19.
• Chen, M., & Tan, G. (2024, July). FANpose: 2D human pose estimation with fully attentional networks under vision transformer baselines. In Third International Symposium on Computer Applications and Information Systems (ISCAIS 2024) (Vol. 13210, pp. 879-884). SPIE.
• Davoudi Kashkoli, M., Javied, A., Barrera-Animas, A. Y., & Davila Delgado, J. M. (2024). A synthetic data approach for object detection in super low resolution images. Proceedings of the 2024 International Conference on Innovation in Artificial Intelligence (ICIAI '24), 86–91.
• Dill, S., Rösch, A., Rohr, M., Güney, G., De Witte, L., Schwartz, E., & Hoog Antink, C. (2023, September). Accuracy Evaluation of 3D Pose Estimation with MediaPipe Pose for Physical Exercises. In Current Directions in Biomedical Engineering (Vol. 9, No. 1, pp. 563-566). De Gruyter.
• Fang, H. S., Li, J., Tang, H., Xu, C., Zhu, H., Xiu, Y., ... & Lu, C. (2022). Alphapose: Whole-body regional multi-person pose estimation and tracking in real-time. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6), 7157-7173.
• Fang, Y., Han, Z., Hu, Z., & Wang, Z. (2021, December). Human Posture Estimation. In 2021 3rd International Conference on Machine Learning, Big Data and Business Intelligence (MLBDBI) (pp. 220-234). IEEE.
• Garg, S., Saxena, A., & Gupta, R. (2023). Yoga pose classification: a CNN and MediaPipe inspired deep learning approach for real-world application. Journal of Ambient Intelligence and Humanized Computing, 14(12), 16551-16562.
• Gao, Q., Liu, J., Ju, Z., & Zhang, X. (2019). Dual-hand detection for human–robot interaction by a parallel network based on hand detection and body pose estimation. IEEE Transactions on Industrial Electronics, 66(12), 9663-9672.
• Gineshidalgo. (2024, Ekim 20). OpenPose: Real-time multi-person keypoint detection library for body, face, hands, and foot estimation. https://github.com/CMU-Perceptual-Computing-Lab/openpose
• Google AI. (2024, Eylül 10). MediaPipe Solutions guide. https://ai.google.dev/edge/mediapipe/solutions/guide
• Grover, A., Arora, D., & Grover, A. (2022, December). Human pose estimation using deep learning techniques. In Proceedings of the 4th International Conference on Information Management & Machine Intelligence (pp. 1-6).
• Hernández, Ó. G., Morell, V., Ramon, J. L., & Jara, C. A. (2021). Human pose detection for robotic-assisted and rehabilitation environments. Applied Sciences, 11(9), 4183.
• Hu, M., Zhang, M., & Yu, K. (2024). Design of sports training information analysis system based on a multi-target visual model under sensor-scale spatial transformation. PeerJ Computer Science, 10, e2030.
• Jafarzadeh, P., Virjonen, P., Nevalainen, P., Farahnakian, F., & Heikkonen, J. (2021, October). Pose estimation of hurdles athletes using openpose. In 2021 International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME) (pp. 1-6). IEEE.
• Jo, B., & Kim, S. (2022). Comparative analysis of OpenPose, PoseNet, and MoveNet models for pose estimation in mobile devices. Traitement du Signal, 39(1), 119.
• Karacı, A., Akyol, K., & Turut, M. U. (2021). Real-Time Turkish Sign Language Recognition Using Cascade Voting Approach with Handcrafted Features. Applied Computer Systems, 26(1), 12-21.
• Kim, J. W., Choi, J. Y., Ha, E. J., & Choi, J. H. (2023). Human pose estimation using mediapipe pose and optimization method based on a humanoid model. Applied sciences, 13(4), 2700.
• Lee, H., Oh, B., & Kim, S. C. (2024). Recognition of Forward Head Posture Through 3D Human Pose Estimation With a Graph Convolutional Network: Development and Feasibility Study. JMIR Formative Research, 8(1), e55476.
• Li, R., Yan, A., Yang, S., He, D., Zeng, X., & Liu, H. (2024). Human Pose Estimation Based on Efficient and Lightweight High-Resolution Network (EL-HRNet). Sensors, 24(2).
• Li, N., Wang, Y., Liu, F., & Huang, W. (2024, August). Real-time multitarget fall detection based on OpenPose. In Seventh International Conference on Advanced Electronic Materials, Computers, and Software Engineering (AEMCSE 2024) (Vol. 13229, pp. 751-755). SPIE.
• Lin, H., Chen, H., & Lin, J. (2024). Deep neural network uncertainty estimation for early oral cancer diagnosis. Journal of Oral Pathology & Medicine, 53(5), 294-302.
• Lyttonhao. (2024, Ekim 15). Detectron2. https://ai.meta.com/tools/detectron2/
• Mishra, A. K., Sahoo, D., Shubhankar, I., & Samal, I. YogaSiddhi: AI-Powered Pose Analysis using MoveNet for Yoga Refinement. International Journal of Computer Applications, 975, 8887.
• Parashar, D., Mishra, O., Sharma, K., & Kukker, A. (2023). Improved Yoga Pose Detection Using MediaPipe and MoveNet in a Deep Learning Model. Revue d'Intelligence Artificielle, 37(5).
• Parle, A., Shinde, R., Chougule, R., & Agrawal, S. (2024, April). YogaWise: Enhancing Yoga with Intelligent Real Time Tracking using TensorFlow MoveNet. In 2024 International Conference on Cognitive Robotics and Intelligent Systems (ICC-ROBINS) (pp. 498-505). IEEE.
• Singh, A. K., Kumbhare, V. A., & Arthi, K. (2021, June). Real-time human pose detection and recognition using mediapipe. In International conference on soft computing and signal processing (pp. 145-154). Singapore: Springer Nature Singapore.
• Sun, K., Xiao, B., Liu, D., & Wang, J. (2019). Deep high-resolution representation learning for human pose estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 5693-5703).
• Song, H., Li, Y., Fu, C., Xue, F., Zhao, Q., Zheng, X., ... & Liu, T. (2024). Using complex networks and multiple artificial intelligence algorithms for table tennis match action recognition and technical-tactical analysis. Chaos, Solitons & Fractals, 178, 114343.
• TensorFlow. (2024, Ekim 1). MoveNet: Ultra fast and accurate pose detection model. https://www.tensorflow.org/hub/tutorials/movenet?hl=tr
• Üstek, İ., Desai, J., Torrecillas, I. L., Abadou, S., Wang, J., Fever, Q., ... & Tsourdos, A. (2023, August). Two-Stage Violence Detection Using ViTPose and Classification Models at Smart Airports. In 2023 IEEE Smart World Congress (SWC) (pp. 797-802). IEEE.
• Wang, J., Sun, K., Cheng, T., Jiang, B., Deng, C., Zhao, Y., ... & Xiao, B. (2020). Deep high-resolution representation learning for visual recognition. IEEE transactions on pattern analysis and machine intelligence, 43(10), 3349-3364.
• Wang, Y., Wang, R., Shi, H., & Liu, D. (2024). MS-HRNet: multi-scale high-resolution network for human pose estimation. The Journal of Supercomputing, 1-23.
• Wu, M. Y., Ting, P. W., Tang, Y. H., Chou, E. T., & Fu, L. C. (2020). Hand pose estimation in object-interaction based on deep learning for virtual reality applications. Journal of Visual Communication and Image Representation, 70, 102802.
• Wu, Q., Xu, G., Zhang, S., Li, Y., & Wei, F. (2020, July). Human 3D pose estimation in a lying position by RGB-D images for medical diagnosis and rehabilitation. In 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (pp. 5802-5805). IEEE.
• Xu, Y., Zhang, J., Zhang, Q., & Tao, D. (2022). Vitpose: Simple vision transformer baselines for human pose estimation. Advances in Neural Information Processing Systems, 35, 38571-38584.
• Xu, Y., Zhang, J., Zhang, Q., & Tao, D. (2023). Vitpose++: Vision transformer for generic body pose estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence.
• Yu, N., Ma, T., Zhang, J., Zhang, Y., Bao, Q., Wei, X., & Yang, X. (2024, October). Adaptive Vision Transformer for Event-Based Human Pose Estimation. In Proceedings of the 32nd ACM International Conference on Multimedia (pp. 2833-2841).
• Zhang, F., Juneau, P., McGuirk, C., Tu, A., Cheung, K., Baddour, N., & Lemaire, E. (2021, June). Comparison of OpenPose and HyperPose artificial intelligence models for analysis of hand-held smartphone videos. In 2021 IEEE International Symposium on Medical Measurements and Applications (MeMeA) (pp. 1-6). IEEE.
• Zhang, M., Zhou, Y., Xu, X., Ren, Z., Zhang, Y., Liu, S., & Luo, W. (2023). Multi-view emotional expressions dataset using 2D pose estimation. Scientific Data, 10(1), 649.
• Zhou, Y., Wang, X., Xu, X., Zhao, L., & Song, J. (2022, July). X-hrnet: Towards lightweight human pose estimation with spatially unidimensional self-attention. In 2022 IEEE international conference on multimedia and expo (ICME) (pp. 01-06). IEEE.
• Zhu, H., Jie, C., & Jiang, S. (2020). Multi-Person Full Body Pose Estimation. arXiv preprint arXiv:2008.10060.