Armed Activity Recognition using Pose-Based Features with Machine Learning and Deep Learning

Abstract

Automatic detection of "armed" activities commonly focuses on the detection of weapons, with the limitations of variety in shapes, types and colours of weapons. This study aims to perform AI-based identification of armed and unarmed activities using postural landmark features of the human body, without requiring weapon visibility. We conducted a comparative analysis on the postures by MediaPipe and YOLOv8-pose frameworks and their impact on the performance of conventional machine learning and deep learning classifiers. Posture landmarks were extracted separately using both frameworks on a total number of 3,866 images (1,934 armed and 1,932 unarmed activities). Conventional machine learning algorithms and Long Short-Term Memory (LSTM) algorithms were trained without data augmentation on GridSearchCV and the early stopping mechanism. The experimental results showed that the YOLOv8-pose-based posture feature landmarks provide higher armed pose activity classification performance than the MediaPipe-based landmarks. The optimized SVM trained on the YOLOv8-pose feature set achieved the highest test accuracy rate of 94.2%, whereas the most successful deep learning model, the YOLOv8-pose-based Bi-LSTM, reached an accuracy rate of 93.9%. Consequently, the findings demonstrate that feature set responsibility can outweigh model complexity of sequence learning-based algorithms in "armed pose" detection for even a lightweight SVM algorithm.

Keywords

• Armed Pose Detection
• deep learning
• gesture recognition
• machine learning

Makine Öğrenmesi ve Derin Öğrenme Kullanarak Duruş Tabanlı Özniteliklerle Silahlı Aktivite Tanıma

Abstract

"Silahlı" faaliyetlerin otomatik belirlenmesi genellikle silahların tespitine odaklanır; ancak bu, silahların şekil, tür ve renk çeşitliliğine bağlı sınırlılıklara sahiptir. Bu çalışma, silah görünürlüğüne gerek duymadan, insan vücudunun duruş referans noktalarını kullanarak silahlı ve silahsız faaliyetlerin yapay zeka tabanlı tanımlanmasını amaçlamaktadır. MediaPipe ve YOLOv8-pose kütüphaneleri ile elde edilen duruşlar ve bunların geleneksel makine öğrenimi ve derin öğrenme sınıflandırıcılarının performansına etkileri üzerinde karşılaştırmalı bir analiz gerçekleştirdik. Duruş referans noktaları, toplam 3.866 görüntü (1.934 silahlı ve 1.932 silahsız faaliyet) üzerinde her iki kütüphane kullanılarak ayrı ayrı çıkarıldı. Geleneksel makine öğrenimi algoritmaları ve Uzun Kısa Süreli Bellek (LSTM) algoritmaları, GridSearchCV ve erken durdurma mekanizması kullanılarak veri artırma yapılmadan eğitildi. Deneysel sonuçlar, YOLOv8-pose tabanlı duruş özelliği referans noktalarının, MediaPipe tabanlı referans noktalarına göre daha yüksek silahlı duruş faaliyeti sınıflandırma performansı sağladığını göstermiştir. Optimize edilmiş SVM, YOLOv8-poz özellik kümesi üzerinde eğitilerek %94,2 ile en yüksek test doğruluk oranına ulaşırken, en başarılı derin öğrenme modeli olan YOLOv8-poz tabanlı Bi-LSTM, %93,9 doğruluk oranına ulaşmıştır. Sonuç olarak, bulgular, hafif bir SVM algoritması için bile "silahlı poz" tespitinde öznitelik kümesi sorumluluğunun, dizi öğrenmeye dayalı algoritmaların model karmaşıklığından daha önemli olabileceğini göstermektedir.

Keywords

• Silahlı Poz Tespiti
• derin öğrenme
• duruş tanıma
• makine öğrenmesi

References

1. Abruzzo B, Carey K, Lowrance C, Sturzinger E, Arnold R, Korpela C. Cascaded neural networks for identification and posture-based threat assessment of armed people. In: IEEE 2019 Int Symp Technol Homel Secur (HST); 05-06 November 2019; Woburn, MA, USA: IEEE. pp.1-7.
2. Salido J, Lomas V, Ruiz-Santaquiteria J, Deniz O. Automatic handgun detection with deep learning in video surveillance images. Appl Sci 2021;11(13):6085.
3. Ruiz-Santaquiteria J, Velasco-Mata A, Vallez N, Bueno G, Alvarez-Garcia JA, Deniz O. Handgun detection using combined human pose and weapon appearance. IEEE Access 2021;9:123815-123826.
4. Lamas A, Tabik S, Montes AC, Pérez-Hernández F, Garcia J, Olmos R, Herrera F. Human pose estimation for mitigating false negatives in weapon detection in video-surveillance. Neurocomputing 2022;489:488-503.
5. Amado-Garfias AJ, Conant-Pablos SE, Ortiz-Bayliss JC, Terashima-Marín H. Improving armed people detection on video surveillance through heuristics and machine learning models. IEEE Access 2024;12:20543-20556.
6. Singh H, Deniz O, Ruiz-Santaquiteria J, Muñoz JD, Bueno G. DeepGun: Deep feature-driven one-class classifier for firearm detection using visual gun features and human body pose estimation. Appl Sci 2025;15(11):5830.
7. Ruiz-Santaquiteria J, Velasco-Mata A, Vallez N, Deniz O, Bueno G. Improving handgun detection through a combination of visual features and body pose-based data. Pattern Recognit 2023;136:109252.
8. Hussain S, Siddiqui HUR, Saleem AA, Raza MA, Alemany-Iturriaga J, Velarde-Sotres Á, et al. Smart Physiotherapy: Advancing arm-based exercise classification with PoseNet and ensemble models. Sensors 2024;24(19):6325.

9. Sarswat D. Real-time AI driven shooting posture assessment and correction for professional and military training using machine learning, OpenCV, and MediaPipe. In: 2024 4th Int Conf Technol Adv Comput Sci (ICTACS);13-15 November 2024; Tashkent, Uzbekistan. pp.1192-1198.
10. Bhatt A, Ganatra A. Explosive weapons and arms detection with singular classification (WARDIC) on novel weapon dataset using deep learning: Enhanced OODA loop. Eng Sci 2022;20(3):252-266.
11. Maligireddy AR, Uppula MR, Rastogi N, Parla YR. Gun detection using combined human pose and weapon appearance. arXiv preprint arXiv:2503.12215, 2025.
12. Bhatt A, Ganatra A. Weapon operating pose detection and suspicious human activity classification using skeleton graphs. Math Biosci Eng 2023;20(2):2669-2690.
13. Li Z, Song X, Chen S, Demachi K. Armed boundary sabotage: A case study of human malicious behaviors identification with computer vision and explainable reasoning methods. Comput Electr Eng 2025;121:109924.
14. Widodo YB, Sibuea S, Agustino R. YOLO in Suspicious Human Activity Recognition for Intelligent Environmental Security Systems: A Review. Jurnal Teknologi Informatika dan Komputer, 2026:12(1): 101-119.
15. Toshev A, Szegedy C. DeepPose: Human pose estimation via deep neural networks. Proc IEEE Conf Comput Vis Pattern Recognit (CVPR) 2014:1653-1660.
16. Rajeti RS, Jyothirmayi T. Object detection using FRCNN with VGG-19. In AIP Conference Proceedings 2026:3345(1): 020257.
17. Eneh A, Ochogwu RE, Ebem D. A Framework for Detection and Recognition of Armed Persons Using Convolutional Neural Networks. Scholar J Computational Science, 2025:2(2):64-72.
18. Palani D, Kumar MV, Pushpakumari AK, Chowdary AN, Reddy IPK, Chidambaram V. Theft detection with computer vision technique using Yolo-V7 algorithm. AIP Conference Proceedings 2025;3175(1):020003.
19. Dphi. Data Sprint 76: Human Action Recognition (HAR) Dataset [Internet]. Kaggle; 2023 [Accessed: Jan 23, 2026]. Available from: https://www.kaggle.com/datasets/meetnagadia/human-action-recognition-har-dataset
20. Google. MediaPipe Pose Landmarker task [Internet]. Google for Developers; 2024 [Accessed: Jan 23, 2026]. Available from: https://ai.google.dev/edge/mediapipe/solutions/vision/pose_landmarker
21. Ultralytics. Pose estimation with Ultralytics YOLOv8 [Internet]. Ultralytics Blog; 2024 [Accessed: Jan 23, 2026]. Available from: https://www.ultralytics.com/blog/pose-estimation-with-ultralytics-yolov8
22. Dong C, Du G. An enhanced real-time human pose estimation method based on modified YOLOv8 framework. Sci Rep 2024;14(1):8012.
23. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, et al. Scikit-learn: Machine learning in Python. J Mach Learn Res 2011;12:2825-2830.
24. Moosaei H, Ketabchi S, Razzaghi M, Tanveer M. Generalized twin support vector machines. Neural Process Lett 2021;53(2):1545-1564.
25. Rumelhart DE, Hinton GE, Williams RJ. Learning representations by back-propagating errors. Nature 1986;323(6088):533-536.
26. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput 1997;9(8):1735-1780.
27. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: A simple way to prevent neural networks from overfitting. J Mach Learn Res 2014;15(1):1929-1958.
28. Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
29. Bishop CM, Nasrabadi NM. Pattern Recognition and Machine Learning. New York, NY, USA: Springer, 2006.
30. Cortes C, Vapnik V. Support-vector networks. Mach Learn 1995;20:273-297.
31. Jo B, Kim S. Comparative analysis of OpenPose, PoseNet, and MoveNet models for pose estimation in mobile devices. Trait Signal 2022;39(1):119-126.