Classification of Violent Activities with Optical Flow Image and Bi-Lstm

Abstract

The need for automated motion recognition systems is increasing due to the rapid increase in the number of security cameras. Although motion recognition is a hot topic in the field of computer vision, the classification of violent scenes is of great importance due to its relation to human and community safety. Optical flow is often used in the detection and modeling of motion in video images. In this study, a method that can be used to recognize violent activities using optical flow and deep learning has been proposed. The components of the optical flow series of a video series were combined into a 3-channel image and pre-trained VGG-16 was input into the convulsive neural network. A Bi-Lstm (Bidirectional long short term memory) classifier has been trained with the deep quality series derived from the VGG-16 network. The proposed method was tested with two different data sets in the literature and comparable and higher classifying results were obtained.

Keywords

• Optical Flow,CNN,Deep Learning,Bi-Lstm

Optik Akış Görüntüsü ve Bi-Lstm ile Şiddet İçeren Hareketlerin Sınıflandırılması

Öz

Otomatik hareket tanıma sistemlerine ihtiyaç, güvenlik kameralarının sayısındaki hızlı artıştan dolayı giderek artmaktadır. Hareket tanıma, bilgisayarlı görü alanında güncel bir araştırma alanı olmasına karşın şiddet içeren sahnelerin tespiti insan ve toplum güvenliğiyle de ilişkili olması sebebiyle büyük önem taşımaktadır. Optik akış video görüntülerindeki hareketlerin tespit ve modellenmesinde sıklıkla kullanılan bir yaklaşımdır. Bu çalışmada optik akış ve derin öğrenme kullanılarak şiddet içeren aktivitelerin tanınmasında kullanılabilecek bir yöntem önerilmiştir. Bir video serisine ait optik akış serisinin bileşenleri birleştirilerek üç kanallı bir görüntü haline getirilmiş ve önceden eğitilmiş VGG-16 evrişimsel (convolutional) sinir ağına girdi olarak verilmiştir. VGG-16 ağından elde edilen derin nitelik serileri ile bir Bi-Lstm (Bidirectional long short term memory) sınıflayıcısı eğitilmiştir. Önerilen yöntem literatürde yer alan iki farklı veri kümesi ile test edilmiş ve literatürde yer alan diğer yaklaşımlar ile karşılattırılabilir ve daha yüksek sınıflama başarımına sahip sonuçlar elde edilmiştir.  

Anahtar Kelimeler

• Optik Akış,CNN,Derin Öğrenme,Bi-Lstm,Bi-Lstm

Kaynakça

1. Nam J, Alghoniemy M, Tewfik AH Audio-visual content-based violent scene characterization. In: Image Processing, 1998. ICIP 98. Proceedings. 1998 International Conference on, 1998. IEEE, pp 353-357
2. Clarin C, Dionisio J, Echavez M, Naval P (2005) DOVE: Detection of movie violence using motion intensity analysis on skin and blood. PCSC 6:150-156
3. Gong Y, Wang W, Jiang S, Huang Q, Gao W Detecting violent scenes in movies by auditory and visual cues. In: Pacific-Rim Conference on Multimedia, 2008. Springer, pp 317-326
4. Lin J, Wang W Weakly-supervised violence detection in movies with audio and video based co-training. In: Pacific-Rim Conference on Multimedia, 2009. Springer, pp 930-935
5. Kooij JF, Liem M, Krijnders JD, Andringa TC, Gavrila DM (2016) Multi-modal human aggression detection. Computer Vision and Image Understanding 144:106-120
6. Hassner T, Itcher Y, Kliper-Gross O Violent flows: Real-time detection of violent crowd behavior. In: Computer Vision and Pattern Recognition Workshops (CVPRW), 2012 IEEE Computer Society Conference on, 2012. IEEE, pp 1-6
7. Gao Y, Liu H, Sun X, Wang C, Liu Y (2016) Violence detection using Oriented VIolent Flows. Image and Vision Computing 48:37-41
8. Rota P, Conci N, Sebe N, Rehg JM Real-life violent social interaction detection. In: Image Processing (ICIP), 2015 IEEE International Conference on, 2015. IEEE, pp 3456-3460

9. Lloyd K, Marshall D, Moore SC, Rosin PL (2016) Detecting Violent Crowds using Temporal Analysis of GLCM Texture. arXiv preprint arXiv:160505106
10. Arceda VM, Ferna K, Guti J (2016) Real time violence detection in video.
11. Dai Q, Zhao R-W, Wu Z, Wang X, Gu Z, Wu W, Jiang Y-G Fudan-Huawei at MediaEval 2015: Detecting Violent Scenes and Affective Impact in Movies with Deep Learning. In: MediaEval, 2015.
12. Bruhn A, Weickert J, Schnörr C (2005) Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods. Int J Comput Vision 61 (3):211-231
13. Schuster M, Paliwal KK (1997) Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing 45 (11):2673-2681
14. Keçeli AS, Keçeli SU, Kaya A Classification of radiolarian fossil images with deep learning methods. In: 2018 26th Signal Processing and Communications Applications Conference (SIU), 2018. IEEE,
15. Yeffet L, Wolf L Local trinary patterns for human action recognition. In: Computer Vision, 2009 IEEE 12th International Conference on, 2009. IEEE, pp 492-497