Intelligent Video Surveillance System Using Faster Regional Convolutional Neural Networks

Abstract

Insecurity remains a major challenge in our society. Government, private organizations, and individuals strive to ensure their possessions are kept safe from intruders. Automated surveillance system plays a key role to ensure that the environment is safe with little human intervention. Therefore, object detection, classification, and tracking are vital in building a robust and remote intelligent video surveillance system to aid security in physical environments. Previous studies used enhanced background subtraction techniques for object detection which recorded notable achievements but performance issues in distinguishing humans, pets and vehicles. For insecurity to be solved more intelligently, deep neural network techniques are employed. In this paper, an intelligent video surveillance system that detects only human intrusion and sends an SMS notification to the user with the registered mobile number was developed. The results of the system performance evaluation recorded an accuracy of 96%, a precision of 94%, and a recall of 98%. The experimental results showed that the intelligent system was suitable for detecting human intrusion, thereby contributing to the safety of physical environments.

Keywords

• Video
• surveillance
• CNN
• deep learning
• Detection

References

1. [1] Y. Kurylyak, “A Real-Time Motion Detection for Video Surveillance System,”. IEEE International Workshop on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, Rende, Italy, 2009, pp. 386-389, doi: 10.1109/IDAACS.2009.5342954.
2. [2] S. W. Ibrahim, “A comprehensive review on intelligent surveillance systems”, CST, vol. 1, no. 1, pp 7-14, May 2016
3. [3] O.M. Olaniyi, S. Ganiyu and S. J. Akam. Intelligent Video Surveillance Systems: A Survey. Balkan Journal of Electrical and Computer Engineering (BAJECE).1(1).pp 57-53
4. [4] A. A. Shafie, F. Hafizhelmi, and K. Zaman, “Smart Video Surveillance System for Vehicle Detection and Traffic Flow Control”. Journal of Engineering Science & Technology (JESTEC). Vol.13 no 7. 2195-2210
5. [5] B. Benjdira, T. Khursheed, A. Koubaa, A. Ammar, and K. Ouni, “Car Detection using Unmanned Aerial Vehicles: Comparison between Faster R-CNN and YOLOv3,” 2019 1st Int. Conf. Unmanned Veh. Syst., pp. 1–6, 2019
6. [6] W. Tan, “Object Detection with Multi-RCNN Detectors,” pp. 193–197.
7. [7] A. H. Sanoob, J. Roselin, and P. Latha, “Smartphone Enabled Intelligent Surveillance System,” no. c, pp. 1–7, 2015, doi: 10.1109/JSEN.2015.2501407.
8. [8] L. W. Yang and C. Y. Su, “Low-cost CNN Design for Intelligent Surveillance System,” 2018 Int. Conf. Syst. Sci. Eng., pp. 1–4, doi: 10.1109/ICSSE.2018.8520133.

9. [9] . M. Olaniyi, J. A. Bala, S. O. Ganiyu, and P. E. Wisdom, “A Systematic Review of Background Subtraction Algorithms for Smart Surveillance System,” vol. 8, no. 1, pp. 35–54, 2020
10. [10] C. Jin, S. Li, and H. Kim, “Real-Time Action Detection in Video Surveillance using Sub-Action Descriptor with Multi-CNN,” pp. 1–29.
11. [11] A. Ullah, K. Muhammad, J. Del Ser, S. W. Baik, and V. Albuquerque, “Activity Recognition using Temporal Optical Flow Convolutional Features and Multi-Layer LSTM,” IEEE Trans. Ind. Electron., vol. PP, no. c, p. 1, 2018, doi: 10.1109/TIE.2018.2881943.
12. [12] H. Kaya, H. Dibeklio, and A. A. Salah, “Kernel ELM and CNN based Facial Age Estimation,” pp. 80–86.
13. [13] A. Antoniou, “A General Purpose Intelligent Surveillance System For Mobile Devices using Deep Learning,” pp. 2879–2886, 2016
14. [14] . Muhammad, S. Khan, S. Member, and V. Palade, “Edge Intelligence-Assisted Smoke Detection in,” IEEE Trans. Ind. Informatics, vol. PP, no. c, p. 1, 2019, doi: 10.1109/TII.2019.2915592
15. [15] . Hargude and M. T. It, “i-surveillance: Intelligent Surveillance System Using Background Subtraction Technique,” vol. 1.
16. [16] C. Gao, P. Li, Y. Zhang, J. Liu, and L. Wang, “Author ’ s Accepted Manuscript People counting based on head detection combining environment Reference : To appear in : Neurocomputing,” Neurocomputing, 2016, doi: 10.1016/j.neucom.2016.01.097.
17. [17] . Y. Nikouei, Y. Chen, S. Song, R. Xu, B. Y. Choi, and T. Faughnan, “Smart surveillance as an edge network service: From harr-cascade, SVM to a Lightweight CNN,” Proc. - 4th IEEE Int. Conf. Collab. Internet Comput. CIC 2018, pp. 256–265, 2018, doi: 10.1109/CIC.2018.00042.
18. [18] Z. Xu, C. Hu, and L. Mei, “Video structured description technology based intelligence analysis of surveillance videos for public security applications,” 2015, doi: 10.1007/s11042-015-3112-5.
19. [19] T. Hussain, K. Muhammad, A. Ullah, Z. Cao, S. W. Baik, and V. H. C. De Albuquerque, “Cloud-assisted multiview video summarization using CNN and bidirectional LSTM,” IEEE Trans. Ind. Informatics, vol. 16, no. 1, pp. 77–86, 2020, doi: 10.1109/TII.2019.2929228.
20. [20] H. Kaya, H. Dibeklio, and A. A. Salah, “Kernel ELM and CNN based Facial Age Estimation,” pp. 80–86.
21. [21] Y. Byeon and S. Pan, “A Surveillance System Using CNN for Face Recognition with Object, Human and Face Detection,” pp. 975–984, doi: 10.1007/978-981-10-0557-2.
22. [22] Nogay, H.S. T.C. Akinci, and M. Yilmaz. "Detection of invisible cracks in ceramic materials using by pre-trained deep convolutional neural network." Neural Computing and Applications 34.2 (2022): 1423-1432.
23. [23] A. N. Shuaibu, A. S. Malik, and I. Faye, “Adaptive Feature Learning CNN for Behavior Recognition in Crowd Scene,” pp. 357–361, 2017
24. [24] H. Ahamed, I. Alam, and M. Islam, “HOG-CNNBasedRealTimeFaceRecognition,” 2018 Int. Conf. Adv. Electr. Electron. Eng., pp. 1–4, 2018.
25. [25] X. Xiang, N. Lv, X. Guo, S. Wang, and A. El Saddik, “Engineering vehicles detection based on modified faster R-CNN for power grid surveillance,” Sensors (Switzerland), vol. 18, no. 7, 2018, doi: 10.3390/s18072258.
26. [26] M. H. Gauswami, “Implementation of Machine Learning for Gender Detection using CNN on Raspberry Pi Platform,” 2018 2nd Int. Conf. Inven. Syst. Control, no. Icisc, pp. 608–613, 2018.
27. [27] D. Chahyati, M. I. Fanany, and A. M. Arymurthy, “Tracking People by Detection Using CNN Features,” Procedia Comput. Sci., vol. 124, pp. 167–172, 2018, doi: 10.1016/j.procs.2017.12.143.
28. [28] D. Chahyati, M. I. Fanany, and A. M. Arymurthy, “Tracking People by Detection Using CNN Features,” Procedia Comput. Sci., vol. 124, pp. 167–172, 2018, doi: 10.1016/j.procs.2017.12.143.
29. [29] H. C. Shin and J. Y. Lee, “Pedestrian Video Data Abstraction and Classification for Surveillance System,” 9th Int. Conf. Inf. Commun. Technol. Converg. ICT Converg. Powered by Smart Intell. ICTC 2018, pp. 1476–1478, 2018, doi: 10.1109/ICTC.2018.8539426.
30. [30] A. Ullah, K. Muhammad, J. Del Ser, S. W. Baik, and V. Albuquerque, “Activity Recognition using Temporal Optical Flow Convolutional Features and Multi-Layer LSTM,” IEEE Trans. Ind. Electron., vol. PP, no. c, p. 1, 2018, doi: 10.1109/TIE.2018.2881943.
31. [31] L. Du, R. Zhang, and X. Wang, "Overview of two-stage detection algorithms," 2020, doi:10.1088/1742-6596/1544/1/012033 R. Arti, “Animal Detection Using Deep Learning Algorithm,” vol. 7, no. 1, pp. 434–439, 2020
32. [32] O., Türk, A. Çalışkan, Acar, E. and B. Ergen. “Palmprint recognition system based on deep region of interest features with the aid of hybrid approach. SIViP ” 17, 3837–3845. 2023.