Emergency Detection With Deep Learning Based Image Processing

Abstract

Today, people can be alone in difficult situations at home and outdoors, while doing sports or continuing their daily lives, due to various reasons. In addition, it is very important to find the missing person or child as soon as possible in the searches for missing people outside the residential area. Therefore, permanent disabilities and deaths occur in cases where such emergency contacts and those who need to be identified are not reached quickly. In this study, two artificial intelligence models have been developed to detect people who have been injured and lost with YOLOv4 and YOLOv4-tiny algorithms, which are deep learning methods. In the developed module, the images coming from the camera are passed through the artificial intelligence model, and the falling positions, standing and sitting positions of the people are detected, and when an emergency is detected, the alarm status is entered and the location/image information is sent to the relevant people. The emergency detection performance values of the developed artificial intelligence model were obtained as 99.04% for YOLOv4 and 97.91% for the model developed with YOLOv4-tiny in terms of mAP. It is possible to use the developed artificial intelligence module in the house environment as well as in UAVs. The use of the prototype obtained in this study by 112 Emergency Call Centers is thought to be useful in detecting and searching emergencies occurring at house or outside.

Keywords

• Artificial intelligence
• deep learning
• YOLO
• emergency detection
• UAV
• C#

Derin Öğrenme Tabanlı Görüntü İşleme İle Acil Durum Tespiti

Abstract

Günümüzde insanlar tek başlarına ev ve dış ortamlarda, spor yaparken veya günlük yaşamlarını sürdürürken çeşitli sebeplerden kaza geçirerek zor durumlarda kalabilmektedir. Ayrıca meskûn mahal dışında meydana gelen kayıp aramalarında da kaybolan kişi veya çocuğu bir an evvel bulunması çok önemlidir. Dolayısıyla bu tür acil ulaşılması ve tespit edilmesi gereken kişilere hızla ulaşılmaması durumlarında kalıcı sakatlıklar ve ölümler meydana gelmektedir. Bu çalışmada derin öğrenme yöntemlerinden olan YOLOv4 ve YOLOv4-tiny algoritmaları ile kazaya uğrayan ve kaybolan insanların tespit edilebilmesi için 2 adet yapay zekâ modeli geliştirilmiştir. Geliştirilen modülde kameradan gelen görüntüler yapay zekâ modelinden geçirilerek insanların düşme pozisyonları, ayakta durma ve oturma pozisyonları algılanmakta ve acil durum tespiti yapıldığında alarm durumuna geçilerek ilgili kişilere konum/görüntü bilgileri gönderilmektedir. Geliştirilen yapay zekâ modeline ait acil durum tespit başarım değerleri mAP cinsinden YOLOv4 için %99,04, YOLOv4-tiny ile geliştirilen model için %97,91 olarak elde edilmiştir. Geliştirilen yapay zekâ modülünün ev ortamında kullanımı mümkün olduğu gibi İHA’ larda kullanılabilmesi mümkündür. Bu çalışmada elde edilen prototipin 112 Acil Çağrı Merkezleri tarafından kullanılması ile ev veya dış ortamda meydana gelen acil durumların tespiti ve arama çalışmalarında faydalı olacağı düşünülmektedir.

Keywords

• Yapay zeka
• derin öğrenme
• YOLO
• acil durumlar
• İHA
• C#

References

1. Han, Q., Zhao, H., Min, W., Cui, H., Zhou, X., Zuo, K., & Liu, R. (2020). A two-stream approach to fall detection with mobilevgg. IEEE Access, 8. https://doi.org/10.1109/ACCESS.2019.2962778
2. Hung, G. L., Sahimi, M. S. Bin, Samma, H., Almohamad, T. A., & Lahasan, B. (2020). Faster R-CNN Deep Learning Model for Pedestrian Detection from Drone Images. SN Computer Science, 1(2). https://doi.org/10.1007/s42979-020-00125-y
3. Jiang, Z., Zhao, L., Li, S., Jia, Y., & Liquan, Z. (2020). Real-time object detection method for embedded devices.
4. Kulshreshtha, M., Chandra, S. S., Randhawa, P., Tsaramirsis, G., Khadidos, A., & Khadidos, A. O. (2021). Oatcr: Outdoor autonomous trash-collecting robot design using yolov4-tiny. Electronics (Switzerland), 10(18). https://doi.org/10.3390/electronics10182292
5. AlexeyAB. (2018). AlexeyAB/darknet: YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet ). https://github.com/AlexeyAB/darknet
6. Lie, W. N., Le, A. T., & Lin, G. H. (2018). Human fall-down event detection based on 2D skeletons and deep learning approach. 2018 International Workshop on Advanced Image Technology, IWAIT 2018. https://doi.org/10.1109/IWAIT.2018.8369778
7. Min, W., Cui, H., Rao, H., Li, Z., & Yao, L. (2018). Detection of Human Falls on Furniture Using Scene Analysis Based on Deep Learning and Activity Characteristics. IEEE Access, 6. https://doi.org/10.1109/ACCESS.2018.2795239
8. Mirmahboub, B., Samavi, S., Karimi, N., & Shirani, S. (2013). Automatic monocular system for human fall detection based on variations in silhouette area. IEEE Transactions on Biomedical Engineering, 60(2), 427–436. https://doi.org/10.1109/TBME.2012.2228262

9. Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., & Ren, D. (2020, April). Distance-IoU loss: Faster and better learning for bounding box regression. In Proceedings of the AAAI conference on artificial intelligence (Vol. 34, No. 07, pp. 12993-13000).
10. Göksu, M., & Alkan, A. (2022). Derin Öğrenme Temelli Robotik Maske Kontrol Sistemi. Fırat Üniversitesi Mühendislik Bilimleri Dergisi, 34(1), 459–471. https://doi.org/10.35234/fumbd.1060378
11. Pulido, C., & Ceron, A. (2022). Towards Real-Time Drone Detection Using Deep Neural Networks. Smart Innovation, Systems and Technologies, 255, 149–159. https://doi.org/10.1007/978-981-16-4884-7_12/COVER
12. Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., & Savarese, S. (2019). Generalized intersection over union: A metric and a loss for bounding box regression. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2019-June, 658–666. https://doi.org/10.1109/CVPR.2019.00075
13. Satoh, H., & Shibata, K. (2020). Improvement of a Monitoring System for Preventing Elderly Fall Down from a Bed. Advances in Intelligent Systems and Computing, 1026. https://doi.org/10.1007/978-3-030-27928-8_108
14. URL1. (2017). Fall detection Dataset. Last Visited. https://falldataset.com/
15. URL2. (2022). Fall Detection Dataset|Kaggle. https://www.kaggle.com/datasets/uttejkumarkandagatla/fall-detection-dataset.
16. URL3. (2018). E22-900T22D User Manual 868M/915M 22dBm DIP New LoRa Wireless Module. https://secureservercdn.net/160.153.138.180/bz0.2b1.myftpupload.com/wp-content/uploads/2021/12/E22-900T22D_UserManual_EN_v1.1.pdf
17. Yang, L., Ren, Y., & Zhang, W. (2016). 3D depth image analysis for indoor fall detection of elderly people. Digital Communications and Networks, 2(1), 24–34. https://doi.org/10.1016/J.DCAN.2015.12.001
18. Zhang, X., Han, L., Dong, Y., Shi, Y., Huang, W., Han, L., González-Moreno, P., Ma, H., Ye, H., & Sobeih, T. (2019). A deep learning-based approach for automated yellow rust disease detection from high-resolution hyperspectral UAV images. Remote Sensing, 11(13). https://doi.org/10.3390/rs11131554
19. Parico, A. I. B., & Ahamed, T. (2021). Real time pear fruit detection and counting using yolov4 models and deep sort. Sensors, 21(14). https://doi.org/10.3390/s21144803
20. Ukhwah, E. N.,Yuniarno, E. M., & Suprapto, Y. K. (2019). Asphalt Pavement Pothole Detection using Deep learning method based on YOLO Neural Network. Proceedings - 2019 International Seminar on Intelligent Technology and Its Application, ISITIA 2019, 35–40. https://doi.org/10.1109/ISITIA.2019.8937176
21. Fu, H., Song, G. ve Wang, Y. (2021). Improved YOLOv4 Marine Target Detection Combined with CBAM. Symmetry, 13(4), 623. doi:10.3390/sym13040623
22. Aicha, A. N., Englebienne, G., van Schooten, K. S., Pijnappels, M., & Kröse, B. (2018). Deep Learning to Predict Falls in Older Adults Based on Daily-Life Trunk Accelerometry. Sensors 2018, Vol. 18, Page 1654, 18(5), 1654. https://doi.org/10.3390/S18051654