Helmet detectionon the construction site with transfer learning and without transfer learning deep networks

Abstract

The widespread use of systems that prioritize human life provides holistic benefits to societies. In order to avoid respiratory contagious diseases, wearing a mouth-nose mask has become mandatory with the Covid-19 pandemic, and workers working in building construction are required to wear a head helmet at the construction site. It is tiring and error-prone to visually check whether the workers working on the construction sites are wearing their helmets. In this age, where artificial intelligence-based computer technologies are developed, the existence of systems that make our lives easier in every sense is promising. In this study, it is proposed to make helmet wearing control automatic with convolutional neural network (CNN) based deep learning in which the image data is meaningful. The limited data set problem was overcome with the transfer learning technique applied to the YOLO V4, V5 and Faster R-CNN models. The effectiveness of the method was examined by including the trainings in which transfer learning was not applied in the experiments. As a result, it was observed that the YOLO V5 model with transfer learning was the most successful among 6 different model trainings with an f1 score of 98%.

Keywords

• Helmet detection
• Deep learning
• Transfer learning
• CNN
• Artificial intelligence

Transfer öğrenmeli ve transfer öğrenmesiz derin ağlar ile inşaat alanında kask tespiti

Öz

İnsan yaşamını önceleyen sistemlerin yaygınlaşması toplumlara bütüncül fayda sağlamaktadır. Solunum yoluyla bulaşıcı hastalıklardan sakınmak için ağız-burun maskesi takmanın Covid-19 pandemisi ile zorunlu hâle geldiği gibi yapı inşaatında çalışan işçilerin inşaat alanında kafa kaskı takması zorunludur. İnşat alanlarında çalışan işçilerin kaskını takıp takmadığının kontrolünü göz ile yapmak yorucu ve hataya açıktır. Yapay zekâ tabanlı bilgisayar teknolojilerinin geliştiği bu çağda hayatımızı her anlamda kolaylaştıran sistemlerin varlığı ümit vaat etmektedir. Bu çalışmada görüntü verisinin anlamlandığı evrişimli sinir ağı (ESA) tabanlı derin öğrenme ile kask takma kontrolünün otomatik yapılması önerilmiştir ve YOLO V4, V5 ve Faster R-CNN modellerine uygulanan transfer öğrenme tekniği ile kısıtlı veri seti probleminin üstesinden gelinmiştir. Deneylerde transfer öğrenme uygulanmayan eğitimlere de yer verilerek yöntemin etkinliği incelenmiştir. Sonuçta transfer öğrenmeli YOLO V5 modelinin %98 f1 skor ile 6 farklı model eğitimi arasında en başarılı olduğu gözlemlenmiştir.

Anahtar Kelimeler

• Kask Tespiti
• Derin Öğrenme
• Transfer Öğrenme
• ESA
• Yapay Zekâ
• Helmet detection
• Deep learning
• Transfer learning
• CNN
• Artificial intelligence

Kaynakça

1. X. Chang and X. M. Liu, Fault tree analysis of unreasonably wearing helmets for builders, Journal of Jilin Jianzhu University, 35, 67–71, 2018.
2. L. Perezand J. Wang, The effectiveness of data augmentation in image classification using deep learning, arXiv, 2017. https://doi.org/10.48550/ARXIV.1712.04621.
3. Sorin, V., Barash, Y., Konen, E., & Klang, E. (2020). Deep-learning natural language processing for oncological applications. The Lancet Oncology, 21 (12), 1553-1556.
4. Bae, H. S., Lee, H. J., & Lee, S. G. (2016, June). Voice recognition based on adaptive MFCC and deep learning. In 2016 IEEE 11th Conference on Industrial Electronics and Applications (ICIEA) (pp. 1542-1546). IEEE.
5. Yuan, Z., Lu, Y., Wang, Z., & Xue, Y. (2014, August). Droid-sec: deeplearning in android malware detection. In Proceedings of the 2014 ACM conference on SIGCOMM (pp. 371-372).
6. Y. Sun, X. Wang and X. Tang, Hybrid deep learning for face verification, IEEE Trans. Pattern Anal. Mach. Intell., 38, 1489–1496. https://doi.org/10.1109/ICCV.2013.188.
7. R.Vinayakumar, K. P.Soman, and P. Poornachandran, Applying convolutional neural network for network intrusion detection. 2017 International Conference on Advances in Computing, pp. 1222-1228, Udupi, India, 2017.
8. K. Teja, L. Jens, S. Felix, H. Stefan,Review on Convolutional Neural Networks (CNN) in vegetation remote sensing, ISPRS Journal of Photogrammetry and Remote Sensing, 173,24-49, 2021. https://doi.org/10.1016/j.isprsjprs.2020.12.010.

9. Moccia, S., Romeo, L., Migliorelli, L., Frontoni, E., Zingaretti, P, Supervised CNN Strategies for Optical Image Segmentation and Classification in Interventional Medicine. Deep Learners and Deep Learner Descriptors for Medical Applications, ISRL, 2020.
10. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., ... &Fei-Fei, L. (2015). Imagenet large scale visual recognition challenge. International journal of computervision, 115(3), 211-252.
11. Lin, T. Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., ... & Zitnick, C. L. (2014, September). Microsoft coco: Commonobjects in context. In European conference on computer vision (pp. 740-755). Springer, Cham.
12. Everingham, M., Van Gool, L., Williams, C. K., Winn, J., & Zisserman, A. (2010). The pascal visual object classes (voc) challenge. International journal of computer vision, 88 (2), 303-338.
13. L.Kelm, A.Laußat, Meins-Becker et al., Mobile passive radio frequency identification (RFID) portal for automated and rapid control of personal protective equipment (PPE) on construction sites, Automation in Construction, 36, 38–52, 2013. https://doi.org/10.1016/j.autcon.2013.08.009.
14. S. Barro-Torres, T. M. Fernandez-Carames, H. J. Perez-Iglesias, and C. J. Escudero, Real-time personal protective equipment monitoring system, Computer Communications, 36, pp. 42–50, 2012.https://doi.org/10.1016/j.autcon.2013.08.009.
15. A. H. M. Rubaiyat, T. T. Toma, M. Kalantari-Khandani et al., Automatic detection of helmet uses for construction safety. İn Proceedings of the 2016 IEEE ACM International Conference on Web Intelligence Workshops (WIW), Omaha, USA, 2016.
16. T. Malisiewicz, A. Gupta, and A. A. Efros, Ensemble of exemplar-svms for object detection and beyond. 2011 IEEE International Conference on. IEEE, 2011.
17. C. C. Liu, J. S. Liao, W. Y. Chen, and J. H. Chen, The Full Motorcycle Helmet Detection Scheme Using Canny Detection, 18th IPPR Conf. On CVGIP, 2005.
18. M. H. Wuand J. Zhao, Automated visual helmet identification based on deep convolutional neural networks. İn Proceedings of the 13th International Symposium on Process Systems Engineering (PSE 2018), San Diego, USA, 2018.
19. Z. Fangbo, Z. Huailin, N. Zhen, Safety Helmet Detection Based on YOLOv5. IEEE, 2021.
20. W. Fan, J. Guoqing, G. Mingyu, H. E. Zhiwei, Y. Yuxiang, Helmet Detection Based On Improved YOLO V3 Deep Model. IEEE, 2019.
21. L. Yange, W. Han, H. Zheng, H. Jianling, W. Weidong, Deep Learning-Based Safety Helmet Detection in Engineering Management Based on Convolutional Neural Networks. Hindawi Advances in Civil Engineering.
22. T. Choudhury, A. Aggarwal, R. Tomar, A Deep Learning Approach to Helmet Detection for Road Safety. NIScPR, 2020.
23. D. Madhuchhanda, B. Oishila, C. Sanjay, Automated Helmet Detection for Multiple Motorcycle Riders using CNN. IEEE, 2019.
24. J. Wei, X. Shiquan, L. Zhen, Z. Yang, M. Hai, L. Shujie, Y. Ye, Real-time automatic helmet detection of motorcyclists in urban traffic using improved YOLOv5 detector. IET Image Processing. https://doi.org/10.1049/ipr2.12295.
25. T. Shilei, L. Gonglin, J. Ziqiangand H. Li, Improved YOLOv5 Network Model and Application in Safety Helmet Detection. IEEE, 2021.
26. G. Rui, M. Yixuan, H. Wanhong, An improved helmet detection method for YOLOv3 on an unbalanced dataset. IEEE, 2021.
27. T. Y.Lin, et al. Microsoft COCO: Common Objects in Context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds) Computer Vision – ECCV 2014. ECCV 2014. LectureNotes in ComputerScience, vol 8693. Springer, Cham. https://doi.org/10.1007/978-3-319-10602-1_48.
28. J. Zicong, Z. Liquan, L. Shuaiyang, J. Yanfei, Real-time object detection method for embedded devices. arXiv.https://doi.org/10.48550/arXiv.2011.04244.
29. K. B. Chethan, R. Punitha, Mohana, YOLOv3 and YOLOv4: Multiple Object Detection for Surveillance Applications. IEEE, 2020.
30. L. Yanfen, W. Hanxiang, L. M. Dang, T. N. Nguyen, D. Han, A. Lee, I. Jang, H. Moon, A Deep Learning-Based Hybrid Framework for Object Detection and Recognition in Autonomous Driving. IEEE. 2020.
31. T. Liu, B. Pang, L. Zhang, W. Yang and X. Sun, Sea Surface Object Detection Algorithm Based on YOLO v4 Fused with Reverse Depth wise Separable Convolution (RDSC) for USV, Journal of Marine Science and Engineering. https://doi.org/10.3390/jmse9070753.
32. A. M. Roy, R. Boseand J. Bhaduri, A fast accurate fine-grain object detection model based on YOLOv4 deep neural network, Neural Computing and Applications.https://doi.org/10.48550/arXiv.2111.00298.
33. J. Yuand W. Zhang, Face Mask Wearing Detection Algorithm Based on Improved YOLO-v4, Sensors. https://doi.org/10.3390/s21093263.
34. S. Li, Y. Li, Y. Li, M. Liand X. Xu, YOLO-FIRI: Improved YOLOv5 for Infrared Image Object Detection. IEEE. 2021.
35. D. Wu, S. Lv, M. Jiang and H. Song, Using channel pruning-based YOLO v4 deep learning algorithm for the real-time and accurate detection of apple flowers in natural environments, Computers and Electronics in Agriculture. https://doi.org/10.1016/j.compag.2020.105742.
36. J. Yuand H. Choi, YOLO MDE: Object Detection with Monocular Depth Estimation, Electronics. https://doi.org/10.3390/electronics11010076.
37. G. E. Hinton and R. R. Salakhutdinov, Reducing the dimensionality of data with neuralnetworks, Science. DOI: 10.1126/science.1127647.
38. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, Gradient-based learning applied to document recognition. IEEE, 1998.
39. LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324.
40. Russakovsky, Olga, et al. "Imagenet large scale visual recognition challenge." International journal of computer vision 115.3 (2015): 211-252.
41. https://cs230.stanford.edu/
42. Make ML, Hard Hat Dataset. https://makeml.app/datasets/hard-hat-workers, Accessed 15 April 2022.
43. J. Grum, Bookreview: pattern recognition and neural networks by B.D. Ripley, International Journal of Microstructure and Materials Properties. Cambridge University Press, 2008.
44. PiotrSkalski, make-sense, https://github.com/SkalskiP/make-sense, Accessed 13 March 2022.
45. glenn-jocher, YOLOv5 Focus() Layer, https://github.com/ultralytics/yolov5 /discussions/3181m1, Accessed 22 February 2022.
46. WS. Mseddi, MA. Sedrine, R. Attia, YOLOv5 Based Visual Localization for Autonomous Vehicles, In Proceedings of the 2021 29th European Signal Processing Conference (EUSIPCO), Dublin, Ireland.
47. G. Yang, W. Feng, J. Jin, Q. Lei, X. Li, G. Guiand, W. Wang, Face Mask Recognition System with YOLOV5 Based on Image Recognition, 2020 IEEE 6th International Conference on Computerand Communications.
48. K. Greff, R. K. Srivastava, J. Koutnik, B. R. Steunebrinkand J. Schmidhuber, LSTM: A search space odyssey. IEEE Transactions on Neural Networks and Learning Systems, 2017.
49. Goodfellow, I., B.Y.C.A., 2016. Deeplearning. MIT Press, 2016.
50. C. Bishop, Pattern recognition and machine learning, Springer-Verlag, New York, 2007.
51. Murphy, K., 2012. Machine learning: A probabilistic perspective. MIT Press, 2012.
52. A. Krizhevsky, I. Sutskeverand G. E. Hinton, Imagenet classification with deep convolutional neural networks, Adv. Neural Inf. Process. Syst. https://doi.org/10.1145/3065386.
53. D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, arXiv. https://doi.org/10.48550/arXiv.1412.6980.
54. P. D. Lima and K. Marfurt, Convolutional Neural Network for Remote-Sensing Scene Classification: Transfer Learning Analysis, Remote Sens. https://doi.org/10.3390/rs12010086.
55. C. Szegedy, W. Liu, Y. Jia et al, arXiv:1409.4842.
56. K. Haoarchive, Training a single AI model can emit as much carbon as five ars in their lifetimes. https://www.technologyreview.com/2019/06/06/239031/training-a-single-ai-model-can-emit-as-much-carbon-as-five-cars-in-their-lifetimes/, Accessed 8 March 2022.
57. L. Torreyand J. Shavlik, Transfer learning Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques. IGI Publishing, Hershey, 2009.
58. Y. Gao, Y. Ruan, C. Fangand S. Yin, Deep learning and transfer learning models of energy consumption forecasting for a building with poor information data, Energy and Buildings. https://doi.org/10.1016/j.enbuild.2020.110156.
59. C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang and C. Liu, A Survey on Deep Transfer Learning, arXiv. https://doi.org/10.48550/arXiv.1808.01974.
60. B.Koçer, Transfer öğrenmede yeni yaklaşımlar. Doktora tezi, Selçuk Üniversitesi. Konya, Türkiye, 2012.
61. F. Chollet, Transfer learning & fine-tuning, https://keras.io/guides/transfer-learning/, Accessed 27 December 2021.
62. F. İ. Eyiokur, Deep convolutional neural network based unconstrained ear recognition. Yüksek Lisans tezi, İstanbul Teknik Üniversitesi, İstanbul, Türkiye, 2018.
63. M. J. Afridi, A. Ross and E. M. Shapiro, On automated source selection for transfer learning in convolutional neural networks, Pattern Recognition. https://doi.org/10.1016/j.patcog.2017.07.019.
64. L. Mou, P. Ghamisiand X. X. Zhu, IEEE Trans. Geosci. Remote Sens., 2018.
65. S.J. Panand Q. Yang. IEEE Trans. Knowl. Data Eng., 2010.
66. M. Gong, H. Yang and P. Zhang, ISPRS J. Photogramm. Remote Sens.
67. L. Hughes, M. Schmitt, L. Mou, Y. Wang and X. Zhu IEEE Geosci. Remote Sens.
68. TY. Lin, et al. Microsoft COCO: Common Objects in Context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds) ComputerVision – ECCV 2014. ECCV 2014. LectureNotes in ComputerScience. https://doi.org/10.1007/978-3-319-10602-1_48.
69. M. Everingham, G. L. Van, C.K.I.Williams, et al., The PASCAL Visual Object Classes (VOC) Challenge. Int J ComputVis. https://doi.org/10.1007/s11263-009-0275-4
70. J. Deng, W. Dong, R. Socher, L. Li, K. Liand L. Fei-Fei. IEEE Computer Vision and Pattern Recognition (CVPR) (2009)
71. S. J. Pan and Q. Yang, A survey on transfer learning, IEEE Trans Knowl Data Eng, 2010.
72. S.Wang, L. Niu and N.Li, Research on image recognition of insulators based on YOLO algorithm, 2018 international conference on power system technology (POWERCON), IEEE, 2018.
73. M. Kartal, O. Duman, Ship detection fromoptical satellite images with deep learning, in: 2019 9th International Conference on Recent Advances in Space Technologies (RAST),2019.
74. Paszke et. al., PyTorch: An Imperative Style, High-Performance Deep Learning Library, Advances in Neural Information Processing Systems. https://doi.org/10.48550/arXiv.1912.01703.