Detection of Red, Yellow, and Green Lights in Real-Time Traffic Lights with YOLO Architecture

Öz

In the study, red, yellow, and green lights at traffic lights were detected in real-world conditions and in real time. To adapt to real-world conditions, A data set was prepared from traffic lights in different locations, lighting conditions, and angles. A total of 5273 photographs of different traffic lights and different burning lamps were used in the data set. Additionally, grayscale, bevel, blur, variability, added noise, changed image brightness, changed color vibrancy, changed perspective, and resized and changed position have been added to photos. With these additions, the error that may occur due to any distortion from the camera is minimized. Four different YOLO architectures were used to achieve the highest accuracy rate on the dataset. As a result, the study obtained the highest accuracy at 98.3% in the YOLOV8 architecture, with an F1-Score of 0.939 and mAP@.5 value of 0.977. Since the work will be done in real time, the number of frames per second (FPS) must be the highest. The highest FPS number was 60 in the YOLOv8 architecture.

Anahtar Kelimeler

• Deep learning
• traffic light
• real-time detection
• YOLOV8

Kaynakça

1. [1]. Diaz-Cabrera, M, Cerri, P, Medici, P. 2015. Robust real-time traffic light detection and distance estimation using a single camera. Expert. Syst. Appl; 42, 3911–3923. https://doi.org/10.1016/j.eswa.2014.12.037
2. [2]. Hosseinyalamdary, S, Yilmaz, A. 2017. A Bayesian approach to traffic light detection and mapping, ISPRS J. Photograms. Remote Sens; 125, 184–192. https://doi.org/10.1016/j.isprsjprs.2017.01.008
3. [3]. Li, X, Ma, H, Wang, X, Zhang, X. 2018. Traffic light recognition for complex scene with fusion detections. IEEE Trans. Intell. Transp. Syst; 19, 199–208. https://doi.org/10.1109/TITS.2017.2749971
4. [4]. Boloor, A, Garimella, K, He, K, Gill, C, Vorobeychik, Y, Zhang, X. 2020. Attacking vi- sion-based perception in end-to-end autonomous driving models. J. Syst. Archit; 101766. . https://doi.org/10.1016/j.sysarc.2020.101766
5. [5]. Jensen, M.P, Philipsen, M.P, Møgelmose, A, Moeslund, T.B, Trivedi, M.M. 2016. Vision for looking at traffic lights: issues, survey, and perspectives. IEEE Trans. Intell. Transp. Syst; 17, 7, 1800–1815. https://doi.org/10.1109/TITS.2015.2509509
6. [6]. Ouyang, Z, Niu, J, Liu, Y, Guizani, M. 2019. Deep cnn-based real-time traffic light detector for self-driving vehicles. IEEE Trans. Mob. Comput; 19, 2, 300–313. https://doi.org/10.1109/TMC.2019.2892451
7. [7]. Kim, J, Cho, H, Hwangbo, M, Choi, J, Canny, J, Kwon, Y.P. Deep traffic light detection for self-driving cars from a large-scale dataset, in: 2018. 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, Hawaii, USA, 4-7 November 2018, pp. 280–285. https://doi.org/10.1109/ITSC.2018.8569575
8. [8]. Jensen, M, Philipsen, M, Møgelmose, A, Moeslund, T, Trivedi, M. 2016. Vision for looking at traffic lights: issues, survey, and perspectives. IEEE Trans. ITS; 17, 7, 1800–1815. https://doi.org/10.1109/TITS.2015.2509509

9. [9]. Behrendt, K, Novak, L, Botros, R. 2017. A deep learning approach to traffic lights: detection, tracking, and classification. IEEE ICRA; pp. 1370–1377. https://doi.org/10.1109/ICRA.2017.7989163
10. [10]. Sommer, L.W, Schuchert, T, Beyerer, J. Fast deep vehicle detection in aerial images. IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE; Santa Rose, California, USA. 24-31 March 2017. pp. 311–319. https://doi.org/10.1109/WACV.2017.41.
11. [11]. Zhang, X, Story, B, Rajan, D. 2021. Night time vehicle detection and tracking by fusing vehicle parts from multiple cameras. IEEE Transa. Intelligent Transp. Syst; pp. 258-265. http://dx.doi.org/10.1109/TITS.2021.3076406
12. [12]. Chen, C, Liu, B, Wan, S, Qiao, P, Pei, Q. 2021. An edge traffic flow detection scheme based on deep learning in an intelligent transportation system. IEEE Trans. Intelligent Trans. Syst; 22, 3, 1840–1852. https://doi.org/10.1155/2021/5583874
13. [13]. Zhang, Z, Zaman, A, Xu, J, Liu, X. 2022. Artificial intelligence-aided railroad trespassing detection and data analytics: methodology and a case study. Accident Anal. Prevention; 168, 106594. https://doi.org/10.1016/j.aap.2022.106594
14. [14]. Tsai, L.W, Hsieh, J.W, Fan, K.C. 2007. Vehicle detection using normalized color and edge map. IEEE Trans. Image Process; 16, 3, 850–864. http://dx.doi.org/10.1109/tip.2007.891147
15. [15]. Ehtesham, H, Yasser, K, Imtiaz, A. 2023. Learning deep feature fusion for traffic light detection. Journal of Engineering Research; 11, 94–99. http://dx.doi.org/10.1155/2020/7286187
16. [16]. Zhenchao, O, Jianwei, N, Tao, R, Yanqi, L, Jiahe, C, Jiyan, W. 2020. MBBNet: An edge IoT computing-based traffic light detection solution for autonomous bus. Journal of Systems Architecture; 109, 101835. http://dx.doi.org/10.1016/j.sysarc.2020.101835
17. [17]. Jean, P, V, M, Lucas, T, Rodrigo, F, B, Thiago, M, Alberto, F, S, Claudine, B, Nicu, S, Thiago, O, S. 2021. Deep traffic light detection by overlaying synthetic context on arbitrary natural images. Computers & Graphics; 94, 76–86. https://doi.org/10.48550/arXiv.2011.03841
18. [18]. Moises, D.C, Pietro, C, Paolo, M. 2015. Robust real-time traffic light detection and distance estimation using a single camera. Expert Systems with Applications; 42, 3911–3923. https://doi.org/10.1016/j.eswa.2014.12.037
19. [19]. Eunseop, L, Daijin, L. 2019 Accurate traffic light detection using deep neural network with focal regression loss. Image and Vision Computing, 87, 24–36. https://doi.org/10.1016/j.imavis.2019.04.003
20. [20]. Feng, G, Yi, W, Yu, Q. 2023. Real-time dense traffic detection using lightweight backbone and improved path aggregation feature pyramid network. Journal of Industrial Information Integration; 31, 100427. https://doi.org/10.1016/j.jii.2022.100427
21. [21]. Chuanxi, N, Kexin, L. 2022. Traffic Light Detection and Recognition Method Based on YOLOv5s and AlexNet. Appl. Sci; 12, 10808. https://doi.org/10.3390/app122110808
22. [22]. Lin, T,Y, Goyal, P, Girshick, R, He, K. Doll´ar, P. Focal loss for dense object detection. Proceedings of the IEEE international conference on computer vision, Venice, Italy, 22-29 Oct. 2017, pp. 2980–2988. https://doi.org/10.1109/ICCV.2017.324
23. [23]. Liu, Y, Sun, P, Wergeles, N, Shang, Y. 2021. A survey and performance evaluation of deep learning methods for small object detection. Expert Syst. Applied, 172, 114602. https://doi.org/10.1016/j.eswa.2021.114602
24. [24]. Yu, W, Yang, T, Chen, C.Towards resolving the challenge of long-tail distribution in uav images for object detection, Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision; Waikoloa, HI, USA. 3-8 June 2021. pp. 3258–3267. https://doi.org/10.48550/arXiv.2011.03822
25. [25]. Sang, J, Wu, Z, Guo, P, Hu, H, Xiang, H, Zhang, Q, Cai, B. 2018. An improved YOLOv2 for vehicle detection. Sensors; 18, 12, 4272. https://doi.org/10.3390/s18124272
26. [26]. Zhang, F, Yang, F, Li, C, Yuan, G. 2019. CMNet: a connect-and-merge convolutional neural network for fast vehicle detection in urban traffic surveillance. IEEE Access; 7, 72660–72671. https://doi.org/10.1155/2021/5583874
27. [27]. Tan, M, Pang, R, Le, and Q.W. Efficientdet: scalable and efficient object detection. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; Seattle, Washington, USA. 14-19 June 2020. pp. 10781–10790. https://doi.org/10.1109/CVPR42600.2020.01079
28. [28]. Xu, R, Lin, H, Lu, K, Cao, L, Liu, Y. 2021. A Forest Fire Detection System Based on Ensemble Learning. MDPI Forests, 12 (217), pp. 1568–1570. https://doi.org/10.3390/f12020217
29. [29]. Xie, X, He, C. Object detection of armored vehicles based on deep learning in battlefield environment. Proceedings - 2017 4th International Conference on Information Science and Control Engineering, ICISCE; Changsha, Chania, 21-23 July 2017. 1568–1570. https://doi.org/10.1109/ICISCE.2017.327
30. [30]. Boyuk, M, Duvar, R, Urhan, O. Deep learning based vehicle detection with images taken from unmanned air vehicle. Proceedings 2020 Innovations in Intelligent Systems and Applications Conference, ASYU; İstanbul, Türkiye, 15-17 October 2020. pp.175. https://doi.org/10.1109/ASYU50717.2020.9259868
31. [31]. Kamran, F, Shahzad, M, Shafait, F. Automated military vehicle detection from low-altitude aerial images. 2018 International Conference on Digital Image Computing: Techniques and Applications, DICTA. Canberra, Australis, 10-13 December 2018, 2 https://doi.org/10.1109/DICTA.2018.8615865
32. [32]. Gupta, P, Pareek, B, Singal, G, Rao, D. V. 2022. Edge device based military vehicle detection and classification from UAV. Multimedia Tools and Applications; 81(14), 19813–19834. https://doi.org/10.1007/S11042-021-11242-Y/FIGURES/12
33. [33]. Kyrkou, C, Plastiras, G, Theocharides, T, Venieris, S. I, Bouganis, C. S. 2018. DroNet: Efficient convolutional neural network detector for real-time UAV applications. Proceedings of the 2018 Design, Automation and Test in Europe Conference and Exhibition; 967–972. https://doi.org/10.23919/DATE.2018.8342149
34. [34]. Sun, Y, Wang, W, Zhang, Q, Ni, H, Zhang, X. 2022. Improved YOLOv5 with transformer for large scene military vehicle detection on SAR image. 2022 7th International Conference on Image, Vision and Computing, ICIVC; 87–93. https://doi.org/10.1109/ICIVC55077.2022.9887095
35. [35]. Yong, S. P, Yeong, Y. C. 2018. Human object detection in forest with deep learning based on drone’s vision. 2018 4th International Conference on Computer and Information Sciences: Revolutionizing Digital Landscape for Sustainable Smart Society, ICCOINS; https://doi.org/10.1109/ICCOINS.2018.8510564
36. [36]. Mansour, A, Hassan, A, Hussein, W. M, Said, E. 2019. Automated vehicle detection in satellite images using deep learning. IOP Conference Series: Materials Science and Engineering; 610(1). https://doi.org/10.1088/1757-899X/610/1/012027
37. [37]. Bayram, A.F, Nabiyev, V. 2023. Detection of Hidden Camouflaged Tanks Based on Deep Learning: Comparative Analysis of the art YOLO Network. Gümüşhane University of Journal of Science and Technology; 182-193. https://doi.org/10.17714/gumusfenbil.1271208
38. [38]. Gelayol, G, Ignacio, M.A, Qi, W, Jose, M.A.C. 2023. Robust Real-Time Traffic Light Detection on Small-Form Platform for Autonomous Vehicles. Journal of Intelligent Transportation System; 1-11 https://doi.org/10.1080/15472450.2023.2205018
39. [39]. Hassan, E, Khalil, Y, Ahmad, I. 2023. Learning Deep Feature Fusion for Traffic Light Detection. Journal of Engineering Research; 11, 94-99. https://doi.org/10.1016/j.jer.2023.100128
40. [40]. Ngoc, H.T, Nguyen, K.H, Hua, H.K, Nguyen. H.V.N, Quach, L. 2023. Optimizing YOLO Performance for Traffic Light Detection and End-to-End Steering Control for Autonomous Vehicles in Gazebo-ROS2. International Journal of Advanced Computer Science and Applications; Vol.14. 7. https://doi.org/10.14569/IJACSA.2023.0140752
41. [41]. Gao, H, Wang, W, Yang, C, Jiao, W, Chen, Z, Zhang, T. 2021. Traffic Signal Image Detection Technology Based on YOLO. Journal of Physics: Conference Series; 1-21. https://doi.org/10.1088/1742-6596/1961/1/012012
42. [42]. Serhan, N.H, Olmary, A. Y. 2022. Traffic Light Detection Using Opencv and YOLO. International Conference on Innovation and Intelligence for Informatics, Computing and Technologies; 604-608. https://doi.org/ 10.1088/1742-6596/1961/1/012012
43. [43]. Omar, W, Lee, I., Lee, G, Park, K.M. 2020. Detection and Localization of Traffic Light Using YOLOv3 and Stereo Vision. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences; Vol.18, 1247-1252. https://doi.org/10.5194/isprs-archives-XLIII-B2-2020-1247-2020
44. [44]. Nui, C, Li, K. 2022. Traffic Light Detection and Recognition Method Based on YOLOv5s and AlexNet, Applied Science; 12, 10808. https://doi.org/10.3390/app122110808