DETECTION OF TRAFFIC SIGNS FOR AUTONOMOUS DRIVING WITH THE DEEP LEARNING METHOD

Abstract

Deep learning practices used in many fields, in particular, in health, military, economy, and production industries, are an important area of artificial intelligence in our age. The object classification and object recognition applications, which play a significant role in the development of autonomous vehicle technologies, constitute the focal point of the deep learning studies. It is clear that the recent studies based on the deep learning models show that they are useful and successful performances for safe driving not only for vehicles but also for pedestrians. It is very crucial and significant that the autonomous systems recognize the traffic signs with high accuracy for a safe driving. Especially, the pedestrian crossing, school district, urban speed limits can be regarded among the most critical traffic signs. In this study, we have used the data set including the traffic signs obtained by our own means to carry out trainings by using faster R-CNN which is regarded as one of the most important recognition architectures. Thanks to the hardware module produced as a result of the operation, we have developed a system that warns the driver of the vehicle with audible warning. The developed hardware module can detect not only the speed limits, traffic signs but also pedestrian crossings and school districts and alert the driver in reel-time. The developed hardware module is based on Arduino and because of the GPS sensor, it can also show the speed of the vehicle. Moreover, we have used Python for the developed software and the dataset trainings have been carried out by using the Tensorflow library. We think that the study will contribute a lot to the recognition of traffic signs for the autonomous vehicle applications.

Keywords

• cnn
• traffic signs recognition
• object detection

Project Number

This work was supported by Research Fund of Isparta University of Applied Sciences. Project Number: 2019-YL1-0004

Otomatik sürüş için derin öğrenme metodu kullanarak trafik işaretlerinin tespiti

Abstract

Günümüzde sağlık, askeri, ekonomi ve üretim endüstrisi başta olmak üzere pek çok alanda kullanılan derin öğrenme uygulamaları yapay zekânın önemli bir alanını oluşturmaktadır. Otonom araç teknolojilerinin gelişiminde önemli bir payı bulunan nesne sınıflandırma ve nesne tanıma uygulamaları derin öğrenme çalışmalarının odak noktasını oluşturmaktadır. Hem araçlar hem de yayalar için güvenli sürüş noktasında derin öğrenme modellerine dayanan uygulamaların başarılı performanslar gösterdiği son yapılan çalışmalarda daha net olarak görülmektedir. Otonom sistemlerin güvenli sürüş için trafik işaretlerini yüksek doğruluk değerleri ile tanıması büyük önem taşımaktadır. Özellikle yaya geçidi, okul bölgesi, şehir içi hız limitleri en kritik trafik işaretleri arasında sayılabilir. Bu tez çalışmasında kendi imkânlarımızla elde ettiğimiz trafik işaretlerinden oluşan veri seti kullanılarak önemli nesne tanıma mimarilerinden olan faster R-CNN ile eğitimler gerçekleştirilmiştir. Çalışma neticesinde ortaya konan donanımsal modül sayesinde aracın sürücüsünü sesli ikazlar ile uyaran bir sistem geliştirilmiştir. Geliştirilen donanımsal modül hız limitlerinin yanı sıra yaya geçidi ve okul bölgesi gibi trafik işaretçilerini tespit ederek gerçek zamanlı olarak sürücüyü uyarabilmektedir. Ayrıca geliştirilen yazılım için Python dili kullanılırken, veri seti eğitimleri Tensorflow kütüphanesi kullanılarak gerçekleştirilmiştir. Çalışmanın otonom araç uygulamalarında trafik işaretlerinin tanınması noktasında bir katkı sağlayacağı düşünülmektedir.

Keywords

• cnn
• nesne tespiti
• trafik işareti tespiti

Project Number

This work was supported by Research Fund of Isparta University of Applied Sciences. Project Number: 2019-YL1-0004

References

1. [1] World Health Organization. (2018). Global Status Report On Road Safety 2018. https://www.who.int/publications/i/item/9789241565684
2. [2] Wijnen, W., & Stipdonk, H. (2016). Social costs of road crashes: An international analysis. Accident Analysis & Prevention, 94, 97-106. [DOI: 10.1016/j.aap.2016.05.006]
3. [3] Uğuz, S., & Büyükgökoğlan, E. (2022). A hybrid CNN-LSTM model for traffic accident frequency forecasting during the tourist season. Tehnički vjesnik, 29(6), 2083-2089. [DOI: 10.17559/TV-20220506004223]
4. [4] Fujiyoshi, H., Hirakawa, T., & Yamashita, T. (2019). Deep learning-based image recognition for autonomous driving. IATSS Research. [DOI: 10.1016/j.iatssr.2019.04.004]
5. [5] Nidhal, A., Ngah, U. K., & Ismail, W. (2014). Real-time traffic congestion detection system. In 2014 5th International Conference on Intelligent and Advanced Systems (ICIAS) (pp. 1-5). IEEE. [DOI: 10.1109/ICIAS.2014.7033872]
6. [6] Zhu, Y., Zhang, C., Zhou, D., Wang, X., Bai, X., & Liu, W. (2016). Traffic sign detection and recognition using fully convolutional network guided proposals. Neurocomputing, 214, 758-766. [DOI: 10.1016/j.neucom.2016.08.059]
7. [7] Shustanov, A., & Yakimov, P. (2017). CNN design for real-time traffic sign recognition. Procedia engineering, 201, 718-725. [DOI: 10.1016/j.proeng.2017.09.416]
8. [8] Changzhen, X., Cong, W., Weixin, M., & Yanmei, S. (2016). A traffic sign detection algorithm based on deep convolutional neural network. In 2016 IEEE International Conference on Signal and Image Processing (ICSIP) (pp. 676-679). IEEE. [DOI: 10.1109/SIPROCESS.2016.7919591]

9. [9] Kulkarni, R., Dhavalikar, S., & Bangar, S. (2018). Traffic Light Detection and Recognition for Self Driving Cars Using Deep Learning. In 2018 Fourth International Conference on Computing Communication Control and Automation (ICCUBEA) (pp. 1-4). IEEE. [DOI: 10.1109/ICCUBEA.2018.8617440]
10. [10] Stallkamp, J., Schlipsing, M., Salmen, J., & Igel, C. (2011). The German traffic sign recognition benchmark: a multi-class classification competition. Neural networks. [DOI: 10.1016/j.neunet.2012.02.016]
11. [11] Stallkamp, J., Schlipsing, M., Salmen, J., & Igel, C. (2012). Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural networks, 32, 323-332. [DOI: 10.1016/j.neunet.2012.02.016]
12. [12] Kim, J., & Lee, M. (2014). Robust Lane Detection Based On Convolutional Neural Network and Random Sample Consensus. In Loo C.K., Yap K.S., Wong K.W., Teoh A., Huang K. (eds) Neural Information Processing. ICONIP 2014. [DOI: 10.1007/978-3-319-12637-1_3]
13. [13] Qian, R., Zhang, B., Yue, Y., Wang, Z., & Coenen, F. (2015). Robust Chinese traffic sign detection and recognition with deep convolutional neural network. In 2015 11th International Conference on Natural Computation (ICNC) (pp. 791-796). IEEE. [DOI: 10.1109/ICNC.2015.7378104]
14. [14] Arcos-Garcia, A., Alvarez-Garcia, J. A., & Soria-Morillo, L. M. (2018). Evaluation of deep neural networks for traffic sign detection systems. Neurocomputing, 316, 332-344. [DOI: 10.1016/j.neucom.2018.07.072]
15. [15] Hussain, S., Abualkibash, M., & Tout, S. (2018). A survey of traffic sign recognition systems based on convolutional neural networks. In 2018 IEEE International Conference on Electro/Information Technology (EIT) (pp. 0570-0573). IEEE. [DOI: 10.1109/EIT.2018.8399772]
16. [16] Vasilev, I., Slater, D., Spacagna, G., Roelants, P., & Zocca, V. (2019). Python Deep Learning: Exploring deep learning techniques and neural network architectures with Pytorch, Keras, and TensorFlow. Packt Publishing Ltd. Birmingham.
17. [17] Mueller, J. P., & Massaron, L. (2019). Deep learning for dummies. John Wiley & Sons, Canada.
18. [18] LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324.
19. [19] Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. [DOI: 10.1016/j.neunet.2012.02.016]
20. [20] Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
21. [21] Singh, A., Patil, D., Reddy, G. M., & Omkar, S. (2017). Disguised Face identification (DFI) with facial Key Points using Spatial Fusion Convolutional Network. In IEEE International Conference on Computer Vision Workshops