mobilenet based traffic sign detection system for mobile mapping: crowdsourced geographical data collection system

Yıl 2024, Cilt: 39 Sayı: 4, 2305 - 2315, 20.05.2024

Ceren Özcan Tatar , Emrah Yılmaz , Abdullah Efe , Berk Sönmez , Yalçın Özdemir , Burak Danışan , Hale İrem Beyaz , Engin Yegnidemir

https://doi.org/10.17341/gazimmfd.1249165

Cited By: 1

Öz

Mobile mapping systems (MMS) have gained increasing interest as a cost-effective means of collecting geospatial data, catering to the digital mapping needs of various domains such as advanced driver assistance systems (ADAS) and intelligent transportation systems (ITS). In the generated maps, the location and class information of traffic signs are particularly crucial for the aforementioned applications. However, the extensive and complex nature of data collected by MMS makes it challenging to infer the location and class of traffic signs. Consequently, researchers have developed artificial intelligence-based methods for processing traffic sign data. In this study, a Crowdsourced Geographical Data Collection System (CGDCS) which is designed for the inference of traffic sign location and class information using artificial intelligence is introduced. CGDCS is a lightweight system that operates on mobile devices, leveraging the MobileNet architecture to detect and classify traffic signs present in real-time camera images, thereby transferring the location and class information of the signs to a database. The study demonstrates that CGDCS is more practical and efficient than traditional methods involving manual processing, semi-traditional methods based on the extraction of shape and color features of traffic signs, and AI-based methods that process field data in high-performance computers using high computer vision and machine learning techniques.

Anahtar Kelimeler

Crowdsourced Geographical Data Collection System (CGDCS) , machine learning , traffic sign , object detection , mobile mapping systems , artificial neural networks

Proje Numarası

1505/3200751 - 2244/119C200

Mobil haritalama amaçlı Mobilenet tabanlı trafik işaretleri tespit sistemi: kitlesel coğrafi bilgi toplama sistemi

Öz

Mobil haritalama sistemleri (Mobile Mapping Systems- MMS) coğrafi veri toplama yetenekleri ile birlikte, gelişmiş sürücü destek sistemleri (Advanced Driver Assistance Systems- ADAS) ve akıllı ulaşım sistemleri (Intelligent Transportation Systems - ITS) gibi birçok uygulama alanın sayısal harita ihtiyacını karşılayabilmektedir. Üretilen haritalarda özellikle trafik işaretlerinin konum ve sınıf bilgilerinin bulunması, bahsi geçen uygulama alanları için önem arz etmektedir. Ancak, MMS tarafından toplanan verilerin geniş ölçekli ve karmaşık olması, trafik işaretlerinin konum-sınıf çıkarımlarını zorlaştırmaktadır. Bu nedenle araştırmacılar, trafik işareti verilerinin işlenmesi için yapay zekâ tabanlı yöntemler geliştirmiştir. Bu çalışmada, trafik işaretlerinin konum ve sınıf bilgilerinin yapay zekâ ile çıkarımına yönelik tasarlanan Kitlesel Coğrafi Bilgi Toplama Sistemi (KCVTS) açıklanmıştır. KCVTS; MobileNet tabanıyla mobil cihazlarda etkinlik gösteren, cihazın gerçek-zamanlı kamera görüntülerinde bulunan trafik işaretlerini tespit eden ve sınıflandıran ve böylece, işaretlerin konum-sınıf bilgilerini veri tabanına aktaran hafif-yapılı bir sistemdir. Çalışmada KCVTS’nin manuel işlem gerektiren geleneksel yöntemlerden, trafik işaretlerinin şekil ve renk gibi özelliklerinin çıkarımına dayanan yarı-geleneksel yöntemlerden ve saha verilerinin merkezdeki güçlü bilgisayarlarda, bilgisayarlı görü ve makine öğrenmesi teknikleri ile işlendiği YZ tabanlı yöntemlerden birçok noktada daha pratik ve verimli olduğu gösterilmiştir.

Anahtar Kelimeler

kitlesel coğrafi veri toplama sistemi (KCVTS) , makine öğrenmesi , trafik işaretleri , mobil haritalama sistemleri , yapay sinir ağları , nesne tespiti

Destekleyen Kurum

Türkiye Bilimsel ve Teknolojik Araştırma Kurumu (TÜBİTAK)

Proje Numarası

1505/3200751 - 2244/119C200

Teşekkür

Bu çalışma Türkiye Bilimsel ve Teknolojik Araştırma Kurumu (TÜBİTAK) 1505 programı 3200751 numaralı proje ve 2244 programı 119C200 numaralı proje ile desteklenmiştir

Kaynakça

• 1. Arcos-García Á., Álvarez-García J. A., Soria-Morillo L. M., Evaluation of deep neural networks for traffic sign detection systems, Neurocomputing, 316, 332-344, 2018.
• 2. Salti S., Petrelli A., Tombari F., Fioraio N., Di Stefano L., Traffic sign detection via interest region extraction, Pattern Recognit., 48 (4), 1039-1049, 2015.
• 3. Qiu Z., Martínez-Sánchez J., Brea V. M., López P., Arias P., Low-cost mobile mapping system solution for traffic sign segmentation using Azure Kinect, Int. J. Appl. Earth Obs. Geoinformation, 112 (102895), 2022.
• 4. Timofte R., Zimmermann K., Van Gool L., Multi-view traffic sign detection, recognition, and 3D localisation, Workshop on Applications of Computer Vision (WACV), Snowbird, UT, USA: 1-8, 2009.
• 5. Li R., Mobile Mapping: An Emerging Technology for Spatial Data Acquisition, The Map Reader, John Wiley & Sons, Ltd, NJ, ABD, 170-177, 2011.
• 6. Tao C., Mobile Mapping Technology for Road Network Data Acquisition, Journal of Geospatial Engineering, 2 (2), 1-13, 2001.
• 7. Frentzos E., Tournas E., Skarlatos D., Developing An Image Based Low-Cost Mobile Mapping System for GIS Data Acquisition, Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci., XLIII-B1-2020, 235-242, 2020.
• 8. Kim G.-H., Sohn H.-G., Song Y.-S., Road Infrastructure Data Acquisition Using a Vehicle-Based Mobile Mapping System, Comput.-Aided Civ. Infrastruct. Eng., 21 (5), 346-356, 2006.
• 9. Manandhar D., Shibasaki R., Vehicle-borne laser mapping system (VLMS) for 3-D GIS, IGARSS 2001. Scanning the Present and Resolving the Future. Proceedings. IEEE 2001 International Geoscience and Remote Sensing Symposium, Sydney, NSW, Australia, 2073-2075, 2001.
• 10. El-Halawany S. I., Lichti D. D., Detecting road poles from mobile terrestrial laser scanning data, GIScience Remote Sens., 50 (6), 704-722, 2013.
• 11. Kumar P., McElhinney C. P., Lewis P., McCarthy T., Automated road markings extraction from mobile laser scanning data, Int. J. Appl. Earth Obs. Geoinformation, 32, 125-137, 2014.
• 12. D. Barber, J. Mills, ve S. Smith-Voysey, “Geometric validation of a ground-based mobile laser scanning system”, ISPRS J. Photogramm. Remote Sens., 63 (1), 128-141, Oca. 2008.
• 13. Hammoudi K., Dornaika F., Paparoditis N., Extracting Building Footprints From 3d Point Clouds Using Terrestrial Laser Scanning at Street Level, ISPRS Workshop on City Models, Roads and Traffic (CMRT), Paris, Fransa, 65-70, 3-4 Eylül 2009.
• 14. Yiğit A. Y., Hamal S.N.G., Ulvi A., Yakar M., Comparative analysis of mobile laser scanning and terrestrial laser scanning for the indoor mapping, Build. Res. Inf., 1-16, 2023.
• 15. Yiğit A.Y., Hamal S.N.G., Yakar M., Ulvi A., Investigation and Implementation of New Technology Wearable Mobile Laser Scanning (WMLS) in Transition to an Intelligent Geospatial Cadastral Information System, Sustainability, 15(9), 7159, 2023.
• 16. He Z., Nan F., Li X., Lee S.-J., Yang Y., Traffic sign recognition by combining global and local features based on semi-supervised classification, IET Intell. Transp. Syst., 14 (5), 323-330, 2020.
• 17. Zhu Y., Yan W. Q., Traffic Sign Recognition Based on Deep Learning Technique, Multimed Tools Appl, 81, 17779–17791 2022.
• 18. De La Escalera A., Armingol J. M., Mata M., Traffic sign recognition and analysis for intelligent vehicles, Image Vis. Comput., 21 (3), 247-258, 2003.
• 19. Seifert C. Paletta L., Jeitler A., Hödl E., Andreu J.P., Luley P., Almer A., Visual object detection for mobile road sign inventory, Lect. Notes Comput. Sci. Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinforma., 3160, 491-495, 2004.
• 20. Arcos-García, J. A. Álvarez-García Á., Soria-Morillo L. M., Deep neural network for traffic sign recognition systems: An analysis of spatial transformers and stochastic optimisation methods, Neural Netw., 99 (January), 158-165, 2018.
• 21. Nuakoh E. B., Roy K., Yuan X., Esterline A., Deep learning approach for U.S. traffic sign recognition, ICDLT '19: Proceedings of the 2019 3rd International Conference on Deep Learning Technologies NY, ABD, 47-50, 2019.
• 22. Pei S., Tang F., Ji Y., Fan J., Ning Z., Localized Traffic Sign Detection with Multi-scale Deconvolution Networks, Proc. - Int. Comput. Softw. Appl. Conf., 1, 355-360, 2018.
• 23. Yao Y., Han L., Du C., Xu X., Jiang X., Traffic sign detection algorithm based on improved YOLOv4-Tiny, Signal Process. Image Commun., 107 (116783), 2022.
• 24. Wu X., Cao H., Traffic Sign Detection Algorithm Based on Improved YOLOv4, J. Phys. Conf. Ser., 2258 (1), 2022. 25. Velamati A., Gopichand G., Traffic Sign Classification Using Convolutional Neural Networks and Computer Vision, Turk. J. Comput. Math. Educ., 12 (3), 4244-4250, 2021.
• 26. Zhou K., Zhan Y., Fu D., Learning region-based attention network for traffic sign recognition, Sens. Switz., 21 (3), 1-21, 2021.
• 27. Batool A., Nisar M. W., Hussain Shah J., Rehman A., Sadad T., IELMNet: An Application for Traffic Sign Recognition using CNN and ELM, 1st Int. Conf. Artif. Intell. Data Anal. CAIDA Riya, Suudi Arabistan, 132-137, 6-7 Nisan 2021.
• 28. Wan H., Gao L., Su M., You Q., Qu H., Sun Q., A Novel Neural Network Model for Traffic Sign Detection and Recognition under Extreme Conditions, J. Sens., 2021, 2021.
• 29. T.-Y. Lin vd., Microsoft COCO: Common Objects in Context. arXiv, http://arxiv.org/abs/1405.0312, 20 Şubat 2015, 18 Temmuz 2023.
• 30. ITU Racing Driverless, TTVS (Türkiye Trafik Veri Seti) https://github.com/ituracingdriverless/TTVS, 03 Nisan 2023, 18 Temmuz 2023.
• 31. GitHub, LabelImg Graphical Image Annotation Tool, https://github.com/heartexlabs/labelImg, 2018, 18 Ekim 2022.
• 32. Howard A. G., MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv, https://arxiv.org/abs/1704.04861 16 Nisan 2017, 18 Ekim 2022.
• 33. Liu W., SSD: Single Shot MultiBox Detector, Computer Vision – ECCV 2016. ECCV 2016. Lecture Notes in Computer Science, 9905, 21-37, 2016.
• 34. Chen B. vd., MnasFPN: Learning latency-aware pyramid architecture for object detection on mobile devices, Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., 13604-13613, 2020.
• 35. Sandler M., Howard A., Zhu M., Zhmoginov A., Chen L. C., MobileNetV2: Inverted Residuals and Linear Bottlenecks, Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., 4510-4520, 2018.
• 36. GitHub, TensorFlow 2 Detection Model Zoo, GitHub, https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md ,2021, 13 Temmuz 2023.
• 37. Kargah-Ostadi N., Waqar A., Hanif A., Automated Real-Time Roadway Asset Inventory using Artificial Intelligence, Transp. Res. Rec., 2674 (11), 220-234, 2020.