GERİ YAYILIMLI BİRLİKTE EVRİM İLE İYİLEŞTİRİLMİŞ DERİN SİNİR AĞLARI KULLANILARAK YOL ÇATLAK TESPİTİ

Year 2024, Volume: 29 Issue: 2, 555 - 566, 30.08.2024

Emirhan Mustafa Anık , Turan Arslan

https://doi.org/10.17482/uumfd.1469361

Abstract

Karayolu esnek üstyapılarındaki çatlaklar genellikle trafik yükleri ve hava koşullarından kaynaklanır. Bu çatlakların genişlemeden tespit edilip gerekli bakımlarının yapılması, yol konforunun sürekliliğini sağlamanın yanı sıra bakım maliyetlerini de azaltacaktır. Bu çalışma, yoldaki çatlakları gerçek zamanlı ve yüksek doğrulukla tespit etmeyi amaçlamaktadır. Bu bağlamda, Geri Yayımlı Birlikte Evrim yaklaşımıyla İyileştirilmiş Derin Sinir Ağları ve görüntü işleme yöntemleri birlikte kullanılmıştır. Ayrıca, çeşitli sayı ve çözünürlüklerde çatlaklı görsel veriler içeren EdmCrack600, AsphaltCrack, CFD ve CrackSegmentation veri setleri kullanılarak yeni bir veri seti oluşturulmuş ve bu veri seti üzerinde Derin Sinir Ağları tabanlı öğrenme gerçekleştirilmiştir. Modelin doğruluğu, CFD veri seti kullanılarak Kesinlik, Duyarlılık ve F1-Skoru ile değerlendirilmiştir. Değerlendirme sonucunda, önerilen yöntemin saniyede 48 görsel üzerinde çatlak tespit edebildiği ve %92,74 Kesinlik, %88,92 Duyarlılık ve %89,61 F1 Skoru başarı oranlarına ulaştığı gözlemlenmiştir.

Keywords

Geri Yayılımlı Birlikte Evrim, Derin Sinir Ağları, Çatlak Tespiti, Görüntü İşleme, Esnek Üstyapı

Road Crack Detection Using Deep Neural Networks Developed via Cooperative Coevolution with Backpropagation

Abstract

Cracks in highway flexible pavements are primarily caused by traffic loads and weather conditions. Detecting these cracks before they expand and conducting necessary maintenance will not only ensure the continuity of road comfort but will also reduce maintenance costs. This study aims to detect cracks on the road in real-time and with high accuracy. In this context, Deep Neural Networks Developed via Cooperative Coevolution with Backpropagation and image processing methods were used together. Moreover, a new data set was obtained by using EdmCrack600, AsphaltCrack, CFD, and CrackSegmentation datasets containing cracked visual data in various numbers and resolutions, and Deep Neural Networks-based learning was performed on this dataset. The accuracy of the model was evaluated with Precision, Recall, and F1-Score using the CFD dataset. As a result of the evaluation, it has been observed that the proposed method can detect cracks on 48 images per second, while it can reach 92.74% Precision, 88.92% Recall, and 89.61% F1-Score success rates.

Keywords

Road Crack Detection, Cooperative Coevolution with Backpropagation, Deep Learning, Image Processing, Flexible Pavement

References

• Amhaz, R., Chambon, S., Idier, J. ve Baltazart, V. (2016) Automatic Crack Detection on Two-Dimensional Pavement Images: An Algorithm Based on Minimal Path Selection, IEEE Transactions on Intelligent Transportation Systems, 17(10), 2718-2729. doi:10.1109/TITS.2015.2477675
• Badrinarayanan, V., Kendall, A. ve Cipolla, R. (2017) SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), 2481-2495. doi:10.1109/TPAMI.2016.2644615
• Cha, Y., Choi, W., Suh, G., Mahmoudkhani, S. ve Büyüköztürk, O. (2018) AutonomousStructural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types, Computer-Aided Civil and Infrastructure Engineering, 33(9), 731-747. doi:10.1111/mice.12334
• Cha, Y.-J., You, K. ve Choi, W. (2016) Vision-based detection of loosened bolts using the Hough transform and support vector machines, Automation in Construction, 71, 181-188. doi:10.1016/j.autcon.2016.06.008
• Chen, T., Cai, Z., Zhao, X., Chen, C., Liang, X., Zou, T. ve Wang, P. (2020) Pavement crack detection and recognition using the architecture of segNet, Journal of Industrial Information Integration, 18, 100144. doi:10.1016/j.jii.2020.100144
• Cui, L., Qi, Z., Chen, Z., Meng, F. ve Shi, Y. (2015) Pavement Distress Detection Using Random Decision Forests (ss. 95-102). doi:10.1007/978-3-319-24474-7_14
• Doğan, G. ve Ergen, B. (2022) Karayollarındaki Asfalt Çatlaklarının Tespiti İçin Yeni Bir Konvolüsyonel Sinir Ağı Tabanlı Yöntem, Fırat Üniversitesi Mühendislik Bilimleri Dergisi, 34(2), 485-494. doi:10.35234/fumbd.1014951
• Du, Y., Pan, N., Xu, Z., Deng, F., Shen, Y. ve Kang, H. (2021) Pavement distress detection and classification based on YOLO network, International Journal of Pavement Engineering, 22(13), 1659-1672. doi:10.1080/10298436.2020.1714047
• Dung, C. V. ve Anh, L. D. (2019) Autonomous concrete crack detection using deep fully convolutional neural network, Automation in Construction, 99, 52-58. doi:10.1016/j.autcon.2018.11.028
• Fan, J., Bocus, M. J., Wang, L. ve Fan, R. (2021) Deep Convolutional Neural Networks for Road Crack Detection: Qualitative and Quantitative Comparisons, 2021 IEEE International Conference on Imaging Systems and Techniques (IST) (ss. 1-6), IEEE. doi:10.1109/IST50367.2021.9651375
• Fan, Z., Lin, H., Li, C., Su, J., Bruno, S. ve Loprencipe, G. (2022) Use of Parallel ResNet for High-Performance Pavement Crack Detection and Measurement, Sustainability (Switzerland), 14(3). doi:10.3390/su14031825
• Fei, Y., Wang, K. C. P., Zhang, A., Chen, C., Li, J. Q., Liu, Y., … Li, B. (2020) Pixel-Level Cracking Detection on 3D Asphalt Pavement Images Through Deep-Learning- Based CrackNet-V, IEEE Transactions on Intelligent Transportation Systems, 21(1), 273-284. doi:10.1109/TITS.2019.2891167
• Gao Z., Zhao X., Cao M., Li Z., Liu K. ve Chen B. M. (2023) Synergizing Low Rank Representation and Deep Learning for Automatic Pavement Crack Detection, IEEE Transactions on Intelligent Transportation Systems, 24(10), 10676-10690. doi: 10.1109/TITS.2023.3275570
• Gavilán, M., Balcones, D., Marcos, O., Llorca, D. F., Sotelo, M. A., Parra, I., … Amírola, A. (2011) Adaptive Road Crack Detection System by Pavement Classification, Sensors, 11(10), 9628-9657. doi:10.3390/s111009628
• Gong, M., Liu, J., Qin, A. K., Zhao, K. ve Tan, K. C. (2021) Evolving Deep Neural Networks via Cooperative Coevolution With Backpropagation, IEEE Transactions on Neural Networks and Learning Systems, 32(1), 420-434. doi:10.1109/TNNLS.2020.2978857
• Huyan, J., Li, W., Tighe, S., Xu, Z. ve Zhai, J. (2020) CrackU‐net: A novel deep convolutional neural network for pixelwise pavement crack detection, Structural Control and Health Monitoring, 27(8). doi:10.1002/stc.2551
• Jayanth Balaji, A., Thiru Balaji, G., Dinesh, M. S., Binoy, N. ve Harish Ram, D. S. (2019) Asphalt Crack Dataset, Mendeley Data. doi:10.17632/xnzhj3x8v4.2
• Krizhevsky, A., Sutskever, I. ve Hinton, G. E. (2017) ImageNet classification with deep convolutional neural networks, Communications of the ACM, 60(6), 84-90. doi:10.1145/3065386
• Lee, B. J. ve Lee, H. “David”. (2004) Position‐Invariant Neural Network for Digital Pavement Crack Analysis, Computer-Aided Civil and Infrastructure Engineering, 19(2), 105-118. doi:10.1111/j.1467-8667.2004.00341.x
• Liu, J., Yang, X., Lau, S., Wang, X., Luo, S., Lee, V. C. ve Ding, L. (2020) Automated pavement crack detection and segmentation based on two‐step convolutional neural network, Computer-Aided Civil and Infrastructure Engineering, 35(11), 1291-1305. doi:10.1111/mice.12622
• Liu, K. ve Chen, B. M. (2023) Industrial UAV-Based Unsupervised Domain Adaptive Crack Recognitions: From Database Towards Real-Site Infrastructural Inspections, IEEE Transactions on Industrial Electronics, 70(9), 9410-9420. doi:10.1109/TIE.2022.3204953
• Liu, K., Han, X. ve Chen, B. M. (2019) Deep Learning Based Automatic Crack Detection and Segmentation for Unmanned Aerial Vehicle Inspections, 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO) (ss. 381-387), IEEE. doi:10.1109/ROBIO49542.2019.8961534
• Ma, D., Fang, H., Wang, N., Xue, B., Dong, J. ve Wang, F. (2022) A real-time crack detection algorithm for pavement based on CNN with multiple feature layers, Road Materials and Pavement Design, 23(9), 2115-2131. doi:10.1080/14680629.2021.1925578
• Mei, Q. ve Gül, M. (2020) A cost effective solution for pavement crack inspection using cameras and deep neural networks, Construction and Building Materials, 256, 119397. doi:10.1016/j.conbuildmat.2020.119397
• Mei, Q., Gül, M. ve Azim, M. R. (2020) Densely connected deep neural network considering connectivity of pixels for automatic crack detection, Automation in Construction, 110, 103018. doi:10.1016/j.autcon.2019.103018
• Moon, H.-G. ve Kim, J.-H. (2011) Inteligent Crack Detecting Algorithm on the Concrete Crack Image Using Neural Network, International Association for Automation and Robotics in Construction (IAARC). doi:10.22260/ISARC2011/0279
• Munawar, H. S., Hammad, A. W. A., Haddad, A., Soares, C. A. P. ve Waller, S. T. (2021) Image-Based Crack Detection Methods: A Review, Infrastructures, 6(8), 115. doi:10.3390/infrastructures6080115
• Naddaf-Sh, S., Naddaf-Sh, M.-M., Kashani, A. R. ve Zargarzadeh, H. (2020) An Efficient and Scalable Deep Learning Approach for Road Damage Detection, 2020 IEEE International Conference on Big Data (Big Data) (ss. 5602-5608), IEEE. doi:10.1109/BigData50022.2020.9377751
• Nguyen, T. S., Begot, S., Duculty, F. ve Avila, M. (2011) Free-form anisotropy: A new method for crack detection on pavement surface images, 2011 18th IEEE International Conference on Image Processing (ss. 1069-1072), IEEE. doi:10.1109/ICIP.2011.6115610
• O’Byrne, M., Schoefs, F., Ghosh, B. ve Pakrashi, V. (2013) Texture Analysis Based Damage Detection of Ageing Infrastructural Elements, Computer-Aided Civil and Infrastructure Engineering, 28(3), 162-177. doi:10.1111/j.1467-8667.2012.00790.x
• Oliveira, H. ve Correia, P. (2009) Automatic road crack segmentation using entropy and image dynamic thresholding, 2009 17th European Signal Processing Conference, 622-626. https://ieeexplore.ieee.org/document/6302929 adresinden erişildi.
• Oliveira, H. ve Correia, P. L. (2014) Automated Visual Inspection of Pavement Crack Detection and Characterization, https://api.semanticscholar.org/CorpusID:9409925 adresinden erişildi.
• Peng, L., Chao, W., Shuangmiao, L. ve Baocai, F. (2015) Research on Crack Detection Method of Airport Runway Based on Twice-Threshold Segmentation, 2015 Fifth International Conference on Instrumentation and Measurement, Computer, Communication and Control (IMCCC) (ss. 1716-1720), IEEE. doi:10.1109/IMCCC.2015.364
• Potter, M. A. ve Jong, K. A. (1994) A cooperative coevolutionary approach to function optimization (ss. 249-257). doi:10.1007/3-540-58484-6_269
• Qin, A. K., Huang, V. L. ve Suganthan, P. N. (2009) Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization, IEEE Transactions on Evolutionary Computation, 13(2), 398-417. doi:10.1109/TEVC.2008.927706
• Qu, Z., Chen, W., Wang, S.-Y., Yi, T.-M. ve Liu, L. (2022) A Crack Detection Algorithm for Concrete Pavement Based on Attention Mechanism and Multi-Features Fusion, IEEE Transactions on Intelligent Transportation Systems, 23(8), 11710-11719. doi:10.1109/TITS.2021.3106647
• Rumelhart, D. E. ve McClelland, J. L. (1987) Learning Internal Representations by Error Propagation, Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations (ss. 318-362). https://ieeexplore.ieee.org/document/6302929 adresinden erişildi.
• Santhi, B., Krishnamurthy, G., .S, S. ve Ramakrishnan, P. K. (2012) Automatic detection of cracks in pavements using edge detection operator, Journal of Theoretical and Applied Information Technology, 36, 199-205. https://www.researchgate.net/publication/289942204_Automatic_detection_of_cracks_in_p avements_using_edge_detection_operator adresinden erişildi.
• Shi, Y., Cui, L., Qi, Z., Meng, F. ve Chen, Z. (2016) Automatic Road Crack Detection Using Random Structured Forests, IEEE Transactions on Intelligent Transportation Systems, 17(12), 3434-3445. doi:10.1109/TITS.2016.2552248
• Staniek, M. (2017) Detection of cracks in asphalt pavement during road inspection processes, Scientific Journal of Silesian University of Technology, Series Transport, 96, 175-184. doi:10.20858/sjsutst.2017.96.16
• Tsai, Y.-C., Kaul, V. ve Mersereau, R. M. (2010) Critical Assessment of Pavement Distress Segmentation Methods, Journal of Transportation Engineering, 136(1), 11-19. doi:10.1061/(ASCE)TE.1943-5436.0000051
• Vyas, V., Singh, A. P. ve Srivastava, A. (2019) Entropy-based fuzzy SWOT decision-making for condition assessment of airfield pavements, International Journal of Pavement Engineering, 22(10), 1226–1237. doi:10.1080/10298436.2019.1671590
• Wang, W. ve Su, C. (2021) Deep Learning-Based Real-Time Crack Segmentation for Pavement Images, KSCE Journal of Civil Engineering, 25(12), 4495-4506. doi:10.1007/s12205-021-0474-2
• Xin Yao. (1999) Evolving artificial neural networks, Proceedings of the IEEE, 87(9), 1423- 1447. doi:10.1109/5.784219
• Yang, M., Yu, K., Zhang, C., Li, Z. ve Yang, K. (2018) DenseASPP for Semantic Segmentation in Street Scenes, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (ss. 3684-3692), IEEE. doi:10.1109/CVPR.2018.00388
• Yeum, C. M. ve Dyke, S. J. (2015) Vision‐Based Automated Crack Detection for Bridge Inspection, Computer-Aided Civil and Infrastructure Engineering, 30(10), 759-770. doi:10.1111/mice.12141
• Zhang, A. A., Wang, K. C. P., Liu, Y., Zhan, Y., Yang, G., Wang, G., … Shang, J. (2022) Intelligent pixel‐level detection of multiple distresses and surface design features on asphalt pavements, Computer-Aided Civil and Infrastructure Engineering, 37(13), 1654-1673. doi:10.1111/mice.12909
• Zhang T., Wang D. ve Lu Y. (2023) ECSNet: An Accelerated Real-Time Image Segmentation CNN Architecture for Pavement Crack Detection, IEEE Transactions on Intelligent Transportation Systems, 24(12), 15105-15112. doi: 10.1109/TITS.2023.3300312
• Zhou, J. (2006) Wavelet-based pavement distress detection and evaluation, Optical Engineering, 45(2), 027007. doi:10.1117/1.2172917
• Zhu, G., Liu, J., Fan, Z., Yuan, D., Ma, P., Wang, M., … Wang, K. C. P. (2023) A lightweight encoder–decoder network for automatic pavement crack detection, Computer-Aided Civil and Infrastructure Engineering. doi:10.1111/mice.13103
• Zou, Q., Cao, Y., Li, Q., Mao, Q. ve Wang, S. (2012) CrackTree: Automatic crack detection from pavement images, Pattern Recognition Letters, 33(3), 227-238. doi:10.1016/j.patrec.2011.11.004
• Zou, Q., Li, Q., Zhang, F., Xiong Qian Wang, Z. ve Wang, Q. (2016) Path voting based pavement crack detection from laser range images, 2016 IEEE International Conference on Digital Signal Processing (DSP) (ss. 432-436), IEEE. doi:10.1109/ICDSP.2016.7868594