A Machine Learning Approach for Simultaneous Classification of Material Types and Cracks

Öz

Exterior structures are susceptible to deformation, which can manifest as cracks on the surface. Deformations that occur on surfaces subjected to daily human use can exacerbate rapidly, potentially leading to irreversible structural damage. They have a potential to result in fatalities. Thus, continuous inspection of these deformations is of invaluable importance. In addition, the identification of the materials comprising the structures is essential to facilitate the implementation of appropriate precautionary measures. However, the inspections are hard to maintain with a solely human workforce. More advanced actions can be taken thanks to the developments in technology. Machine Learning methods could be used in this area where human workforce is ineffective. In this regard, an end-to-end Machine Learning approach was proposed in this study. The power of classical feature extraction methods and Artificial Neural Networks were combined to detect cracks and material of the surface simultaneously. The 2D Discrete Wavelet Transform and statistical properties gained from Gray Level Co-Occurrence Matrix were utilized in the feature extraction mechanism, and an ANN structure was designed. The findings of the study indicate that the proposed mechanism achieved an acceptable level of accuracy for recognizing the structural deformations, despite the challenges posed by the complexity of the problem.

Anahtar Kelimeler

• material recognition
• machine learning
• crack detection
• Classification

Kaynakça

1. D. Ai, G. Jiang, S.-K. Lam, P. He, and C. Li, “Computer vision framework for crack detection of civil infrastructure—A review,” Engineering Applications of Artificial Intelligence, 117 (2023) 10547; 10.1016/j.engappai.2022.105478.
2. E. Mohammed Abdelkader, “On the hybridization of pre-trained deep learning and differential evolution algorithms for semantic crack detection and recognition in ensemble of infrastructures,” Smart and Sustainable Built Environment, 11(3) (2022) 740–764; 10.1108/SASBE-01-2021-0010.
3. L. Attard, C. J. Debono, G. Valentino, M. Di Castro, A. Masi, and L. Scibile, “Automatic crack detection using mask R-CNN,” In: 11th international symposium on image and signal processing and analysis (ISPA), IEEE, (2019), 152–157.
4. G. Lu, X. He, Q. Wang, F. Shao, J. Wang, and X. Zhao, “MSCNet: A Framework with a Texture Enhancement Mechanism and Feature Aggregation for Crack Detection,” IEEE Access, 10 (2022) 26127–26139; 10.1109/ACCESS.2022.3156606.
5. Z. Xu et al., “Pavement crack detection from CCD images with a locally enhanced transformer network,” International Journal of Applied Earth Observation and Geoinformation, 110 (2022) 102825; https://doi.org/10.1016/j.jag.2022.102825.
6. P. Gupta and M. Dixit, “Image-based crack detection approaches: a comprehensive survey,” Multimedia Tools and Applications, 81(28) (2022) 40181–40229; https://doi.org/10.1007/s11042-022-13152-z.
7. L. Ali, F. Alnajjar, W. Khan, M. A. Serhani, and H. Al Jassmi, “Bibliometric analysis and review of deep learning-based crack detection literature published between 2010 and 2022,” Buildings, 12(4) (2022) 432; https://doi.org/10.3390/buildings12040432.
8. N. Safaei, O. Smadi, A. Masoud, and B. Safaei, “An automatic image processing algorithm based on crack pixel density for pavement crack detection and classification,” International Journal of Pavement Research and Technology, 15(1) (2022) 159–172; https://doi.org/10.1007/s42947-021-00006-4.

9. A. Mohan and S. Poobal, “Crack detection using image processing: A critical review and analysis,” Alexandria Engineering Journal, 57(2) (2018) 787–798; https://doi.org/10.1016/j.aej.2017.01.020.
10. A. Ahmadi, S. Khalesi, and A. Golroo, “An integrated machine learning model for automatic road crack detection and classification in urban areas,” International Journal of Pavement Engineering, 23(10) (2022) 3536–3552; https://doi.org/10.1080/10298436.2021.1905808.
11. J. Illingworth and J. Kittler, “A survey of the hough transform,” Computer Vision, Graphics, and Image Processing, 44(1) (1988) 87–116; https://doi.org/10.1016/S0734-189X(88)80033-1.
12. C. D. Sutton, “Classification and regression trees, bagging, and boosting,” Handbook of statistics, 24 (2005) 303–329; https://doi.org/10.1016/S0169-7161(04)24011-1.
13. V. Jakkula, “Tutorial on support vector machine (svm),” School of EECS, Washington State University, 37(2.5) (2006).
14. G. Zhang, X. Huang, S. Z. Li, Y. Wang, and X. Wu, “Boosting Local Binary Pattern (LBP)-Based Face Recognition,” In: Chinese Conference on Biometric Recognition, (2014) 179–186.
15. R. Bro and A. K. Smilde, “Principal component analysis,” Analytical Methods, 6(9) (2014) 2812–2831; https://doi.org/10.1039/C3AY41907J.
16. C. Chen, H. Seo, C. H. Jun, and Y. Zhao, “Pavement crack detection and classification based on fusion feature of LBP and PCA with SVM,” International Journal of Pavement Engineering, 23(9) (2022) 3274–3283; https://doi.org/10.1080/10298436.2021.1888092.
17. N.-D. Hoang, “Automatic detection of asphalt pavement raveling using image texture based feature extraction and stochastic gradient descent logistic regression” Automation in Construction, 105 (2019) 102843; https://doi.org/10.1016/j.autcon.2019.102843
18. R. M. Haralick, K. Shanmugam, and I. Dinstein, “Textural Features for Image Classification,” IEEE Transactions on Systems, Man, and Cybernetics, SMC-3 (6) (1973) 610–621;10.1109/TSMC.1973.4309314.
19. Y. LeCun and Y. Bengio, “Convolutional networks for images, speech, and time series,” The handbook of brain theory and neural networks, 3361 (10) (1995) 255-258, 1995.
20. R. Rojas “The backpropagation algorithm,” In: Neural Networks. Springer, Berlin, Heidelberg (1996) 149–182.
21. C. Liu and B. Xu, “A night pavement crack detection method based on image-to-image translation,” Computer-Aided Civil and Infrastructure Engineering, 37(13) (2022) 1737–1753; https://doi.org/10.1111/mice.12849.
22. Q. Yang, W. Shi, J. Chen, and W. Lin, “Deep convolution neural network-based transfer learning method for civil infrastructure crack detection,” Automation in Construction, 116 (2020) 103199; https://doi.org/10.1016/j.autcon.2020.103199.
23. X. Zhang, D. Rajan, and B. Story, “Concrete crack detection using context-aware deep semantic segmentation network,” Computer-Aided Civil and Infrastructure Engineering, 34(11) (2019) 951–971; https://doi.org/10.1111/mice.12477.
24. A. Chordia, S. Sarah, M. K. Gourisaria, R. Agrawal, and P. Adhikary, “Surface Crack Detection Using Data Mining and Feature Engineering Techniques,” In: IEEE 4th International Conference on Computing, Power and Communication Technologies (GUCON), (2021) 1–7.
25. L. Cong, J. Shi, T. Wang, F. Yang, and T. Zhu, “A method to evaluate the segregation of compacted asphalt pavement by processing the images of paved asphalt mixture,” Construction and Building Materials, 224 (2019) 622–629; https://doi.org/10.1016/j.conbuildmat.2019.07.041.
26. S. Dorafshan, R. J. Thomas, and M. Maguire, “SDNET2018: An annotated image dataset for non-contact concrete crack detection using deep convolutional neural networks,” Data in brief, 21 (2018) 1664–1668; https://doi.org/10.1016/j.dib.2018.11.015.
27. M. Mirbod and M. Shoar, “Intelligent Concrete Surface Cracks Detection using Computer Vision, Pattern Recognition, and Artificial Neural Networks,” Procedia Computer Science, 217 (2023) 52–61; https://doi.org/10.1016/j.procs.2022.12.201.
28. I. Daubechies, “Ten lectures on wavelets”, SIAM, 1992.
29. M. Sloński, “A comparison of deep convolutional neural networks for image-based detection of concrete surface cracks,” Computer assisted methods in Engineering and Science, 26 (2) (2019) 105–112; http://dx.doi.org/10.24423/cames.267.
30. R. Chianese, A. Nguyen, V. Gharehbaghi, T. Aravinthan, and M. Noori, “Influence of image noise on crack detection performance of deep convolutional neural networks,” In: Proceedings of the 10th International Conference on Structural Health Monitoring of Intelligent Infrastructure (SHMII 10), (2021) 1681 – 1688.