Faster R-CNN Structure for Computer Vision-based Road Pavement Distress Detection

Öz

Smart cities can be controlled in all aspects and it is desired to have a structure that is planned to have controllable feedback. Asphalt is generally used as pavement material on roads that provide transportation of vehicles such as cars and buses on the highway. Asphalt material is deformed due to weather conditions, heavy vehicle passage. In the smart city structure, similar deformations should be reported to the relevant unit. In this article, it was tried to determine the deteriorations on the asphalt by selecting the data set obtained from a region with image processing methods and deep learning technique. With the action camera placed in an automobile, a total of 4315 asphalt images with various distortions and without any deterioration were used as dataset. The dataset was classified using a pixel-based Faster Region-based Convolutional Neural Network. Accuracy, precision and sensitivity values were used to make the performance result obtained as a result of classification meaningful. With this proposed method, the average accuracy rate was 93.2%. With these results, an approach that can automatically detect asphalt deterioration in smart city structures has been developed.

Anahtar Kelimeler

• Deep Learning
• Faster R-CNN
• Image Processing
• Pavement Distress Detection

Bilgisayarlı Görü Tabanlı Yol Kaplaması Tehlikeleri Tespiti için Faster R-CNN Yapısı

Öz

Akıllı şehirler tüm yönüyle kontrol altına alınabilir ve kontrol edilebilir geri bildirimleri olması planlanan yapıya sahip olması istenmektedir. Karayolunda otomobil, otobüs gibi taşıtların ulaşımını sağlayan yollarda kaplama malzemesi olarak genellikle asfalt kullanılmaktadır. Asfalt malzemesi hava koşulları, yoğun araç geçişi gibi sebeplerden deforme olmaktadır. Akıllı şehir yapısında buna benzer deformelerin ilgili birime iletilmesi gerekmektedir. Bu makalede görüntü işleme yöntemleri ve derin öğrenme tekniği ile bir bölgeden elde edilen veri seti seçilerek asfalt üzerinde bozulmalar tespit edilmeye çalışılmıştır. Bir otomobile yerleştirilen aksiyon kamerası ile çeşitli bozulmaların olduğu ve herhangi bir bozulmanın olmadığı toplamda 4315 adet asfalt görüntüsü veri seti olarak kullanılmıştır. Piksel tabanlı çalışan Daha Hızlı Bölge Tabanlı Evrişimli Sinir Ağı kullanılarak veri seti sınıflandırılmıştır. Sınıflandırma sonucunda elde edilen performans sonucunun anlamlandırılabilir olması için doğruluk, kesinlik ve duyarlılık değerleri kullanılmıştır. Önerilen bu yöntem ileortalama doğruluk oranı %93.2 elde edilmiştir. Bu sonuçlar ile akıllı şehir yapılarında asfalt bozulmaları için otomatik olarak tespit yapabilen bir yaklaşım geliştirilmiştir.

Anahtar Kelimeler

• Derin Öğrenme
• Faster R-CNN
• Görüntü İşleme
• Kaplama Tehlikesi Tespiti

Kaynakça

1. [1] Gopalakrishnan K., Khaitan S. K., Choudhary A. and Agrawal A., “Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection”, Construction and Building Materials, 157: 322-330, (2017).
2. [2] Bello-Salau H., Aibinu A. M., Onwuka E. N., Dukiya J. J., Onumanyi A. J. and Ighagbon A. O., “Development of a laboratory model for automated road defect detection”, Journal of Telecommunication, Electronic and Computer Engineering, 8: 97-101, (2016).
3. [3] Shi Y., Cui L., Qi Z., Meng F. and Chen Z., “Automatic road crack detection using random structured forests”, IEEE Transactions on Intelligent Transportation Systems, 17: 1-12, (2016).
4. [4] Li B., Wang K. C. P., Zhang A., Yang E. and Wang G., “Automatic classification of pavement crack using deep convolutional neural network”, International Journal of Pavement Engineering, 21: 457-463, (2020).
5. [5] Majidifard H., Jin P., Adu-Gyamfi Y. and Buttlar W. G., “Pavement image datasets: a new benchmark dataset to classify and densify pavement distresses”, Transportation Research Record, 2674: 328-339, (2020).
6. [6] Zhang D., Li Q., Chen Y., Cao M., He L. and Zhang B., “An efficient and reliable coarse-to-fine approach for asphalt pavement crack detection”, Image and Vision Computing, 57: 130-146, (2017).
7. [7] Shahnazari H., Tutunchian M. A., Mashayekhi M. and Amini A. A., “Application of soft computing for prediction of pavement condition index”, Journal of Transportation Engineering, 138: 1495-1506, (2012).
8. [8] Xu W. and Tang Z., “Pavement crack detection based on saliency and statistical features”, IEEE International Conference on Image Processing, Melbourne Australia, 175–198, (2013).

9. [9] Dubrofsky E., “Homography Estimation”,Master of Science Thesis, the University of British Columbia, (2009).
10. [10] Simonyan K. and Zisserman B., “Very deep convolutional networks for large-scale image recognition”, the International Conference on Learning Representations, San Diego USA, 1-15, (2015).
11. [11] Xie D., Zhang L. and Bai L., “Deep learning in visual computing and signal processing”, Applied Computational Intelligence and Soft Compututing, 2017: 1–13, (2017).
12. [12] Girshick R., “Fast R-CNN”, the IEEE International Conference on Computer Vision, Santiago Chile, 1440-1448, (2015).
13. [13] Ren S., He K., Girshick R. and Sun J., “Faster R-CNN: Towards real-time object detection with region proposal networks”, the Advances in Neural Information Processing Systems, 91-99, (2015).
14. [14] Everingham M., Van Gool L., Williams C. K., Winn J. and Zisserman A., “The pascal visual object classes (VOC) challenge”, International Journal of Computer Vision, 88: 303-338, (2010).
15. [15] Turpin A. and Scholer F., “User performance versus precision measures for simple search tasks”, the ACM International Conference on Research and Development in Information Retrieval, Washington USA, 11-18, (2006).
16. [16] Song L. and Wang X., “Faster region convolutional neural network for automated pavement distress detection”, Road Materials and Pavement Design, 22: 23-41, (2021).
17. [17] Du Y., Pan N., Xu Z., Deng F., Shen Y. and Kang H., “Pavement distress detection and classification based on YOLO network”, International Journal of Pavement Engineering, in press.
18. [18] Gao J., Yuan D., Tong Z., Yang J. and Yu D., “Autonomous pavement distress detection using ground penetrating radar and region-based deep learning”, Measurement, 164: 108077, (2020).
19. [19] Mei Q. and Gül M., “A cost effective solution for pavement crack inspection using cameras and deep neural networks”, Construction and Building Materials, 256:119397, (2020).
20. [20] Huidrom L., Das L. K. and Sud S. K., “Method for automated assessment of potholes, cracks and patches from road surface video clips”, Procedia-Social and Behavioral Sciences, 104, 312-321, (2013).
21. [21] Ibragimov E., Lee H. J., Lee J. J. and Kim N., “Automated pavement distress detection using region based convolutional neural networks”, International Journal of Pavement Engineering, 1-12, (2020).
22. [22] Cha Y. J., Choi W. And Büyüköztürk O., “Deep learning-based crack damage detection using convolutional neural networks”, Computer-Aided Civil and Infrastructure Engineering, 32(5): 361-378, (2017).