Detection of Defects in Railway Fasteners Using YOLOv4 and Fuzzy Logic

Öz

Fasteners in railways are the most important component of rail systems. Fasteners clamp other components together and allow the train to move safely on rails. Therefore, the detection of damaged fasteners is important to ensure the safety of rail transport. Detection of fasteners is usually done visually by trained employees. Thousands of kilometers of line must be inspected by people to ensure safety standards. However, this method is very limited in terms of speed and can cause negligence. Therefore, there is a need for the development of automatic control systems. In this study, a new method based on YOLOv4 and fuzzy logic is proposed to detect broken fasteners. The fastener image is divided into 6 separate parts and labeled. Confidence values of 6 parts were created by applying the YOLOv4 algorithm to the fastener images. The resulting 6 different confidence values are given as the input value for the fuzzy logic structure and a result value in percent about the health status of the fastener is produced. Experiment results showed that the correct detection rate was over 99%.

Anahtar Kelimeler

• Railways
• Fastener
• YOLOv4
• Fuzzy logic

Proje Numarası

120E097

Demiryolu Bağlantı Elemanlarında Bulunan Kusurların YOLOv4 ve Bulanık Mantık Kullanarak Tespiti

Öz

Demiryollarında bulunan bağlantı elemanları, raylı sistemlerin en önemli bileşenidir. Bağlantı elemanları, diğer bileşenleri birbirine bağlar ve trenin raylar üzerinde güvenli bir şekilde hareket etmesini sağlar. Bu nedenle, hasarlı bağlantı elemanlarının tespiti, demiryolu taşımacılığının güvenliğini sağlamak için önemlidir. Bağlantı elemanlarının kontrolü genellikle eğitimli çalışanlar tarafından görsel olarak yapılır. Güvenlik standartlarını sağlayabilmek için binlerce kilometrelik hat, insanlar tarafından denetlenmelidir. Ancak bu yöntem, hız açısından oldukça sınırlıdır ve ihmallere neden olabilir. Bu nedenle, otomatik denetim sistemlerinin geliştirilmesine ihtiyaç vardır. Bu çalışmada, kırık bağlantı elemanlarını tespit etmek için, YOLOv4 ve bulanık mantık yapısına dayanan yeni bir yöntem önerilmiştir. Bağlantı elemanı görüntüsü 6 ayrı parçaya bölünerek etiketlenmiştir. Bağlantı elemanı görüntülerine YOLOv4 algoritmasının uygulanması ile 6 parçanın güven değerleri oluşturulmuştur. Oluşan 6 farklı güven değeri bulanık mantık yapısı için giriş değeri olarak verilmiştir ve bağlantı elemanının sağlık durumu hakkında yüzde cinsinden sonuç değeri üretilmiştir. Deney sonuçları, doğru tespit oranının %99 üzerinde olduğunu göstermiştir.

Anahtar Kelimeler

• Demiryolları
• Bağlantı Elemanı
• YOLOv4
• Bulanık Mantık

Destekleyen Kurum

TÜRKİYE BİLİMSEL VE TEKNOLOJİK ARAŞTIRMA KURUMU

Proje Numarası

120E097

Kaynakça

1. [1] Xu, T., Wang, G., Wang, H., Yuan, T., & Zhong, Z. (2016), “Gap measurement of point machine using adaptive wavelet threshold and mathematical morphology,” Sensors, 16(12), 2006.
2. [2] Bokhman, E. D., Boronachin, A. M., Filatov, Y. V., Larionov, D. Y., Podgornaya, L. N., Shalymov, R. V., & Zuzev, G. N. (2014, September). “Optical-inertial system for railway track diagnostics,” In 2014 DGON Inertial Sensors and Systems (ISS) (pp. 1-17). IEEE.
3. [3] Ng, A. K., Martua, L., & Sun, G. (2019, September), “Dynamic modelling and acceleration signal analysis of rail surface defects for enhanced rail condition monitoring and diagnosis,” In 2019 4th International Conference on Intelligent Transportation Engineering (ICITE) (pp. 69-73). IEEE.
4. [4] He, B., Luo, J., Ou, Y., Xiong, Y., & Li, B. (2020). “Railway fastener defects detection under various illumination conditions using fuzzy C-Means part model,” Transportation Research Record, 0361198120977182.
5. [5] Marino, F., Distante, A., Mazzeo, P. L., & Stella, E. (2007), “A real-time visual inspection system for railway maintenance: automatic hexagonal-headed bolts detection,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 37(3), 418-428.
6. [6] Alippi, C., Casagrande, E., Scotti, F., & Piuri, V. (2000), “Composite real-time image processing for railways track profile measurement,” IEEE Transactions on instrumentation and measurement, 49(3), 559-564.
7. [7] Yang, J., Tao, W., Liu, M., Zhang, Y., Zhang, H., & Zhao, H. (2011), “An efficient direction field-based method for the detection of fasteners on high-speed railways,” Sensors, 11(8), 7364-7381.
8. [8] Yuan, X., Liu, B., & Chen, H. (2017, October), “Algorithm and program design for fastener locating and detection using wavelet transformation and template matching,” In 2017 IEEE 17th International Conference on Communication Technology (ICCT) (pp. 1116-1121). IEEE.

9. [9] De Ruvo, P., Distante, A., Stella, E., & Marino, F. (2009, November), “A GPU-based vision system for real time detection of fastening elements in railway inspection,” In 2009 16th IEEE International Conference on Image Processing (ICIP) (pp. 2333-2336). IEEE.
10. [10] Kocbek, S., & Gabrys, B. (2019, November), “Automated machine learning techniques in prognostics of railway track defects,” In 2019 International Conference on Data Mining Workshops (ICDMW) (pp. 777-784). IEEE.
11. [11] Feng, H., Jiang, Z., Xie, F., Yang, P., Shi, J., & Chen, L. (2013), “Automatic fastener classification and defect detection in vision-based railway inspection systems,” IEEE transactions on instrumentation and measurement, 63(4), 877-888.
12. [12] Dou, Y., Huang, Y., Li, Q., & Luo, S. (2014), “A fast template matching-based algorithm for railway bolts detection,”International Journal of Machine Learning and Cybernetics,5(6), 835-844.
13. [13] Dong, B., Li, Q., Wang, J., Huang, W., Dai, P., & Wang, S. (2019, November), “An end-to-end abnormal fastener detection method based on data synthesis,” In 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI) (pp. 149-156). IEEE.
14. [14] Chen, J., Liu, Z., Wang, H., & Liu, K. (2017, October), “High-speed railway catenary components detection using the cascaded convolutional neural networks,” In 2017 IEEE International Conference on Imaging Systems and Techniques (IST) (pp. 1-6). IEEE.
15. [15] Wei, X., Wei, D., Suo, D., Jia, L., & Li, Y. (2020), “Multi-target defect identification for railway track line based on image processing and improved YOLOv3 model,” IEEE Access, 8, 61973-61988.
16. [16] Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2015), “You only look once: Unified, real-time object detection,” arXiv 2015. arXiv preprint arXiv:1506.02640.
17. [17] Xu, X., Lei, Y., & Yang, F. (2018), “Railway subgrade defect automatic recognition method based on improved faster R-CNN,” Scientific Programming, 2018.
18. [18] Guo, F., Qian, Y., & Shi, Y. (2021), “Real-time railroad track components inspection based on the improved YOLOv4 framework,” Automation in Construction, 125, 103596.
19. [19] Wei, X., Yang, Z., Liu, Y., Wei, D., Jia, L., & Li, Y. (2019), “Railway track fastener defect detection based on image processing and deep learning techniques: A comparative study,” Engineering Applications of Artificial Intelligence, 80, 66-81.
20. [20] Wang, C. Y., Liao, H. Y. M., Wu, Y. H., Chen, P. Y., Hsieh, J. W., & Yeh, I. H. (2020), “CSPNet: A new backbone that can enhance learning capability of CNN,” In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops (pp. 390-391).
21. [21] Woo, S., Park, J., Lee, J. Y., & Kweon, I. S. (2018), “Cbam: Convolutional block attention modüle,” In Proceedings of the European conference on computer vision (ECCV) (pp. 3-19).
22. [22] He, K., Zhang, X., Ren, S., & Sun, J. (2015), “Spatial pyramid pooling in deep convolutional networks for visual recognition,” IEEE transactions on pattern analysis and machine intelligence, 37(9), 1904-1916.
23. [23] Liu, S., Qi, L., Qin, H., Shi, J., & Jia, J. (2018), “Path aggregation network for instance segmentation,” In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 8759-8768).
24. [24] Boulkroune, A. (2016), “A fuzzy adaptive control approach for nonlinear systems with unknown control gain sign,” Neurocomputing, 179, 318-325.
25. [25] Zadeh, L. A. (1965). Zadeh, Fuzzy sets. Inform Control, 8, 338-353.