Detection of Defects in Railway Fasteners Using DCGAN and Siamese Neural Network

Öz

Detection of defects in fasteners is an important part of railway inspection. Therefore, in recent years, automatic inspection systems are needed for fast and reliable inspection of fasteners. Methods such as deep learning are used in automatic control systems. However, such methods require a lot of datasets for training. A traditional convolutional neural network cannot learn features with a small dataset. Although it is easy to generate a dataset of solid fasteners for training, it is quite difficult to generate a dataset of defective fasteners. It may be necessary to collect images from hundreds of kilometers of railroad tracks to create this type of dataset. For this reason, in this study, artificial deformed fastener images were created using DCGAN and the dataset was reproduced. Then, the defect status of the connectors with the siamese neural network was examined. In the study, there are two fastener classes as healty and deformed. Similarity scores between images of different classes were calculated for each class. The basic idea is to identify fasteners using similarity scores and comparing. In the experimental results, 98,23% accuracy rate was obtained for the proposed method and its advantage over traditional methods was demonstrated.

Anahtar Kelimeler

• Siamese Neural Network
• Fastener
• Defect Detection

Proje Numarası

120E097

DCGAN ve Siyam Sinir Ağını Kullanarak Demiryolu Bağlantı Elemanlarındaki Kusurların Tespiti

Öz

Bağlantı elemanlarındaki kusurların tespiti, demiryolu denetiminin önemli bir parçasıdır. Bu nedenle son yıllarda, bağlantı elemanlarının hızlı ve güvenilir bir şekilde denetlenebilmesi için otomatik denetim sistemlerine ihtiyaç duyulmaktadır. Otomatik denetim sistemlerinde derin öğrenme gibi yöntemler kullanılmaktadır. Ancak bu tür yöntemler, eğitim için çok fazla veri setine ihtiyaç duyarlar. Geleneksel bir evrişimli sinir ağı küçük bir veri seti ile özellikleri öğrenemez. Eğitim işlemi için sağlam bağlantı elemanlarından oluşan veri setini oluşturmak kolay olmasına rağmen kusurlu bağlantı elemanlarından oluşan veri setini oluşturmak oldukça zordur. Bu tür veri setini oluşturmak için yüzlerce kilometre demiryolundan görüntü toplanması gerekebilir. Bu nedenle bu çalışmada, DCGAN kullanılarak yapay deforme bağlantı elemanı görüntüleri oluşturulup veri seti çoğaltılmıştır. Ardından, siyam sinir ağı ile bağlantı elemanlarının kusur durumu incelenmiştir. Çalışmada, sağlam ve deforme olmak üzere iki bağlantı elemanı sınıfı bulunmaktadır. Her sınıf için farklı sınıfların görüntüleri arasındaki benzerlik puanları hesaplanmıştır. Temel fikir, bağlantı elemanlarını benzerlik puanlarını kullanarak ve karşılaştırma yaparak tanımlamaktır. Deneysel sonuçlarda, önerilen yöntem için %98,23 doğruluk oranı elde edilerek, geleneksel yöntemlere göre avantajı gösterilmiştir.

Anahtar Kelimeler

• Siyam Sinir Ağı
• Bağlantı Elemanı
• Kusur Tespiti

Destekleyen Kurum

TÜRKİYE BİLİMSEL VE TEKNOLOJİK ARAŞTIRMA KURUMU

Proje Numarası

120E097

Teşekkür

Bu çalışma 120E097 Nolu TUBITAK projesi tarafından desteklenmiştir.

Kaynakça

1. [1] Z. Peng, C. Wang, Z. Ma, and H. Liu, “A multifeature hierarchical locating algorithm for hexagon nut of railway fasteners,” IEEE Trans. Instrum. Meas., vol. 69, no. 3, pp. 693–699, 2020.
2. [2] I. Aydin, E. Akin, and M. Karakose, “Defect classification based on deep features for railway tracks in sustainable transportation,” Appl. Soft Comput., vol. 111, no. 107706, p. 107706, 2021.
3. [3] J. Yang, W. Tao, M. Liu, Y. Zhang, H. Zhang, and H. Zhao, “An efficient direction field-based method for the detection of fasteners on high-speed railways,” Sensors (Basel), vol. 11, no. 8, pp. 7364–7381, 2011.
4. [4] H. Ma, “A real time detection method of track fasteners missing of railway based on machine vision,” International Journal of Performability Engineering, 2018.
5. [5] R. Geirhos, D. H. J. Janssen, H. H. Schütt, J. Rauber, M. Bethge, and F. A. Wichmann, “Comparing deep neural networks against humans: object recognition when the signal gets weaker,” arXiv [cs.CV], 2017.
6. [6] M. Ferguson, R. Ak, Y.-T. T. Lee, and K. H. Law, “Detection and segmentation of manufacturing defects with Convolutional Neural Networks and transfer learning,” arXiv [cs.CV], 2018.
7. [7] J. Shi, Z. Li, T. Zhu, D. Wang, and C. Ni, “Defect detection of industry wood veneer based on NAS and multi-Channel Mask R-CNN,” Sensors (Basel), vol. 20, no. 16, p. 4398, 2020.
8. [8] Y. Yang, X. Zhou, Y. Liu, Z. Hu, and F. Ding, “Wood defect detection based on depth extreme learning machine,” Appl. Sci. (Basel), vol. 10, no. 21, p. 7488, 2020.

9. [9] B. Su, H. Chen, P. Chen, G. Bian, K. Liu, and W. Liu, “Deep learning-based solar-cell manufacturing defect detection with complementary attention network,” IEEE Trans. Industr. Inform., vol. 17, no. 6, pp. 4084–4095, 2021.
10. [10] L. Xu, S. Lv, Y. Deng, and X. Li, “A weakly supervised surface defect detection based on convolutional neural network,” IEEE Access, vol. 8, pp. 42285–42296, 2020.
11. [11] S. Faghih-Roohi, S. Hajizadeh, A. Nunez, R. Babuska, and B. De Schutter, “Deep convolutional neural networks for detection of rail surface defects,” in 2016 International Joint Conference on Neural Networks (IJCNN), 2016, pp. 2584–2589.
12. [12] X. Wei, D. Wei, D. Suo, L. Jia, and Y. Li, “Multi-target defect identification for railway track line based on image processing and improved YOLOv3 model,” IEEE Access, vol. 8, pp. 61973–61988, 2020.
13. [13] H. Cui, J. Li, Q. Hu, and Q. Mao, “Real-time inspection system for ballast railway fasteners based on point cloud deep learning,” IEEE Access, vol. 8, pp. 61604–61614, 2020.
14. [14] T. Bai, J. Yang, G. Xu, and D. Yao, “An optimized railway fastener detection method based on modified Faster R-CNN,” Measurement (Lond.), vol. 182, no. 109742, p. 109742, 2021.
15. [15] E. Güçlü, İ. Aydin, K. Şahbaz, E. Akin, and M. Karaköse, “Demiryolu bağlantı elemanlarında bulunan kusurların YOLOv4 ve bulanık mantık kullanarak tespiti,” Demiryolu Mühendisliği, no. 14, pp. 249–262, 2021.
16. [16] Y. Ou, J. Luo, B. Li, and B. He, “A classification model of railway fasteners based on computer vision,” Neural Computing and Applications, 2019.
17. [17] J. Liu, Y. Teng, X. Ni, and H. Liu, “A fastener inspection method based on defective sample generation and deep convolutional neural network,” IEEE Sens. J., vol. 21, no. 10, pp. 12179–12188, 2021.
18. [18] G. Koch, “Siamese neural networks for one-shot image recognition,” Toronto.edu. [Online]. Available: http://www.cs.toronto.edu/~gkoch/files/msc-thesis.pdf. [Accessed: 31-Oct-2021].
19. [19] M. S. Kim, T. Park, and P. Park, “Classification of steel surface defect using Convolutional Neural Network with few images,” in 2019 12th Asian Control Conference (ASCC), 2019, pp. 1398–1401.
20. [20] S. Wu, Y. Wu, D. Cao, and C. Zheng, “A fast button surface defect detection method based on Siamese network with imbalanced samples,” Multimed. Tools Appl., vol. 78, no. 24, pp. 34627–34648, 2019.
21. [21] A. Nagy and L. Czúni, “Detecting object defects with fusioning convolutional Siamese neural networks,” in Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, 2021.
22. [22] A. Aggarwal, M. Mittal, and G. Battineni, “Generative adversarial network: An overview of theory and applications,” International Journal of Information Management Data Insights, vol. 1, no. 1, p. 100004, 2021.
23. [23] X. Dong et al., “Fast efficient algorithm for enhancement of low lighting video,” in 2011 IEEE International Conference on Multimedia and Expo, 2011, pp. 1–6.
24. [24] K. He, J. Sun, and X. Tang, “Single image haze removal using dark channel prior,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 12, pp. 2341–2353, 2011.
25. [25] P. L. Suarez, A. D. Sappa, and B. X. Vintimilla, “Infrared image colorization based on a triplet DCGAN architecture,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017, pp. 18–23.
26. [26] I. J. Goodfellow et al., “Generative Adversarial Nets,” Neurips.cc. [Online]. Available: https://proceedings.neurips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf. [Accessed: 31-Oct-2021].
27. [27] A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” arXiv [cs.LG], 2015.
28. [28] J. Bromley et al., “Signature verification using a ‘Siamese’ time delay neural network,” Intern. J. Pattern Recognit. Artif. Intell., vol. 07, no. 04, pp. 669–688, 1993.
29. [29] A. F. Agarap, “Deep Learning using Rectified Linear Units (ReLU),” arXiv [cs.NE], 2018.
30. [30] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv [cs.LG], 2014.