Classification of Steel Surface Defects Using Bilinear CNN

Öz

Steel is one of the most widely used building materials in the industry. With the increasing competition among steel manufacturers, the surface quality of steel products has become more important. Defects that may occur on steel surfaces can cause bigger problems when they are not detected. Today, steel surface defect detection systems have replaced traditional defect detection methods. Surface imperfections have an anomalous appearance as opposed to the appearance of solid steel. Using deep learning-based methods to detect these defects has many advantages over expensive methods. Therefore, with Industry 4.0, computer vision-based methods are more widely used for the detection of defects that may occur on steel products. In this study, bilinear convolutional neural network (Bilinear-CNN) is used to classify defects that may occur on steel surfaces. In the dataset used for training, defect and non-defect data are very similar to each other. The bilinear pooling method is capable of extracting higher order and spatially unordered information. Thus, it has been shown to achieve high performance in similar datasets. The performance of the proposed method has been evaluated for different networks. Bilinear Xception model obtained the highest result with an accuracy rate of 98.26%. The results show that the bilinear convolutional neural network achieves high performance in classifying datasets consisting of similar images.

Anahtar Kelimeler

• Steel surface defects
• Defect detection
• Bilinear CNN

Çift Doğrusal CNN Kullanarak Çelik Yüzey Kusurlarının Sınıflandırılması

Öz

Çelik, endüstride oldukça fazla kullanılan yapı malzemelerinden biridir. Çelik üreticileri arasındaki rekabetin artmasıyla birlikte çelik ürünlerin yüzey kalitesi daha önemli bir hale gelmiştir. Çelik yüzeylerde oluşabilecek kusurlar tespit edilemediğinde daha büyük sorunlara neden olabilmektedir. Günümüzde, çelik yüzey kusurlarını algılama sistemleri, geleneksel kusur tespit yöntemlerinin yerini almıştır. Yüzey kusurları, sağlam çelik görünümünden farklı olarak anormal görünüme sahiptir. Bu kusurların tespiti için derin öğrenme tabanlı yöntemlerin kullanılması, pahalı yöntemlere göre birçok avantaja sahiptir. Bu nedenle, Endüstri 4.0 ile birlikte çelik ürünler üzerinde oluşabilecek kusurların tespiti için bilgisayarlı görmeye dayalı yöntemler daha yaygın olarak kullanılmaktadır. Bu çalışmada, çelik yüzeylerde oluşabilecek kusurların sınıflandırılması için çift doğrusal evrişim sinir ağı (Bilinear-CNN) kullanılmıştır. Eğitim için kullanılan veri kümesinde kusurlu ve kusursuz veriler birbirine oldukça benzerdir. Çift doğrusal havuzlama yöntemi, daha yüksek dereceli ve uzamsal sırasız bilgileri çıkarabilme yeteneğine sahiptir. Böylece benzer veri kümelerinde yüksek performans elde ettiği gösterilmiştir. Önerilen yöntemin performansı farklı ağlar için değerlendirilmiştir. %98,26 doğruluk oranıyla en yüksek sonucu Bilinear Xception modeli elde etmiştir. Sonuçlar, çift doğrusal evrişimli sinir ağının benzer görüntülerden oluşan veri kümelerini sınıflandırmada yüksek performans elde ettiğini göstermektedir.

Anahtar Kelimeler

• Çelik yüzey kusurları
• Kusur tespiti
• Çift doğrusal CNN

Kaynakça

1. Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T. S., & Huang, H. (2011). Process knowledge based multi-class support vector classification (PK-MSVM) approach for surface defects in hot rolling. Expert Systems with Applications, 38(6), 7251-7262.
2. Choi, W., Huh, H., Tama, B. A., Park, G., & Lee, S. (2019). A neural network model for material degradation detection and diagnosis using microscopic images. IEEE Access, 7, 92151-92160.
3. Zhou, Q., Chen, R., Huang, B., Liu, C., Yu, J., & Yu, X. (2019). An automatic surface defect inspection system for automobiles using machine vision methods. Sensors, 19(3), 644.
4. Zheng, X., Zheng, S., Kong, Y., & Chen, J. (2021). Recent advances in surface defect inspection of industrial products using deep learning techniques. The International Journal of Advanced Manufacturing Technology, 113, 35-58.
5. Jiahui, C. O. N. G., Yunhui, Y. A. N., & Dong, D. (2010). Application of Gabor filter in strip surface defect detection. Journal of Northeast University (Natural Science Edition), 31(2), 257-260.
6. Ryu, S. G., Koo, G., & Kim, S. W. (2020). An adaptive selection of filter parameters: defect detection in steel image using wavelet reconstruction method. ISIJ International, 60(8), 1703-1713.
7. Mao, T., Ren, L., Yuan, F., Li, C., Zhang, L., Zhang, M., & Chen, Y. (2019, May). Defect recognition method based on HOG and SVM for drone inspection images of power transmission line. In 2019 international conference on high performance big data and intelligent systems (HPBD&IS) (pp. 254-257). IEEE.
8. Boudiaf, A., Benlahmidi, S., Harrar, K., & Zaghdoudi, R. (2022). Classification of surface defects on steel strip images using convolution neural network and support vector machine. Journal of Failure Analysis and Prevention, 22(2), 531-541.

9. Güçlü, E., Aydın, İ., Şener, T. K., & Erhan, A. K. I. N. Çelik Yüzeylerdeki Kusurların Tespiti için Derin Öğrenme Tabanlı Gömülü Sistem Tasarımı. EMO Bilimsel Dergi, 12(2), 27-33.
10. Konovalenko, I., Maruschak, P., Brezinová, J., Viňáš, J., & Brezina, J. (2020). Steel surface defect classification using deep residual neural network. Metals, 10(6), 846.
11. Gao, Y., Gao, L., Li, X., & Yan, X. (2020). A semi-supervised convolutional neural network-based method for steel surface defect recognition. Robotics and Computer-Integrated Manufacturing, 61, 101825.
12. He, Y., Song, K., Meng, Q., & Yan, Y. (2019). An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE Transactions on Instrumentation and Measurement, 69(4), 1493-1504.
13. Lee, S. Y., Tama, B. A., Moon, S. J., & Lee, S. (2019). Steel surface defect diagnostics using deep convolutional neural network and class activation map. Applied Sciences, 9(24), 5449.
14. Fu, G., Sun, P., Zhu, W., Yang, J., Cao, Y., Yang, M. Y., & Cao, Y. (2019). A deep-learning-based approach for fast and robust steel surface defects classification. Optics and Lasers in Engineering, 121, 397-405.
15. Gao, Y., Gao, L., Li, X., & Yan, X. (2020). A semi-supervised convolutional neural network-based method for steel surface defect recognition. Robotics and Computer-Integrated Manufacturing, 61, 101825.
16. Li, M., Wang, H., & Wan, Z. (2022). Surface defect detection of steel strips based on improved YOLOv4. Computers and Electrical Engineering, 102, 108208.
17. Karaduman, G., Aydin, I., Akin, E., & Özdemir, S. (2022, August). Detection of the Steel Faults Based on Deep Learning. In 2022 International Conference on INnovations in Intelligent SysTems and Applications (INISTA) (pp. 1-2). IEEE.
18. Zhou, F., Liu, G., Xu, F., & Deng, H. (2019). A generic automated surface defect detection based on a bilinear model. Applied Sciences, 9(15), 3159.
19. Liu, T., Zheng, P., Chen, H., & Zhang, L. (2023). An attention-based bilinear feature extraction mechanism for fine-grained laser welding molten pool/keyhole defect recognition. Journal of Manufacturing Processes, 87, 150-159.
20. Tang, Z., Tian, E., Wang, Y., Wang, L., & Yang, T. (2020). Nondestructive defect detection in castings by using spatial attention bilinear convolutional neural network. IEEE Transactions on Industrial Informatics, 17(1), 82-89.
21. Yang, D., Cui, Y., Yu, Z., & Yuan, H. (2021). Deep learning based steel pipe weld defect detection. Applied Artificial Intelligence, 35(15), 1237-1249.
22. Otsu, N. (1979). A threshold selection method from gray-level histograms. IEEE transactions on systems, man, and cybernetics, 9(1), 62-66.
23. Luo, J. H., & Wu, J. X. (2017). A survey on fine-grained image categorization using deep convolutional features. Acta Autom. Sin, 43(8), 1306-1318.
24. Lin, T. Y., RoyChowdhury, A., & Maji, S. (2015). Bilinear CNN models for fine-grained visual recognition. In Proceedings of the IEEE international conference on computer vision (pp. 1449-1457).
25. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L. C. (2018). Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4510-4520).
26. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
27. Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
28. Chollet, F. (2016). Xception: deep learning with depthwise separable convolutions (2016). arXiv preprint arXiv:1610.02357.