Result Weighting-Based Resnet Feature Pyramid Network Architecture for Surface Defect Detection

Abstract

Surface defect detection is very important in manufacturing systems to ensure high quality products. Unlike manual inspections under human supervision, automatic surface defect detection is both efficient and highly accurate. In this study, a Result Weighting-based Resnet Feature Pyramid Network (SA-RÖPA) model has been developed for automatic pixel-level surface defect detection. In the first stage of the proposed model, the pre-trained Resnet50 network was used, and feature maps were extracted from the different levels of this network. In the second stage, Feature Pyramid Model was applied to these feature maps in order to hierarchically share important information in defect detection. In the third stage, 4 different error detection results were obtained by using these feature maps. In the last stage, four different results obtained using the developed Result Weighting (SA) module were effectively combined. The proposed SA-ROPA model has been tested with MT, MVTex-Doku, and AITEX datasets, which are widely used in defect detection studies. In experimental studies, the mIoU value obtained for the MT and AITEX datasets using the proposed model was calculated as 79.92%, 76.37%, and 82.72%, respectively. These results have shown that the proposed SA- ROPA model is more successful than other state-of-the-art models.

Keywords

• Surface Defect Detection
• Pyramid Feature Network
• Convolutional Neural Network
• Segmentation

Yüzey Hata Tespiti için Sonuç Ağırlıklandırma Tabanlı Resnet Öznitelik Piramit Ağ Mimarisi

Abstract

Yüzey hata tespiti, imalat sistemlerinde yüksek kalitede ürün sağlanması açısından oldukça önemlidir. İnsan gözetimi altında yapılan manuel denetimlerin aksine, otomatik yüzey hatası tespiti hem verimli hem de yüksek doğruluktadır. Bu çalışmada piksel seviyesinde otomatik yüzey hata tespiti için Sonuç Ağırlıklandırma tabanlı Resnet Öznitelik Piramit Ağ (SA-RÖPA) modeli geliştirilmiştir. Önerilen modelin ilk aşamasında, önceden eğitilmiş Resnet50 ağı kullanılmış ve bu ağının farklı seviyelerinden öznitelik haritaları çıkartılmıştır. İkinci aşamada, hata tespitinde önemli bilgileri hiyerarşik olarak paylaşmak için bu öznitelik haritalarına Öznitelik Piramit Modeli uygulanmıştır. Üçüncü aşamada, bu öznitelik haritaları kullanılarak 4 farklı hata tespit sonucu elde edilmiştir. Son aşamada, geliştirilen Sonuç Ağırlıklandırma (SA) modülü kullanılarak elde edilen 4 farklı sonuç etkili bir şekilde birleştirilmiştir. Önerilen SA-RÖPA modeli, hata tespit çalışmalarında yaygın olarak kullanılan MT, MVTex-Doku ve AITEX veri kümeleri ile test edilmiştir. Deneysel çalışmalarda, önerilen model kullanılarak MT, MVTex-Doku ve AITEX veri kümeleri için elde edilen mIoU değeri, sırayla %79,92, %76,37 ve %82,72 olarak hesaplanmıştır. Bu sonuçlar, önerilen SA-RÖPA modelinin diğer son teknoloji modellerine göre daha başarılı olduğunu göstermiştir.

Keywords

• Yüzey Hata Tespiti
• Öznitelik Piramit Ağı
• Evrişimsel Sinir Ağı
• Bölütleme

References

1. [1] D. Zhang, K. Song, J. Xu, Y. He, M. Niu, and Y. Yan, “MCnet: Multiple Context Information Segmentation Network of No-Service Rail Surface Defects,” IEEE Transactions on Instrumentation and Measurement, vol. 70, 2021, doi: 10.1109/TIM.2020.3040890.
2. [2] H. Uzen, M. Turkoglu, and D. Hanbay, “Texture defect classification with multiple pooling and filter ensemble based on deep neural network,” Expert Systems with Applications, vol. 175, p. 114838, Aug. 2021, doi: 10.1016/j.eswa.2021.114838.
3. [3] K. Hanbay, M. F. Talu, and Ö. F. Özgüven, “Fabric defect detection systems and methods—A systematic literature review,” Optik, vol. 127, no. 24, pp. 11960–11973, Dec. 2016, doi: 10.1016/j.ijleo.2016.09.110.
4. [4] H. Dong, K. Song, Y. He, J. Xu, Y. Yan, and Q. Meng, “PGA-Net: Pyramid Feature Fusion and Global Context Attention Network for Automated Surface Defect Detection,” IEEE Transactions on Industrial Informatics, vol. 16, no. 12, pp. 7448–7458, Dec. 2020, doi: 10.1109/TII.2019.2958826.
5. [5] J. Cao, G. Yang, and X. Yang, “A Pixel-Level Segmentation Convolutional Neural Network Based on Deep Feature Fusion for Surface Defect Detection,” IEEE Transactions on Instrumentation and Measurement, vol. 70, 2021, doi: 10.1109/TIM.2020.3033726.
6. [6] H. Y. T. Ngan, G. K. H. Pang, and N. H. C. Yung, “Automated fabric defect detection-A review,” Image and Vision Computing, vol. 29, no. 7. Elsevier Ltd, pp. 442–458, Jun. 01, 2011. doi: 10.1016/j.imavis.2011.02.002.
7. [7] X. Xie, A Review of Recent Advances in Surface Defect Detection using Texture analysis Techniques Figure 1: Example defects on different types of surfaces-from left: Steel, vol. 7, no. 3. 2008, pp. 1–22. Accessed: Jan. 08, 2021. [Online]. Available: https://www.raco.cat/index.php/ELCVIA/article/view/150223
8. [8] G. Song, K. Song, and Y. Yan, “EDRNet: Encoder-Decoder Residual Network for Salient Object Detection of Strip Steel Surface Defects,” IEEE Transactions on Instrumentation and Measurement, vol. 69, no. 12, pp. 9709–9719, Dec. 2020, doi: 10.1109/TIM.2020.3002277.

9. [9] P. M. Bhatt et al., “Image-Based Surface Defect Detection Using Deep Learning: A Review,” Journal of Computing and Information Science in Engineering, vol. 21, no. 4. American Society of Mechanical Engineers (ASME), Aug. 01, 2021. doi: 10.1115/1.4049535.
10. [10] H. Firat, “3B ESA Tabanlı ResNet50 Kullanılarak Hiperspektral Görüntülerin Sınıflandırılması Classification of Hyperspectral Images Using 3D CNN Based ResNet50,” 2021 29th Signal Processing and Communications Applications Conference (SIU), pp. 6–9, 2021, doi: 10.1109/SIU53274.2021.9477899.
11. [11] H. Firat and D. Hanbay, “Classification of Hyperspectral Images Using 3D CNN Based ResNet50,” 2021 29th Signal Processing and Communications Applications Conference (SIU), pp. 1–4, Jun. 2021, doi: 10.1109/SIU53274.2021.9477899.
12. [12] H. Uzen, H. Firat, A. Karci, and D. Hanbay, “Automatic Thresholding Method Developed with Entropy for Fabric Defect Detection,” Sep. 2019. doi: 10.1109/IDAP.2019.8875890.
13. [13] J. Long, E. Shelhamer, and T. Darrell, “Fully Convolutional Networks for Semantic Segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 4, pp. 640–651, Nov. 2014.
14. [14] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137–1149, Jun. 2017, doi: 10.1109/TPAMI.2016.2577031.
15. [15] W. Liu et al., “SSD: Single Shot MultiBox Detector,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9905 LNCS, pp. 21–37, Dec. 2015, doi: 10.1007/978-3-319-46448-0_2.
16. [16] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2016-December, pp. 779–788, Jun. 2015.
17. [17] A. Chaurasia and E. Culurciello, “LinkNet: Exploiting encoder representations for efficient semantic segmentation,” 2017 IEEE Visual Communications and Image Processing, VCIP 2017, vol. 2018-January, pp. 1–4, Feb. 2018, doi: 10.1109/VCIP.2017.8305148.
18. [18] T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature Pyramid Networks for Object Detection,” Dec. 2016, Accessed: May 05, 2021. [Online]. Available: http://arxiv.org/abs/1612.03144
19. [19] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9351, Springer Verlag, 2015, pp. 234–241. doi: 10.1007/978-3-319-24574-4_28.
20. [20] Y. Huang, C. Qiu, and K. Yuan, “Surface defect saliency of magnetic tile,” The Visual Computer, vol. 36, no. 1, pp. 85–96, Jan. 2020, doi: 10.1007/s00371-018-1588-5.
21. [21] J. Lu et al., “SCueU-Net: Efficient Damage Detection Method for Railway Rail,” IEEE Access, vol. 8, pp. 125109–125120, 2020, doi: 10.1109/ACCESS.2020.3007603.
22. [22] X. Dong, C. J. Taylor, and T. F. Cootes, “Defect Detection and Classification by Training a Generic Convolutional Neural Network Encoder,” IEEE Transactions on Signal Processing, vol. 68, pp. 6055–6069, 2020, doi: 10.1109/TSP.2020.3031188.
23. [23] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2016, vol. 2016-Decem, pp. 770–778. doi: 10.1109/CVPR.2016.90.
24. [24] W. Fang et al., “Recognizing Global Reservoirs From Landsat 8 Images: A Deep Learning Approach,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 12, no. 9, pp. 3168–3177, Aug. 2019, doi: 10.1109/JSTARS.2019.2929601.
25. [25] J. Silvestre-Blanes, T. Albero-Albero, I. Miralles, R. Pérez-Llorens, and J. Moreno, “A Public Fabric Database for Defect Detection Methods and Results,” Autex Research Journal, vol. Vol. 19, no. 4, 2019, doi: 10.2478/aut-2019-0035.
26. [26] H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, “Pyramid scene parsing network,” Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, vol. 2017-January, pp. 6230–6239, Nov. 2017, doi: 10.1109/CVPR.2017.660.
27. [27] J. Liu, K. Song, M. Feng, Y. Yan, Z. Tu, and L. Zhu, “Semi-supervised anomaly detection with dual prototypes autoencoder for industrial surface inspection,” Optics and Lasers in Engineering, vol. 136, p. 106324, Jan. 2021, doi: 10.1016/j.optlaseng.2020.106324.
28. [28] Q. Zhou, J. Mei, Q. Zhang, S. Wang, and G. Chen, “Semi-supervised fabric defect detection based on image reconstruction and density estimation,” Textile Research Journal, vol. 91, no. 9–10, pp. 962–972, May 2021, doi: 10.1177/0040517520966733.
29. [29] W. Yuxiang, M. Shiyi, X. Xiang, and H. Shanshan, “DCSNet: A Surface Defect Classification and Segmentation Model by One-Class Learning,” Journal of Physics: Conference Series, vol. 1914, no. 1, p. 012037, May 2021, doi: 10.1088/1742-6596/1914/1/012037.