InceptionV3 Based Enriched Feature Integration Network Architecture for Pixel-Level Surface Defect Detection

Öz

In this study, InceptionV3 based Enriched Feature Integration Network (Inc-EFIN) architecture was developed for automatic detection of surface defects. In the proposed architecture, features of all levels of the InceptionV3 network architecture with the same height and width are extracted and combined. As a result of merging, 5 feature maps with different dimensions were obtained. Channel-Based Squeeze and Excitation (CSE) block has been applied to reveal important details in these feature maps. The CSE block strengthens important details by examining the channels in the feature map. In Feature Pyramid Network (FPN) module, information from low-level feature maps containing spatial details were transferred to high-level feature maps containing semantic details. Then, for the final feature map in the proposed architecture, 4 different feature maps obtained at the end of the FPN module were combined using the Feature Integration and Signification (FIS) module. The feature map combined in the FIS module was passed through the Spatial and Channel-based Squeeze and Excitation (SCSE) block, enhancing the spatial and semantic information that may be important for defect detection in the best way. Defect detection results were obtained by using convolution and sigmoid layers in the last layer of the Inc-EFIN architecture. MT, MVTec-Texture, and DAGM datasets were used to calculate the pixel-level defect detection success of the Inc-EFIN architecture. In experimental studies, Inc-EFIN architecture achieved higher performance than the latest technologies in the literature with 77.44% mIoU, 81.2% mIoU and 79.46% mIoU performance results in MT, MVTec-Texture and DAGM datasets, respectively. Keywords: Pixel-Level Surface Defects Detection, Convolutional Neural Network, Squeeze and Excitation Block, Feature Pyramid Networks

Anahtar Kelimeler

• Pixel-level surface defects detection
• Convolutional neural network
• Squeeze and Excitation Block
• Feature Pyramid Networks

Proje Numarası

FDK-2021-2725

Piksel seviyesinde yüzey hata tespiti için InceptionV3 tabanlı zenginleştirilmiş öznitelik entegrasyon ağ mimarisi

Öz

Bu çalışmada, otomatik yüzey hata tespiti için InceptionV3 tabanlı Zenginleştirilmiş Öznitelik Entegrasyon Ağ (Inc-ZÖEA) mimarisi geliştirilmiştir. Önerilen mimaride, InceptionV3 ağ mimarisinin her seviyesindeki öznitelikleri aynı yükseklik ve genişliğe sahip öznitelikler çıkartılmış ve birleştirilmiştir. Birleştirme sonucunda farklı boyutlara sahip olan 5 öznitelik haritası elde edilmiştir. Bu öznitelik haritalarındaki önemli detayları ortaya çıkartmak için Kanal Bazlı Sıkma ve Uyarlama (KSU) bloğu uygulanmıştır. KSU bloğu, öznitelik haritasındaki kanalları inceleyerek önemli ayrıntıları güçlendirmektedir. Öznitelik Piramit Ağ (ÖPA) modülünde mekânsal detayları içeren düşük seviyeli öznitelik haritalarındaki bilgiler, anlamsal detayları içeren yüksek seviyeli öznitelik haritalarına aktarılmıştır. Daha sonra önerilen mimaride nihai öznitelik haritası için Öznitelik Entegrasyon ve Anlamlandırma (ÖEA) modülü kullanılarak ÖPA modülünün sonunda elde edilen 4 farklı öznitelik haritaları birleştirilmiştir. ÖEA modülünde birleştirilen öznitelik haritası Mekânsal ve Kanal Bazlı Sıkma ve Uyarlama (MKSU) bloğundan geçirilerek hata tespiti için önemli olabilecek mekânsal ve anlamsal bilgiler en iyi şekilde güçlendirilmiştir. Inc-ZÖEA mimarisinin son katmanında evrişim ve sigmoid katmanları kullanılarak hata tespit sonucu elde edilmiştir. Inc-ZÖEA mimarisinin piksel seviyesinde hata tespit başarısını ölçmek için MT, MVTec-Doku ve DAGM veri setleri kullanılmıştır. Deneysel çalışmalarda, MT, MVTec-Doku ve DAGM veri setlerinde sırası ile Inc-ZÖEA mimarisi %77,44 mIoU, %81,2 mIoU ve %79,46 mIoU başarım sonuçları ile literatürde yer alan son teknolojilere göre daha yüksek başarımlar sağlamıştır

Anahtar Kelimeler

• Piksel Seviyesinde yüzey kusurları algılama
• Evrişimsel sinir ağı
• Sıkma ve Uyarma Bloğu
• Özellik Piramit Ağı

Destekleyen Kurum

İnönü Üniversitesi Bilimsel Araştırma Projeleri Koordinasyon Birimi

Proje Numarası

FDK-2021-2725

Teşekkür

Bu çalışma İnönü Üniversitesi Bilimsel Araştırma Projeleri Koordinasyon Birimi tarafından desteklenmiştir (Proje Numarası: FDK-2021-2725).

Kaynakça

1. 1. K. Hanbay, M. F. Talu, and Ö. F. Özgüven, Fabric defect detection systems and methods—A systematic literature review, Optik 127(24), 11960–11973, 2016.
2. 2. H. Dong, K. Song, Y. He, J. Xu, Y. Yan, and Q. Meng, PGA-Net: Pyramid Feature Fusion and Global Context Attention Network for Automated Surface Defect Detection, IEEE Transactions on Industrial Informatics 16(12), 7448–7458, 2020.
3. 3. M. H. Karimi and D. Asemani, Surface defect detection in tiling Industries using digital image processing methods: Analysis and evaluation, ISA Transactions 53(3), 834–844, 2014.
4. 4. S. R. Aghdam, E. Amid, and M. F. Imani, A fast method of steel surface defect detection using decision trees applied to LBP based features, Proceedings of the 2012 7th IEEE Conference on Industrial Electronics and Applications, ICIEA 2012 1447–1452, 2012.
5. 5. J. A. Tsanakas, D. Chrysostomou, P. N. Botsaris, and A. Gasteratos, Fault diagnosis of photovoltaic modules through image processing and Canny edge detection on field thermographic measurements, http://dx.doi.org/10.1080/14786451.2013.826223 34(6), 351–372, 2013.
6. 6. K. L. Mak, P. Peng, and K. F. C. Yiu, Fabric defect detection using morphological filters, Image and Vision Computing 27(10), 1585–1592, 2009.
7. 7. X. Bai, Y. Fang, W. Lin, L. Wang, and B. F. Ju, Saliency-based defect detection in industrial images by using phase spectrum, IEEE Transactions on Industrial Informatics 10(4), 2135–2145, 2014.
8. 8. G. Liu and X. Zheng, Fabric defect detection based on information entropy and frequency domain saliency, The Visual Computer 2020 37:3 37(3), 515–528, 2020.

9. 9. X. Dong, C. J. Taylor, and T. F. Cootes, A Random Forest-Based Automatic Inspection System for Aerospace Welds in X-Ray Images, IEEE Transactions on Automation Science and Engineering, 2020.
10. 10. L. Qiu, X. Wu, and Z. Yu, A High-Efficiency Fully Convolutional Networks for Pixel-Wise Surface Defect Detection, IEEE Access 7, 15884–15893, 2019.
11. 11. J. Cao, G. Yang, and X. Yang, A Pixel-Level Segmentation Convolutional Neural Network Based on Deep Feature Fusion for Surface Defect Detection, IEEE Transactions on Instrumentation and Measurement 70,, 2021.
12. 12. H. Firat, 3B ESA Tabanlı ResNet50 Kullanılarak Hiperspektral Görüntülerin Sınıflandırılması Classification of Hyperspectral Images Using 3D CNN Based ResNet50, 2021 29th Signal Processing and Communications Applications Conference, SIU) 6–9, 2021.
13. 13. H. Uzen, H. Firat, A. Karci, and D. Hanbay, Automatic Thresholding Method Developed with Entropy for Fabric Defect Detection, in 2019 International Conference on Artificial Intelligence and Data Processing Symposium, IDAP 2019, Institute of Electrical and Electronics Engineers Inc., 2019.
14. 14. H. Uzen, M. Turkoglu, and D. Hanbay, Texture defect classification with multiple pooling and filter ensemble based on deep neural network, Expert Systems with Applications 175, 114838, 2021.
15. 15. L. Yi, G. Li, and M. Jiang, An End-to-End Steel Strip Surface Defects Recognition System Based on Convolutional Neural Networks, steel research international 88(2), 1600068, 2017.
16. 16. S. Jain, G. Seth, A. Paruthi, U. Soni, and G. Kumar, Synthetic data augmentation for surface defect detection and classification using deep learning, Journal of Intelligent Manufacturing 2020 1–14, 2020.
17. 17. J. Long, E. Shelhamer, and T. Darrell, Fully Convolutional Networks for Semantic Segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence 39(4), 640–651, 2014.
18. 18. S. Ren, K. He, R. Girshick, and J. Sun, Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, IEEE Transactions on Pattern Analysis and Machine Intelligence 39(6), 1137–1149, 2017.
19. 19. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, SSD: Single Shot MultiBox Detector, Lecture Notes in Computer Science, including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 9905 LNCS, 21–37, 2015.
20. 20. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, You Only Look Once: Unified, Real-Time Object Detection, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2016-December, 779–788, 2015.
21. 21. Y. He, K. Song, Q. Meng, and Y. Yan, An End-to-End Steel Surface Defect Detection Approach via Fusing Multiple Hierarchical Features, IEEE Transactions on Instrumentation and Measurement 69(4), 1493–1504, 2020.
22. 22. S. Yanan, Z. Hui, L. Li, and Z. Hang, Rail Surface Defect Detection Method Based on YOLOv3 Deep Learning Networks, Proceedings 2018 Chinese Automation Congress, CAC 2018 1563–1568, 2019.
23. 23. J. Redmon and A. Farhadi, YOLOv3: An Incremental Improvement,, 2018.
24. 24. H. Yuan, H. Chen, S. Liu, J. Lin, and X. Luo, A deep convolutional neural network for detection of rail surface defect, 2019 IEEE Vehicle Power and Propulsion Conference, VPPC 2019 - Proceedings, 2019.
25. 25. M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, MobileNetV2: Inverted Residuals and Linear Bottlenecks, IEEE Computer Society, 2018), pp. 4510–4520.
26. 26. A. Chaurasia and E. Culurciello, LinkNet: Exploiting encoder representations for efficient semantic segmentation, 2017 IEEE Visual Communications and Image Processing, VCIP 2017 2018-January, 1–4, 2018.
27. 27. T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, Feature Pyramid Networks for Object Detection,, 2016.
28. 28. O. Ronneberger, P. Fischer, and T. Brox, U-Net: Convolutional Networks for Biomedical Image Segmentation, in Lecture Notes in Computer Science, Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer Verlag, 2015), 9351, pp. 234–241.
29. 29. Y. Huang, C. Qiu, and K. Yuan, Surface defect saliency of magnetic tile, The Visual Computer 36(1), 85–96, 2020.
30. 30. J. Lu, B. Liang, Q. Lei, X. Li, J. Liu, J. Liu, J. Xu, and W. Wang, SCueU-Net: Efficient Damage Detection Method for Railway Rail, IEEE Access 8, 125109–125120, 2020.
31. 31. X. Dong, C. J. Taylor, and T. F. Cootes, Defect Detection and Classification by Training a Generic Convolutional Neural Network Encoder, IEEE Transactions on Signal Processing 68, 6055–6069, 2020.
32. 32. M. Rudolph, B. Wandt, and B. Rosenhahn, Same Same But DifferNet: Semi-Supervised Defect Detection with Normalizing Flows, arXiv, 2020.
33. 33. J. Liu, K. Song, M. Feng, Y. Yan, Z. Tu, and L. Zhu, Semi-supervised anomaly detection with dual prototypes autoencoder for industrial surface inspection, Optics and Lasers in Engineering 136, 106324, 2021.
34. 34. T. Defard, A. Setkov, A. Loesch, and R. Audigier, PaDiM: A Patch Distribution Modeling Framework for Anomaly Detection and Localization, Lecture Notes in Computer Science, including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12664 LNCS, 475–489, 2021.
35. 35. D. S. Tan, Y.-C. Chen, T. P.-C. Chen, and W.-C. Chen, TrustMAE: A Noise-Resilient Defect Classification Framework using Memory-Augmented Auto-Encoders with Trust Regions,, 2020.
36. 36. J. Wang, G. Xu, C. Li, Z. Wang, and F. Yan, Surface Defects Detection Using Non-convex Total Variation Regularized RPCA with Kernelization, IEEE Transactions on Instrumentation and Measurement 70,, 2021.
37. 37. Y. Song, Z. Liu, J. Wang, R. Tang, G. Duan, and J. Tan, Multiscale Adversarial and Weighted Gradient Domain Adaptive Network for Data Scarcity Surface Defect Detection, IEEE Transactions on Instrumentation and Measurement 70,, 2021.
38. 38. X. Cheng and J. Yu, RetinaNet with Difference Channel Attention and Adaptively Spatial Feature Fusion for Steel Surface Defect Detection, IEEE Transactions on Instrumentation and Measurement 70,, 2021.
39. 39. S. Deitsch, V. Christlein, S. Berger, C. Buerhop-Lutz, A. Maier, F. Gallwitz, and C. Riess, Automatic Classification of Defective Photovoltaic Module Cells in Electroluminescence Images, Solar Energy 185, 455–468, 2018.
40. 40. Z. Lin, H. Ye, B. Zhan, and X. Huang, An Efficient Network for Surface Defect Detection, Applied Sciences 2020, Vol. 10, Page 6085 10(17), 6085, 2020.
41. 41. F. Akhyar, C. Y. Lin, K. Muchtar, T. Y. Wu, and H. F. Ng, High efficient single-stage steel surface defect detection, 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS 2019, 2019.
42. 42. L. Liu, Y. Zhu, M. R. Ur Rahman, P. Zhao, and H. Chen, Surface Defect Detection of Solar Cells Based on Feature Pyramid Network and GA-Faster-RCNN, Proceedings - 2nd China Symposium on Cognitive Computing and Hybrid Intelligence, CCHI 2019 292–297, 2019.
43. 43. J. Wang and Z. Meng, Deformable Feature Pyramid Network for Aluminum Profile Surface Defect Detection, Journal of Physics: Conference Series 1544(1), 012074, 2020.
44. 44. S. Wang, X. Xia, L. Ye, and B. Yang, Steel Surface Defect Detection Using Transfer Learning and Image Segmentation, 2020 IEEE 6th International Conference on Computer and Communications, ICCC 2020 420–425, 2020.
45. 45. H. Li, X. Fu, and T. Huang, Research on Surface Defect Detection of Solar Pv Panels Based on Pre-Training Network and Feature Fusion, IOP Conference Series: Earth and Environmental Science 651(2), 022071, 2021.
46. 46. J. Luo, Z. Yang, S. Li, and Y. Wu, FPCB Surface Defect Detection: A Decoupled Two-Stage Object Detection Framework, IEEE Transactions on Instrumentation and Measurement 70,, 2021.
47. 47. Z. Fan, C. Li, Y. Chen, J. Wei, G. Loprencipe, X. Chen, and P. Di Mascio, Automatic crack detection on road pavements using encoder-decoder architecture, Materials 13(13), 1–18, 2020.
48. 48. K. Bousabarah, M. Ruge, J. S. Brand, M. Hoevels, D. Rueß, J. Borggrefe, N. Große Hokamp, V. Visser-Vandewalle, D. Maintz, H. Treuer, and M. Kocher, Deep convolutional neural networks for automated segmentation of brain metastases trained on clinical data, Radiation Oncology 15(1), 1–9, 2020.
49. 49. J. Wang, P. Lv, H. Wang, and C. Shi, SAR-U-Net: Squeeze-and-excitation block and atrous spatial pyramid pooling based residual U-Net for automatic liver segmentation in Computed Tomography, Computer Methods and Programs in Biomedicine 208, 106268, 2021.
50. 50. A. G. Roy, N. Navab, and C. Wachinger, Recalibrating Fully Convolutional Networks With Spatial and Channel Squeeze and Excitation Blocks, IEEE Transactions on Medical Imaging 38(2), 540–549, 2019.
51. 51. J. Hu, L. Shen, S. Albanie, G. Sun, and E. Wu, Squeeze-and-Excitation Networks, IEEE Transactions on Pattern Analysis and Machine Intelligence 42(8), 2011–2023, 2017.
52. 52. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, Rethinking the Inception Architecture for Computer Vision, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2016-December, 2818–2826, 2015.
53. 53. H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, Pyramid scene parsing network, Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 2017-January, 6230–6239, 2017.
54. 54. P. Bergmann, M. Fauser, D. Sattlegger, and C. Steger, MVTEC ad-A comprehensive real-world dataset for unsupervised anomaly detection, in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society, 2019), 2019-June, pp. 9584–9592.
55. 55. M. Wieler and T. Hahn, Weakly Supervised Learning for Industrial Optical Inspection | Heidelberg Collaboratory for Image Processing, HCI), https://hci.iwr.uni-heidelberg.de/content/weakly-supervised-learning-industrial-optical-inspection (accessed May 07, 2021).