A Novel Deep Learning Method for Detecting Defects in Mobile Phone Screen Surface Based on Machine Vision

Year 2023, Volume: 27 Issue: 2, 442 - 451, 30.04.2023

İsmail AKGÜL

https://doi.org/10.16984/saufenbilder.1221346

Abstract

With the innovations in technology, the interest in the use of mobile devices is increasing day by day. Any defect that may occur during the production of smart mobile phones, which is among mobile devices, causes significant damage to both the manufacturer and the user. The careful detection of defects that may occur on the screen glass, which is one of the most striking defects among these defects, with the human eye significantly affects the workforce cost. Therefore, it is important to detect defects with the help of software. In recent years, many methods based on machine vision have been developed for the detection of any object or difference in the image.
In this study, a new model structure called Yolo-MSD, based on machine vision and the Yolo-v3 deep learning model, which detects and classifies oil, scratch, and stain defect types on the glass on the touch screen surface used in the design of smart mobile phones, is proposed. The proposed model structure (Yolo-MSD) is obtained by reducing the number of blocks in the Darknet-53 network structure developed in Yolo-v3. As a result of the training, a success rate of 98.50% with the Yolo-v3 model and 98.72% with the Yolo-MSD model was achieved in detecting and classifying defect types. Therefore, it has been observed that the Yolo-MSD model structure is better than the Yolo-v3 model structure by making better feature extraction from the types of defects on the screen glass since it is both faster and has less complexity.

Keywords

Machine vision, deep learning, Yolo-v3, Yolo-MSD, defect detection

References

• [1] C. Jian, J. Gao, Y. Ao, “Automatic surface defect detection for mobile phone screen glass based on machine vision,” Applied Soft Computing, vol. 52, pp. 348-358, 2017.
• [2] L. Meiju, Z. Rui, G. Xifeng, Z. Junrui, “Application of improved Otsu threshold segmentation algorithm in mobile phone screen defect detection,” In 2020 Chinese Control And Decision Conference (CCDC), Hefei, 2020, pp. 4919-4924.
• [3] L. Yuan, Z. Zhang, X. Tao, “The development and prospect of surface defect detection based on vision measurement method,” In 2016 12th World Congress on Intelligent Control and Automation (WCICA), Guilin, 2016, pp. 1382-1387.
• [4] Z. C. Yuan, Z. T. Zhang, H. Su, L. Zhang, F. Shen, F. Zhang, “Vision-based defect detection for mobile phone cover glass using deep neural networks,” International Journal of Precision Engineering and Manufacturing, vol. 19, no. 6, 2018.
• [5] Y. Lv, L. Ma, H. Jiang, “A mobile phone screen cover glass defect detection model based on small samples learning,” In 2019 IEEE 4th International Conference on Signal and Image Processing (ICSIP), Wuxi, 2019, pp. 1055-1059.
• [6] J. Jiang, P. Cao, Z. Lu, W. Lou, Y. Yang, “Surface defect detection for mobile phone back glass based on symmetric convolutional neural network deep learning,” Applied Sciences, vol. 10, no. 10, pp. 1-13, 2020.
• [7] Y. Zhu, R. Ding, W. Huang, P. Wei, G. Yang, Y. Wang, “HMFCA-Net: Hierarchical multi-frequency based Channel attention net for mobile phone surface defect detection,” Pattern Recognition Letters, vol.153, pp. 118-125, 2022.
• [8] J. Pan, D. Zeng, Q. Tan, Z. Wu, Z. Ren, “EU‐Net: A novel semantic segmentation architecture for surface defect detection of mobile phone screens,” IET Image Processing, vol. 6, pp. 2568–2576, 2022.
• [9] J. Zhang, Y. Li, C. Zuo, M. Xing, “Defect detection of mobile phone screen based on improved difference image method,” In 2019 International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS), Shanghai, 2019, pp. 86-92.
• [10] Z. Ren, F. Fang, N. Yan, Y. Wu, “State of the art in defect detection based on machine vision,” International Journal of Precision Engineering and Manufacturing-Green Technology, vol. 9, pp. 661–691, 2021.
• [11] M. Eshkevari, M. J. Rezaee, M. Zarinbal, H. Izadbakhsh, “Automatic dimensional defect detection for glass vials based on machine vision: A heuristic segmentation method,” Journal of Manufacturing Processes, vol. 68, pp. 973-989, 2021.
• [12] C., Jian, J. Gao, Y. Ao, “Imbalanced defect classification for mobile phone screen glass using multifractal features and a new sampling method,” Multimedia Tools and Applications, vol. 76, no. 22, 24413-24434, 2017.
• [13] H. Chen, “CNN-based surface defect detection of smartphone protective screen,” 3rd International Symposium on Big Data and Applied Statistics, Kunming, China, 2020, pp. 1-7.
• [14] S. Qi, J. Yang, Z. Zhong, “A review on industrial surface defect detection based on deep learning technology,” In 2020 the 3rd international conference on machine learning and machine intelligence, Hangzhou, China, 2020, pp. 24-30.
• [15] W. Huang, C. Zhang, X. Wu, J. Shen, Y. Li, “The detection of defects in ceramic cell phone backplane with embedded system,” Measurement, vol. 181 no. 2021, pp. 1-7, 2021.
• [16] Y. Park, I. S. Kweon, “Ambiguous surface defect image classification of AMOLED displays in smartphones,” IEEE Transactions on Industrial Informatics, vol. 12, no. 2, pp. 597-607, 2016.
• [17] T. Wang, C. Zhang, R. Ding, G. Yang, “Mobile phone surface defect detection based on improved faster r-cnn,” In 2020 25th International Conference on Pattern Recognition (ICPR), Milan, 2021, pp. 9371-9377.
• [18] C. Li, X. Zhang, Y. Huang, C. Tang, S. Fatikow, “A novel algorithm for defect extraction and classification of mobile phone screen based on machine vision,” Computers & Industrial Engineering, vol. 146, no. 2020, pp. 1-14, 2020.
• [19] J. Zhang, R. Ding, M. Ban, T. Guo, “FDSNeT: An Accurate Real-Time Surface Defect Segmentation Network,” In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore, 2022, pp. 3803-3807
• [20] C. Wang, C. Li, Y. Huang, X. Zhang, “Surface defect inspection and classification for glass screen of mobile phone,” In Tenth International Conference on Graphics and Image Processing (ICGIP 2018), Chengdu, 2019, pp. 527-536.
• [21] Z. Jianguo, L. Ying, Q. Jiakun, J. Tiantian, L. Jun, “Surface scratch detection of mobile phone screen based on machine vision,” Journal of Applied Optics, vol. 41, no. 5, pp. 984-989, 2020.
• [22] “Github”, Nov. 02, 2022. [Online]. Available:https://github.com/jianzhang96/MSD
• [23] D. Tzutalin, “LabelImg”, Nov. 8, 2022. [Online]. Available: https://github.com/tzutalin/labelImg
• [24] J. Redmon, A. Farhadi, “Yolov3: An incremental improvement,” arXiv preprint, pp. 1-6, 2018.
• [25] Google Colaboratory, “Colab”, Nov. 11, 2022. [Online]. Available: https://colab.research.google.com