Detection of circuit components on hand-drawn circuit images by using faster R-CNN method

Abstract

In this study, one of deep learning methods, which has been very popular in recent years, is employed for the detection and classification of circuit components in hand-drawn circuit images. Each circuit component located in different positions on the scanned images of hand-drawn circuits, which are frequently used in electrical and electronics engineering, is considered as a separate object. In order to detect the components on the circuit image, Faster Region Based Convolutional Neural Network (R-CNN) method is used instead of conventional methods. With the Faster R-CNN method, which has been developed in recent years to detect and classify objects, preprocessing on image data is minimized, and the feature extraction phase is done automatically. In the study, it is aimed to detect and classify four different circuit components in the scanned images of hand-drawn circuits with high accuracy by using the Python programming language on the Google Colab platform. The circuit components to be detected on the hand-drawn circuits are specified as resistor, inductor, capacitor, and voltage source. For the training of the model used, a data set was created by collecting 800 circuit images consisting of hand drawings of different people. For the detection of the components, the pretrained Faster R-CNN Inception V2 model was used after fine tuning and arrangements depending on the process requirements. The model was trained in 50000 epochs, and the success of the trained model has been tested on the circuits drawn in different styles on the paper. The trained model was able to detect circuit components quickly and with a high rate of performance. In addition, the loss graphics of the model were examined. The proposed method shows its efficiency by quickly detecting each of the 4 different circuit components on the image and classifying them with high performance.

Keywords

• Circuit components
• Classification
• Deep learning
• Faster R-CNN
• Hand-drawn circuits
• Object detection

References

1. 1. Liu, Y. and Y. Xiao, Circuit sketch recognition. Department of Electrical Engineering Stanford University Stanford, CA, 2013.
2. 2. Moetesum, M., et al., Segmentation and recognition of electronic components in hand-drawn circuit diagrams. EAI Endorsed Transactions on Scalable Information Systems, 2018. 5(16).
3. 3. Angadi, M. and R.L. Naika, Handwritten circuit schematic detection and simulation using computer vision approach. International Journal of Computer Science and Mobile Computing, 2014. 3(6): p. 754-761.
4. 4. Dewangan, A. and A. Dhole, KNN based hand drawn electrical circuit recognition. International Journal for Research in Applied Science & Engineering Technology, 2018. 6(6): p. 1111-1115.
5. 5. Dinesh, R., Handwritten Electronic Components Recognition: An Approach Based On HOG+ SVM. Journal of Theoretical & Applied Information Technology, 2018. 96(13).
6. 6. Wang, H., T. Pan, and M.K. Ahsan, Hand-drawn electronic component recognition using deep learning algorithm. International Journal of Computer Applications in Technology, 2020. 62(1): p. 13-19.
7. 7. Sevli, O. and N. Kemaloğlu, Turkish Sign Language digits classification with CNN using different optimizers. International Advanced Researches and Engineering Journal, 2020. 4(3): p. 200-207.
8. 8. Lee, H. and H. Kwon, Going Deeper With Contextual CNN for Hyperspectral Image Classification. IEEE Trans Image Process, 2017. 26(10): p. 4843-4855.

9. 9. Coşkun, M., et al., Face recognition based on convolutional neural network. In 2017 International Conference on Modern Electrical and Energy Systems (MEES). 2017. IEEE.
10. 10. Li, Q., et al., Medical image classification with convolutional neural network. In 2014 13th international conference on control automation robotics & vision (ICARCV). 2014. IEEE.
11. 11. Sardoğan, M., Y. Özen, and A. Tuncer, Faster R-CNN Kullanarak Elma Yaprağı Hastalıklarının Tespiti. Düzce Üniversitesi Bilim ve Teknoloji Dergisi, 2020: p. 1110-1117.
12. 12. Cömert, O., M. Hekim, and A. Kemal, Faster R-CNN Kullanarak Elmalarda Çürük Tespiti. Uluslararası Mühendislik Araştırma ve Geliştirme Dergisi. 11(1): p. 335-341.
13. 13. Ren, Y., C. Zhu, and S. Xiao, Deformable faster r-cnn with aggregating multi-layer features for partially occluded object detection in optical remote sensing images. Remote Sensing, 2018. 10(9): p. 1470.
14. 14. Julca-Aguilar, F.D. and N.S. Hirata, Symbol detection in online handwritten graphics using faster R-CNN. In 2018 13th IAPR International Workshop on Document Analysis Systems (DAS). 2018. IEEE.
15. 15. Yang, J., P. Ren, and X. Kong, Handwriting Text Recognition Based on Faster R-CNN. In 2019 Chinese Automation Congress (CAC). 2019. IEEE.
16. 16. GitHub - tzutalin/labelImg: ?️ LabelImg is a graphical image annotation tool and label object bounding boxes in images. [cited 2021 March 18]; Available from: https://github.com/tzutalin/labelImg.
17. 17. Yuan, Z., Y. Lu, and Y. Xue, Droiddetector: android malware characterization and detection using deep learning. Tsinghua Science and Technology, 2016. 21(1): p. 114-123.
18. 18. Bengio, Y., Learning deep architectures for AI. 2009: Now Publishers Inc.
19. 19. LeCun, Y., Y. Bengio, and G. Hinton, Deep learning. nature, 2015. 521(7553): p. 436-444.
20. 20. Lei, X. and Z. Sui, Intelligent fault detection of high voltage line based on the Faster R-CNN. Measurement, 2019. 138: p. 379-385.
21. 21. Girshick, R., et al., Rich feature hierarchies for accurate object detection and semantic segmentation. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2014.
22. 22. Girshick, R., Fast r-cnn. In Proceedings of the IEEE international conference on computer vision. 2015.
23. 23. Ren, S., et al., Faster r-cnn: Towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01497, 2015.
24. 24. Lin, G., et al., Smoke detection on video sequences using 3D convolutional neural networks. Fire Technology, 2019. 55(5): p. 1827-1847.
25. 25. Lei, J., et al., Efficient power component identification with long short-term memory and deep neural network. EURASIP Journal on Image and Video Processing, 2018. 2018(1): p. 1-14.
26. 26. Szegedy, C., et al., Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
27. 27. Welcome to Colaboratory - Colaboratory. [cited 2021 March 5]; Available from: https://colab.research.google.com/notebooks/intro.ipynb#scrollTo=5fCEDCU_qrC0.