Enhanced license plate recognition using deep learning and block-based approach

Abstract

This study investigates the effectiveness of current deep learning techniques in license plate detection and makes essential contributions. Instead of classifying the characters on Turkish license plates with a single classifier, the characters are divided into blocks of numbers and letters using various image processing techniques, and a separate classifier is used for each block. The proposed approach improves character classification accuracy and license plate recognition accuracy. This approach eliminated the possibility of misclassifying similar letters and numbers and improved the character classification accuracy from 95.9% to 99.6%. In addition, a new character feature dataset was created, and a deep learning model was trained on this dataset. Integrating this model into the system increased the classification accuracy to 99.7%. The YOLOv8 object detection model, trained using CUDA technology, achieved a mAP of 98.9%. The overall accuracy of the whole system in license plate recognition reached 97.3%. This study proves the effectiveness of current deep learning methods and the proposed block-based character recognition approach in license plate recognition.

Keywords

• License plate recognition
• Image processing
• Convolutional neural networks
• Object detection
• Deep learning

References

1. [1] M. Arora, A. Jain, S. Rustagi, and T. Yadav, “Automatic number plate recognition system using optical character recognition,” International Journal of Scientific Research in Computer Science, Engineering and Information Technology, pp. 986–992, 2019.
2. [2] B. Lavanya and G. Lalitha, “Multileveled ALPR using Block-Binary-Pixel-Sum Descriptor and Linear SVC,” International Journal of Advanced Computer Science and Applications, vol. 13, no. 5, 2022.
3. [3] Y. Kumar, K. Kaur, and G. Singh, “Machine learning aspects and its applications towards different research areas,” in 2020 International conference on computation, automation and knowledge management (ICCAKM), IEEE, 2020, pp. 150–156.
4. [4] W. Weihong and T. Jiaoyang, “Research on license plate recognition algorithms based on deep learning in complex environment,” IEEE Access, vol. 8, pp. 91661–91675, 2020.
5. [5] S. Rahman, J. H. Rony, J. Uddin, and M. A. Samad, “Real-Time Obstacle Detection with YOLOv8 in a WSN Using UAV Aerial Photography,” J Imaging, vol. 9, no. 10, p. 216, 2023.
6. [6] G. Jocher, A. Chaurasia, and J. Qiu, “Ultralytics YOLOv8,” 2023. [Online]. Available: https://github.com/ultralytics/ultralytics
7. [7] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779–788.
8. [8] C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao, “YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 7464–7475.

9. [9] G. Aslı and A. DOĞAN, “License Plate Recognition System Based on Artificial Intelligence with Different Approach,” El-Cezeri, vol. 10, no. 1, pp. 121–136, 2023.
10. [10] H. Shi and D. Zhao, “License Plate Recognition System Based on Improved YOLOv5 and GRU,” IEEE Access, vol. 11, pp. 10429–10439, 2023.
11. [11] A. O. Salau, T. K. Yesufu, and B. S. Ogundare, “Vehicle plate number localization using a modified GrabCut algorithm,” Journal of King Saud University-Computer and Information Sciences, vol. 33, no. 4, pp. 399–407, 2021.
12. [12] T.-A. Pham, “Effective deep neural networks for license plate detection and recognition,” Vis Comput, vol. 39, no. 3, pp. 927–941, 2023.
13. [13] N. Duan, J. Cui, L. Liu, and L. Zheng, “An end to end recognition for license plates using convolutional neural networks,” IEEE Intelligent Transportation Systems Magazine, vol. 13, no. 2, pp. 177–188, 2019.
14. [14] N. Awalgaonkar, P. Bartakke, and R. Chaugule, “Automatic license plate recognition system using ssd,” in 2021 International Symposium of Asian Control Association on Intelligent Robotics and Industrial Automation (IRIA), IEEE, 2021, pp. 394–399.
15. [15] N. P. Ap, T. Vigneshwaran, M. S. Arappradhan, and R. Madhanraj, “Automatic number plate detection in vehicles using faster R-CNN,” in 2020 International conference on system, computation, automation and networking (ICSCAN), IEEE, 2020, pp. 1–6.
16. [16] B. Pu, “Research on Chinese license plate recognition algorithm based on convolution neural network,” in Journal of Physics: Conference Series, IOP Publishing, 2021, p. 032055.
17. [17] M. Driss, I. Almomani, R. Al-Suhaimi, and H. Al-Harbi, “Automatic Saudi Arabian License Plate Detection and Recognition Using Deep Convolutional Neural Networks,” in International Conference of Reliable Information and Communication Technology, Springer, 2021, pp. 3–15.
18. [18] C. Cheng, L. Mei, and J. Zhang, “License plate recognition via deep convolutional neural network,” in IOP Conference Series: Earth and Environmental Science, IOP Publishing, 2018, p. 062030.
19. [19] A. Singh and P. Singh, “License Plate Recognition for Traffic Management,” Journal of Management and Service Science (JMSS), vol. 1, no. 2, pp. 1–14, 2021.
20. [20] I. R. Khan et al., “Automatic License Plate Recognition in Real-World Traffic Videos Captured in Unconstrained Environment by a Mobile Camera,” Electronics (Basel), vol. 11, no. 9, p. 1408, 2022.
21. [21] R. Laroca, L. A. Zanlorensi, G. R. Gonçalves, E. Todt, W. R. Schwartz, and D. Menotti, “An efficient and layout‐independent automatic license plate recognition system based on the YOLO detector,” IET Intelligent Transport Systems, vol. 15, no. 4, pp. 483–503, 2021.
22. [22] S. Montazzolli and C. Jung, “Real-time brazilian license plate detection and recognition using deep convolutional neural networks,” in 2017 30th SIBGRAPI conference on graphics, patterns and images (SIBGRAPI), IEEE, 2017, pp. 55–62.
23. [23] S. M. Silva and C. R. Jung, “Real-time license plate detection and recognition using deep convolutional neural networks,” J Vis Commun Image Represent, vol. 71, p. 102773, 2020.
24. [24] L. Zhang, P. Wang, H. Li, Z. Li, C. Shen, and Y. Zhang, “A Robust Attentional Framework for License Plate Recognition in the Wild,” IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 11, pp. 6967–6976, Nov. 2021, doi: 10.1109/TITS.2020.3000072.
25. [25] Q. Huang, Z. Cai, and T. Lan, “A Single Neural Network for Mixed Style License Plate Detection and Recognition,” IEEE Access, vol. 9, pp. 21777–21785, 2021, doi: 10.1109/ACCESS.2021.3055243.
26. [26] J. A et al., “Enhancing Public Safety through License Plate Recognition for Counter terrorism through Deep Learning Technique,” in 2023 4th International Conference on Signal Processing and Communication (ICSPC), IEEE, Mar. 2023, pp. 96–100. doi: 10.1109/ICSPC57692.2023.10125687.
27. [27] Y. Zou et al., “License plate detection and recognition based on YOLOv3 and ILPRNET,” Signal Image Video Process, vol. 16, no. 2, pp. 473–480, Mar. 2022, doi: 10.1007/s11760-021-01981-8.
28. [28] A. Tourani, A. Shahbahrami, S. Soroori, S. Khazaee, and C. Y. Suen, “A robust deep learning approach for automatic iranian vehicle license plate detection and recognition for surveillance systems,” IEEE Access, vol. 8, pp. 201317–201330, 2020.
29. [29] M. M. Khan, M. U. Ilyas, I. R. Khan, S. M. Alshomrani, and S. Rahardja, “A review of license plate recognition methods employing neural networks,” IEEE Access, 2023.
30. [30] P. Roboflow Universe, “License Plate Recognition Dataset.” Accessed: Mar. 02, 2024. [Online]. Available: https://universe.roboflow.com/roboflow-universe-projects/license-plate-recognition-rxg4e
31. [31] R. K. Varma P, S. Ganta, H. K. B, and P. Svsrk, “A Novel Method for Indian Vehicle Registration Number Plate Detection and Recognition using Image Processing Techniques,” Procedia Comput Sci, vol. 167, pp. 2623–2633, 2020, doi: 10.1016/j.procs.2020.03.324.
32. [32] K. Range, “YOLOv8 by RangeKing.” Accessed: Aug. 09, 2024. [Online]. Available: https://github.com/RangeKing
33. [33] R. G. Jha and A. Samlodia, “GPU-acceleration of tensor renormalization with PyTorch using CUDA,” Comput Phys Commun, vol. 294, p. 108941, 2024.
34. [34] Z. Y. Tan, S. N. Basah, H. Yazid, and M. J. A. Safar, “Performance analysis of Otsu thresholding for sign language segmentation,” Multimed Tools Appl, vol. 80, pp. 21499–21520, 2021.
35. [35] J. Sigut, M. Castro, R. Arnay, and M. Sigut, “OpenCV basics: A mobile application to support the teaching of computer vision concepts,” IEEE Transactions on Education, vol. 63, no. 4, pp. 328–335, 2020.
36. [36] S. Dörterler, “Hybridization of k-means and meta-heuristics algorithms for heart disease diagnosis,” New Trends in Engineering and Applied Natural Sciences, p. 55, 2022.
37. [37] S. Dörterler, H. Dumlu, D. Özdemir, and H. Temurtaş, “Hybridization of Meta-heuristic Algorithms with K-Means for Clustering Analysis: Case of Medical Datasets,” Gazi Journal of Engineering Sciences, vol. 10, no. 1, pp. 1–11, Apr. 2024, doi: 10.30855/gmbd.0705N01.
38. [38] M. Safran, A. Alajmi, and S. Alfarhood, “Efficient Multistage License Plate Detection and Recognition Using YOLOv8 and CNN for Smart Parking Systems,” J Sens, vol. 2024, 2024.
39. [39] F. Aydemir and S. Arslan, “A System Design With Deep Learning and IoT to Ensure Education Continuity for Post-COVID,” IEEE Transactions on Consumer Electronics, 2023.
40. [40] O. Sahin, “Music Genre Classification Based on Song Titles with Long Short-Term Memory,” in 2023 IEEE International Students’ Conference on Electrical, Electronics and Computer Science (SCEECS), IEEE, 2023, pp. 1-5.
41. [41] A. Zaafouri, M. Sayadi, and W. Wu, “A Vehicle License Plate Detection and Recognition Method Using Log Gabor Features and Convolutional Neural Networks,” Cybern Syst, vol. 54, no. 1, pp. 88–103, 2023.