MobileMRZNet: Efficient and Lightweight MRZ Detection for Mobile Devices

Abstract

The Machine Readable Zone (MRZ) is a standardized section found on identification documents (IDs) that adhere to the International Civil Aviation Association (ICAO) Document 9303. The MRZ region contains sensitive personal information about the document holder, and a portion of this information is utilized to establish communication between the passive chip within the ID and a mobile device via Near Field Communication (NFC) protocol. This communication is crucial as the data retrieved from the ID's chip is subsequently used in authentication steps such as face or fingerprint recognition. Thus, accurate detection of the personal information within the MRZ region is vital. In this research study, we propose a fast and lightweight approach for MRZ region detection called MobileMRZNet, which is based on the BlazeFace model. The MobileMRZNet architecture is specifically designed for mobile Graphical Processing Units (GPUs) and enables rapid and precise detection of MRZ regions. To train and evaluate the model, a dataset consisting of both simulated and real data was created using Turkish national IDs. The BlazeFace model was reconfigured and trained specifically for MRZ region detection. The detector, based on BlazeFace and trained on augmented real and simulated data, demonstrates excellent generalization capabilities for deployment with real IDs. Both qualitative and quantitative results confirm the superiority of our proposed method. The mean Intersection over Union (IoU) for the first frame, without utilizing any layout alignment for IDs, achieves an accuracy of approximately 81%. For character recognition, the method achieves 100% accuracy after three consecutive frames. The model operates in less than 10 milliseconds on a mobile device, and its size is around 400 KB, making it significantly fast, lightweight, and robust compared to any existing MRZ detection methods.

Keywords

• MRZ detection
• travel documents
• ID cards
• BlazeFace model
• biometric identification

Supporting Institution

Kobil Teknoloji Lti. Şti.

Project Number

TUBITAK-TEYDEB Project No. 3201086

References

1. [1] Hall, R., Dodds, G., Triggs, S. ”The World of William Notman. David R. Godine”, pp. 46, 47, 1993. ISBN 9780879239398. Retrieved 2022- 10-13.
2. [2] Doulman, Jane, and David Lee (2008). Every Assistance & Protection: A History of the Australian Passport. Federation Press. p. 56. ISBN 9781862876873. Retrieved 2022-10-13.
3. [3] ICAO Doc 9303: Machine Readable Travel. Seventh Edition. Parts 2-7 (International Civil Aviation Organization, 2015)
4. [4] Monnerat, J., Vaudenay, S., & Vuagnoux, M. ”About machine-readable travel documents”, Springer, 2007.
5. [5] Want, R., ”Near field communication”, IEEE Pervasive Computing, vol. 10, no. 3, pp. 4–7, 2011.
6. [6] Chan, P. K., Choy, C. S., Chan, C. F., & Pun, K. P., ”Preparing smartcard for the future: from passive to active”, IEEE Transactions on Consumer Electronics, vol. 50, no. 1, pp. 245–250, 2004.
7. [7] Hartl, A., Arth, C., and Schmalstieg, D., “Real-time detection and recognition of machine-readable zones with mobile devices”, 10th International Conference on Computer Vision Theory and Applications, vol. 3, pp. 79-–87, 2015.
8. [8] Visilter, Y. V., Zheltov, S. Y., & Lukin, A. A., ”Development of OCR system for portable passport and visa reader”, In Document Recognition and Retrieval VI, Vol. 3651, pp. 194–199, 1999.

9. [9] Kwon, Y.-B. and Kim, J.-H. “Recognition based verification for the machine readable travel documents”, In 7th International Workshop on Graphics Recognition, pp. 1-–10, 2007.
10. [10] Kim, K.-B. and Kim, S. “A passport recognition and face verification using enhanced fuzzy ART based RBF network and PCA algorithm,” Neurocomputing, vol. 71, pp. 3202–3210, 2008.
11. [11] Kim, J. ”Recognition of Passport MRZ Information Using Combined Neural Networks”, Journal of Korea Society of Digital Industry and Information Management, vol. 15, no. 4, pp. 149–157, 2019.
12. [12] Liu, Y., James, H., Gupta, O., & Raviv, D. ”MRZ code extraction from visa and passport documents using convolutional neural networks”, International Journal on Document Analysis and Recognition (IJDAR), vol. 25, no.1, pp. 29–39, 2022.
13. [13] Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L. C. ”Mobilenetv2: Inverted residuals and linear bottlenecks”, In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4510–4520, 2018.
14. [14] Tretyakov, K.: PassportEye: Extraction of machine-readable zone information from passports, visas and id-cards via OCR (2016). URL https://github.com/konstantint/PassportEye
15. [15] doubango.org, UltimateMRZ (2020). https://github.com/Douban goTelecom/ultimateMRZ-SDK, Accessed 01 Oct 2023.
16. [16] Kostro, D., Zasso, M.: (2020). URLhttps://github.com/image-js/mrzdetection
17. [17] Lyu, P., Liao, M., Yao, C., Wu, W., Bai, X.: Masktextspotter: An end-toend trainable neural network forspotting text with arbitrary shapes. In: Proceedings ofthe European Conference on Computer Vision (ECCV), pp. 67-–83, 2018.
18. [18] Xing, L., Tian, Z., Huang, W., Scott, M.R. ”Convolutional Character Networks”, arXiv:1910.07954[cs], 2019.
19. [19] Arlazarov, V.V., Bulatov, K., Chernov, T., Arlazarov, V.L.: MIDV- 500: a dataset for identity document analysis and recognition on mobile devices in video stream. Comput. Opt. 43, 818–824 (2019).
20. [20] Bulatov, K.,Matalov, D., Arlazarov, V.V.: MIDV-2019: Challenges of the Modern Mobile-Based Document OCR. ICMV 2019, pp. 114332N1–114332N6. Bellingham,Washington 98227-0010USA (2020).
21. [21] Bazarevsky, V., Kartynnik, Y., Vakunov, A., Raveendran, K., & Grundmann, M. (2019). BlazeFace: Sub-millisecond neural face detection on mobile gpus. arXiv preprint arXiv:1907.05047.
22. [22] Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., & Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.
23. [23] Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., & Berg, A. C. (2016, October). Ssd: Single shot multibox detector. In European conference on computer vision (pp. 21-37). Springer, Cham.
24. [24] Chen, S., Liu, Y., Gao, X., & Han, Z. ”Mobilefacenets: Efficient cnns for accurate real-time face verification on mobile devices”, In Chinese onference on Biometric Recognition, pp. 428–438, 2018.
25. [25] Bayar, N., G¨uzel, K., & Kumlu, D. ”A Novel BlazeFace Based Preprocessing for MobileFaceNet in Face Verification”, 45th International Conference on Telecommunications and Signal Processing (TSP), pp. 179–182, 2022.