Mask Fitting on Face Images Based on Morphing and Masked Face Recognition

Abstract

Viruses such as Covid-19 cause serious respiratory diseases, making the use of face masks important. For this reason, it is expected that face recognition and verification systems can also work with masked faces. There are no masked faces in the data sets created for face recognition systems, and today, masks of different models and patterns can be used in daily life. These reduce the success of face recognition systems. In this study, first, a wide masked face data set was produced by fitting different types of masks to the images in the existing face datasets. In the production of masked faces close to nature, the automatic mask fitting process was carried out with the image morphing technique. Then, a deep learning based model was developed for the recognition of masked/unmasked faces, and the dataset created with the automatic mask fitting technique was used to train the model. In the experiments using CASIA-WebFace and LFW (Labeled Faces in the Wild) datasets, masked face recognition performance was achieved over 96.5%.

Keywords

• Masked face recognition
• Biometric identification
• Automatic mask fitting
• Image morphing
• Deep learning

Yüz Görüntülerine Morflemeye Dayalı Maske Giydirme ve Maskeli Yüz Tanıma

Abstract

Covid-19 gibi virüslerin ciddi solunum yolu hastalıklarına neden olması, yüz maskelerinin kullanımını önemli hale getirmiştir. Bu nedenle yüzden kişi doğrulama ve tanıma yapan sistemlerin maskeli yüzler üzerinde de çalışabilmesi beklenmektedir. Yüz tanıma sistemleri için oluşturulan veri setlerinde maskeli yüzler olmamakla birlikte günümüzde farklı model ve desenlerde maskeler kullanılabilmektedir. Bunlar yüz tanıma sistemlerinin başarısını düşürmektedir. Bu çalışmada öncelikle maskeli yüz veri seti üretmek için mevcut yüz veri setlerine farklı tipteki maskelerin giydirilmesine çalışılmıştır. Morfleme tekniği kullanılarak yüzün pozisyonlarına uygun olarak doğala yakın otomatik giydirme işlemi gerçekleştirilmiştir. Daha sonra maskeli/maskesiz yüzlerin tanınması için derin öğrenmeye dayalı bir model geliştirilmiş ve otomatik maske giydirme tekniği ile oluşturulan veri seti denenmiştir. CASIA-WebFace ve LFW (Labeled Faces in the Wild) veri setleri kullanılarak gerçekleştirilen deneylerde %96.5’in üzerinde maskeli yüz tanıma başarımı elde edilmiştir.

Keywords

• Maskeli yüz tanıma
• Biyometrik kimlik tanıma
• Otomatik maske giydirme
• Görüntü morfleme
• Derin öğrenme

References

1. [1] Dlib C++ Kütüphanesi, 2021. http://dlib.net/ (Erişim Tarihi : 15.12.2021).
2. [2] Yüz Hizalama Detektörü GTX kütüphanesi, 2015. https://github.com/davisking/dlib-models (Erişim Tarihi : 22.09.2021).
3. [3] Schroff, F., Kalenichenko, D., Philbin, J. 2015. Facenet: A Unified Embedding for Face Recognition and Clustering. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 7-12 June, Boston, MA, USA, 815-823.
4. [4] Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A. A. 2017. Inception-v4, Inception-Resnet and The Impact of Residual Connections on Learning. Thirty-first AAAI Conference on Artificial Intelligence (AAAI17), 4-9 February, San Francisco, California USA, 4278-4284. 2017.
5. [5] Zhang, K., Zhang, Z., Li, Z., Qiao, Y. 2016. Joint face Detection and Alignment Using Multitask Cascaded Convolutional Networks. IEEE Signal Processing Letters, 23(10), 1499 – 1503.
6. [6] Qi, C., Yang, L. 2020. Face Recognition in The Scene of Wearing A Mask. 2020 International Conference on Advance in Ambient Computing and Intelligence (ICAACI). 12-13 September, Ottawa, ON, Canada, 77-80.
7. [7] Alrikabi, J.M., Alibraheemi, K. H. 2021. A Combination Approach for Masked Face Recognition Based on Deep Learning. AM, 9(3), 499–520.
8. [8] Wang, Z., Wang, G., Huang, B., Xiong, Z., Hong, Q., Wu, H., Yi, P., Jiang, K., Wang, N., Pei, Y., Chen, H., Miao,Y., Huang, Z., Liang, J. 2020. Masked Face Recognition Dataset and Application. arXiv preprint arXiv:2003.09093.

9. [9] Mazli Shahar M.S., Mazalan, L. 2021. Face Identity for Face Mask Recognition System. 2021 IEEE 11th IEEE Symposium on Computer Applications &Industrial Electronics (ISCAIE). 3-4 April, Penang, Malaysia, 42-47.
10. [10] Hariri, W. 2021. Efficient Masked Face Recognition Method during the COVID-19 Pandemic. arXiv preprint arXiv:2105.03026.
11. [11] Vu, H.N., Nguyen, M.H., Pham, C. 2021. Masked face recognition with convolutional neural networks and local binary patterns. Applied Intelligence, Springer, Online.
12. [12] Ejaz, M.S., Islam, M. R. 2019. Masked Face Recognition Using Convolutional Neural Network. 2019 International Conference on Sustainable Technologies for Industry 4.0 (STI), 24-25 December, Dhaka, Bangladesh, 1-6.
13. [13] Jeevan, G., Zacharias, G.C., Nair, M.S., Rajan, J. 2022. An Empirical Study of The Impact of Masks on Face Recognition. Pattern Recognition, Volume 122(2022) 108308.
14. [14] Luo, X., He, X., Qing, L., Chen, X., Liu, L., Xu, Y. 2020. EyesGAN: Synthesize Human Face from Human Eyes. Neurocomputing, 404(2020), 213-226.
15. [15] SMalakar, S., Chiracharit, W., Chamnongthai, K., Charoenpong, T. 2021. Masked Face Recognition Using Principal Component Analysis and Deep Learning. 2021 18th International Conference on Electrical Engineering/Electronics Computer Telecommunications and Information Technology (ECTI-CON), 19-22 May, Chiang, Thailand, 785-788.
16. [16] Anwar, A., Raychowdhury, A. 2020. Masked Face Recognition for Secure Authentication. arXiv preprint arXiv:2008.11104.
17. [17] Jang, Y., Gunes, H., Patras, I. 2019. Registration-Free Face-ssd: Single Shot Analysis of Smiles, Facial Attributes and Affect in The Wild. Computer Vision and Image Understanding, 182, 17–29.
18. [18] Menon, A. 2019. Towards Data Science. https://towardsdatascience.com/face-detection-in-2-minutes-using-opencv-python-90f89d7c0f81 (Erişim Tarihi: 22.04.2021)
19. [19] Szegedy. C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A. 2015. Going Deeper with Convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 7-12 June, Boston, MA, 1–9.
20. [20] Ioffe, S., Szegedy, C., 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. ICML, arXiv: 1502.03167v3.
21. [21] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z. 2016. Rethinking The Inception Architecture for Computer Vision. 2016 IEEE Conference on Computer Vision and Pattern Recognition, 27-30 June, Las Vegas, NV, USA, 2818–2826.
22. [22] He, K., Zhang, X., Ren, S., Sun, J. 2016. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition, 27-30 June, Las Vegas, NV, USA, 770–778.
23. [23] Yi, D., Lei, Z. Liao, S., Li, S. Z. 2014. Learning Face Representation From Scratch. arXiv:1411.7923.
24. [24] Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E. 2008. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments, Workshop on Faces in ’Real-Life’ Images: Detection, Alignment, and Recognition, Erik Learned-Miller and Andras Ferencz and Frédéric Jurie, October 2008, Marseille, France. ffinria-00321923.