INVESTIGATION OF THE RELATIONSHIP BETWEEN SEPTORHINOPLASTY AND FACIAL RECOGNITION SYSTEMS

Abstract

Objectives: To investigate the temporal verification performance of the facial recognition systems after septorhinoplasty. Material and Method: The study population included male and female patients who underwent septorhinoplasty at our institution between January 2022 and December 2023. Pre and postoperative photographs were taken at 1, 2, and 4 weeks using the same camera, under the same distance, and under the same lighting conditions. In this techniqueagnostic study, the analysis focused on the overall effect of the procedure rather than the impact of specific surgical manoeuvres. The change over time (preoperative, postoperative weeks 1, 2, 4) was compared based on the mean distance values in the face recognition systems. Results: The evaluation was conducted on 119 patients, comprising 75 females and 44 males with a mean age of 26.9±7.34 years (range, 1856 years). When the accuracy rates of the face recognition systems were evaluated, the highest performance rate was obtained with the Euclidean metric for the VGGFace system (94.85%). Among the face extraction methods, the RetinaFace (99.40%) and Mtcnn (99.19%) methods had the highest accuracy rates with the Euclid ean metric in the VGGFace face recognition system. There was a significant correlation between the mean distance value (0.378) in the preoperativepostoperative 2nd week evaluation (02) and the mean distance value (0.279) in the 2nd−4th week evaluation (r=0.747, p=0.004). Conclusions: The alteration of facial components and appearance following septorhinoplasty remains a challenge for postoperative biometric verification using current facial recognition technologies. Rhinologists should be aware of the relationship between septorhinoplasty and facial recognition systems.

Keywords

• facial recognition technology
• septorhinoplasty
• biometric identification
• rhinoplasty
• deep learning

SEPTORİNOPLASTİ VE YÜZ TANIMA SİSTEMLERİ ARASINDAKİ İLİŞKİNİN ARAŞTIRILMASI

Abstract

Amaç: Bu çalışmanın amacı septorinoplasti sonrası yüz tanıma sistemlerinin zamansal doğrulama performansını araştırmaktır. Gereç ve Yöntemler: Çalışma popülasyonu, Ocak 2022 ile Aralık 2023 arasında kurumumuzda septorinoplasti geçiren erkek ve kadın hastaları içermektedir. Ameliyat öncesi ve sonrası fotoğraflar, aynı kamera kullanılarak, aynı mesafe ve ışık koşullarında 1, 2 ve 4. haftalarda çekilmiştir. Ameliyat öncesi ve sonrası fotoğraflar, sep torinoplastinin genel etkisini teknikagnostik bir yaklaşımla değer lendirmek üzere sekiz farklı yüz tanıma sistemi kullanılarak analiz edilmiştir. Zaman içindeki değişim (ameliyat öncesi, ameliyat son rası 1, 2, 4. haftalar) yüz tanıma sistemlerindeki ortalama mesafe değerlerine göre karşılaştırılmıştır. Bulgular: Değerlendirme, ortalama yaşları 26,9±7,34 yıl (aralığı, 1856 yıl) olan 75 kadın ve 44 erkek olmak üzere toplam 119 hastadan yapılmıştır. Yüz tanıma sistemlerinin doğruluk oranları değerlendirildiğinde, en yüksek performans oranı VGGFace sis temi için Öklid metriği ile elde edildi (%94,85). Yüz çıkarma yöntemleri arasında, Retinaface (%99,40) ve Mtcnn (%99,19) yön temleri, VGGFace yüz tanıma sisteminde Öklid metriği en yüksek doğruluk oranlarına sahipti. Ameliyat öncesi sonrası 2. hafta değerlendirmesinde (02) ortalama mesafe değeri (0,378) ile 2.4. hafta değerlendirmesinde ortalama mesafe değeri (0,279) arasında anlamlı bir korelasyon vardı (r=0,747, p=0,004). Sonuç: Septorinoplasti sonrası yüz bileşenlerinin ve görünümünün değişmesi, mevcut yüz tanıma teknolojileriyle ameliyat sonrası biyometrik doğrulama için bir zorluk olmaya devam etmektedir. Rinologlar septorinoplasti ile yüz tanıma sis temleri arasındaki ilişkinin farkında olmalıdır.

Keywords

• Yüz tanıma teknolojisi
• septorinoplasti
• biyometrik tanımlama
• rinoplasti
• derin öğrenme

References

1. American Society for Aesthetic Plastic Surgery. Plastic surgery statistics report. 2023 Aug 30. https://www.plasticsurgery.org/documents/news/ statistics/2020/plastic-surgery-statistics-full-report-2020.pdf google scholar
2. Bouguila J, Khochtali H. Facial plastic surgery and face recognition algorithms: interaction and challenges. A scoping review and future directions. J Stomatol Oral Maxillofac Surg 2020;121(6):696-703. google scholar
3. Singh R, Vatsa M, Bhatt HS, Bharadwaj S, Noore A, Nooreyezdan SS. Plastic surgery: A new dimension to face recognition. IEEE T Inf Foren Sec 2010;5(3):441-8. google scholar
4. Zuo KJ, Saun TJ, Forrest CR. Facial recognition technology: a primer for plastic surgeons. Plast Reconstr Surg 2019;143(6):1298-306. google scholar
5. Bansal A, Shetty NP. Matching before and after surgery faces. Procedia Comput. Sci 2018;132:141-8. google scholar
6. Ricanek K, Tesafaye T. Morph: a longitudinal image database of normal adult age-progression. 7th international conference on automatic face and gesture recognition. 2018. Southampton, UK. google scholar
7. Singh R, Vatsa M, Noore A. Effect of plastic surgery on face recognition: A preliminary study. 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. 2009. Miami, FL, USA. google scholar
8. Khedgaonkar R, Raghuwanshi MM, Singh KR. Patch-based face recognition under plastic surgery. 2018 First International Conference on Secure Cyber Computing and Communication (ICSCCC). 2018. Jalandhar, India google scholar

9. Gupta N, Hughes SJ, Chirwa R, Cheng Q. Facial recognition software use on surgically altered faces: A systematic review. J Craniofac Surg 2022;33(8):2443-6. google scholar
10. Chude-Olisah CC, Sulong GB, Chude-Okonkwo UAK, Hashim SZM. Edge- based representation and recognition for surgically altered face images. 7th International Conference on Signal Processing and Communication Systems (ICSPCS). 2013. Carrara, Australia. google scholar
11. Deng J, Guo J, Yang N, Xue I, Kotsia I, Zafeiriou S. Arcface: additive angular margin loss for deep face recognition. IEEE Trans Pattern Anal Mach Intell 2022;44(10):5962-79. google scholar
12. Kim CH, Most SP. Photography and photodocumentation for the rhinoplasty patient. Clin Plast Surg 2022;49(1):13-22. google scholar
13. Henderson CDRJL, Larrabee Jr WF, Krieger HBD. Photographic standards for facial plastic surgery. Arch Facial Plast Surg 2005;7(5):331-3. google scholar
14. Larrosa F, Dura MJ, Roura J, Hernandez A. Rhinoplasty planning with an iPhone app: analysis of otolaryngologists response. Eur Arch Otorhinolaryngol 2013;270:2473-7. google scholar
15. Centers for Disease Control and Prevention. Health insurance portability and accountability act of 1996 (HIPAA). 2023 Aug 30 https://www.cdc.gov/phlp/php/ resources/health-insurance-portability-and-accountability-act-of-1996-hipaa. html google scholar
16. Bradski G. The openCV library. Dr Dobb’s journal: Software tools for the professional programmer. J Comput Commun 2000;25(11):120-3. google scholar
17. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, et al. Ssd: Single shot multibox detector. 14th European Conference Computer Vision–ECCV 2016. 2016. Amsterdam, Holland google scholar
18. Dalal N, Triggs B. Histograms of oriented gradients for human detection. 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05). IEEE 2005;1:886-93. google scholar
19. Xiang J, Zhu G. Joint face detection and facial expression recognition with MTCNN. 2017 4th international conference on information science and control engineering (ICISCE). 2017. Changsha, China. google scholar
20. Deng J, Guo J, Ververas E, Kotsia I, Zafeiriou S. Retinaface: Single-shot multi-level face localisation in the wild. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. Seattle, WA, USA. google scholar
21. Serengil SI, Ozpinar A. Hyperextended lightface: A facial attribute analysis framework. 2021 International Conference on Engineering and Emerging Technologies (ICEET). 2021. Istanbul, Turkey. google scholar
22. Huang GB, Mattar M, Berg T, Learned-Miller E. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Workshop on faces in 'Real-Life' Images: Detection, Alignment, and Recognition. 2008. Marseille, France. google scholar
23. Parkhi OM, Vedaldi A, Zisserman A. Deep face recognition. Proceedings of the British Machine Vision Conference. 2015. Swansea, UK. google scholar
24. Schroff F, Kalenichenko D, Philbin J. Facenet: A unified embedding for face recognition and clustering. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). 2015. Boston, MA, USA. google scholar
25. Huang GB, Mattar M, Berg T, Learned-Miller E. Labeled faces in the wild: Updates and new reporting procedures. Dept Comput Sci 2014;14(003):1-5. google scholar
26. Amos B, Ludwiczuk B, Satyanarayanan M. Openface: A general-purpose face recognition library with mobile applications. CMU School of Computer Science 2016;6(2):20. google scholar
27. Dlib Python API Tutorials. 2023 Aug 30. http://dlib.net/python/index.html. google scholar
28. Sun Y, Wang X, Tang X. Deep learning face representation from predicting 10,000 classes. Proceedings of the IEEE conference on computer vision and pattern recognition. 2014. Columbus, OH, USA. google scholar
29. Taigman Y, Yang M, Ranzato M, Wolf L. Deepface: Closing the gap to human-level performance in face verification. 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2014. Columbus, OH, USA. google scholar
30. Keras insightface. 2023 Aug 30. https://github.com/leondgarse/Keras_ insightface google scholar
31. Kaur P, Krishan K, Sharma SK, Kanchan T. Facial-recognition algorithms: A literature review. Med Sci Law 2020;60(2):131-9. google scholar
32. Sitepu SE, Jati G, Alhamidi MR, Caesarendra W, Jatmiko W. FaceNet with RetinaFace to Identify Masked Face. 6th International Workshop on Big Data and Information Security (IWBIS).2021. Depok, Indonesia. google scholar
33. Şirin Y. Efficient 2d and 3d Image Retrieval with Enhanced Skeleton Points. [Doctoral thesis]. Ankara: TOBB University; 2016. google scholar
34. Çelikhasi C, Ulu A, Sayar A. Comparison of DNA sequences using simulation. Veri Bilimi 2018;1(1):15-9. google scholar
35. Öztürk Y. Content based comparison of traditional methods and convolutional neural networks in medical images. Doctoral thesis. Sakarya: Sakarya University; 2019. google scholar
36. Mistry Y, Ingole DT, Ingole MD. Content based image retrieval using hybrid features and various distance metric. J Electr Syst Inf Technol 2018;5(3):874-88. google scholar
37. Khosla G, Rajpal N, Singh J. Evaluation of Euclidean and Manhanttan metrics in content based image retrieval system. 2nd International Conference on Computing for Sustainable Global Development (INDIACom). IEEE; 2015. New Delhi, India. google scholar
38. Sokolova M, Japkowicz N, Szpakowicz S. Beyond accuracy, F-score and ROC: a family of discriminant measures for performance evaluation. Australasian joint conference on artificial intelligence.2006. Berlin, Heidelberg. google scholar
39. Lee J, Jung S, Lee H, Seo J, Choi Y, Bae H, et al. Quantitative anatomical analysis of facial expression using a 3D motion capture system: Application to cosmetic surgery and facial recognition technology. Clin Anat 2015;28(6):735-44. google scholar
40. Ologunde R. Plastic surgery and the biometric e-passport: Implications for facial recognition. Plast Surg Hand Surg 2015;49(2):127. google scholar
41. Rathgeb C, Dogan D, Stockhardt F, De Marsico M, Busch C. Plastic surgery: An obstacle for deep face recognition? IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2020. Seattle, WA, USA. google scholar
42. Oloyede MO, Hancke GP, Myburgh HC. A review on face recognition systems: Recent approaches and challenges. Multimed Tools Appl 2020;79:27891-922. google scholar
43. Erdogmus N, Kose N, Dugelay J-L. Impact analysis of nose alterations on 2d and 3d face recognition. 2012 IEEE 14th International Workshop on Multimedia Signal Processing (MMSP). IEEE; 2012. Banff, AB, Canada. google scholar
44. Wolen J. Impact assessment of facial recognition algorithms’ performance when modifying nose dimensions. Thesis. Virginia: West Virginia University. 2015. google scholar
45. Pavri S, Zhu VZ, Steinbacher DM. Postoperative edema resolution following rhinoplasty: a three-dimensional morphometric assessment. Plast Reconstr Surg 2016;138(6):973-9. google scholar
46. Suri S, Sankaran A, Vatsa M, Singh R. On matching faces with alterations due to plastic surgery and disguise.IEEE 9th International Conference on Biometrics Theory, Applications and Systems. IEEE; 2018; Redondo Beach, CA, USA. google scholar
47. Ali ASO, Sagayan V, Malik A, Aziz A. Proposed face recognition system after plastic surgery. IET Comput Vis 2016;10(5):344-50. google scholar
48. Sabharwal T, Gupta R, Son LH, Kumar R, Jha S. Recognition of surgically altered face images: an empirical analysis on recent advances. Artif Intell Rev 2019;52:1009–40. google scholar
49. Deot N, Kiprovski A, Hatala A, Obayemi Jr A, Suryadevara A, Davila RO. Evaluation of mobile and digital single‐lens reflex photography for facial surgical analysis. Laryngoscope 2023;133(10):2590-6. google scholar
50. Derakhshan A, Gadkaree SK, Barbarite ER, Lindeborg MM, Bhama PK, Shaye DA. Quantifying facial distortion in modern digital photography. Laryngoscope 2023;134(3):1234-8. google scholar