TY - JOUR T1 - Farklı derin sinir ağı modelleri birleştirilerek yakın kızılötesi görüntülerde yüz tespiti TT - Face Detection in near infrared images by ensembling different deep neural networks AU - Unluhısarcıklı, Ahmet AU - Toprak, Ahmet Nusret PY - 2024 DA - April Y2 - 2024 DO - 10.28948/ngumuh.1373458 JF - Niğde Ömer Halisdemir Üniversitesi Mühendislik Bilimleri Dergisi JO - NÖHÜ Müh. Bilim. Derg. PB - Niğde Ömer Halisdemir Üniversitesi WT - DergiPark SN - 2564-6605 SP - 473 EP - 489 VL - 13 IS - 2 LA - tr AB - Yüz tespiti, güvenlik, sağlık, endüstri, biyometri gibi alanlarda kritik bir rol oynamaktadır. Bahsi geçen alanlarda, aydınlatma durumundan bağımsız olarak yüz tespitinin doğru ve verimli bir şekilde gerçekleştirilebilmesi büyük öneme sahiptir. Ancak, yetersiz aydınlatma koşullarında verimli bir şekilde yüz tespiti yapmak zor olabilmektedir. Bu problem doğrultusunda bu çalışmada, yetersiz aydınlatma koşullarında dahi verimli çalışabilen, yakın kızılötesi görüntüler üzerinde yüz tespitine odaklanan bir yaklaşım sunulmaktadır. Bu amaçla öncelikle literatürde iyi bilinen sekiz yüz tespiti derin sinir ağı modeli belirlenmiş ve yakın kızılötesi görüntülerdeki başarımlarını ortaya koymak amacıyla görsel ve sayısal olarak karşılaştırılmıştır. Ayrıca ilgili yöntemlerin çalışma zamanlarının karşılaştırmasına yönelik deneyler CPU ve GPU ortamında gerçekleştirilmiştir. Çalışma kapsamında önerilen yaklaşım birden fazla yüz tespiti modelinin ürettiği sınırlayıcı kutu topluluklarının beraber analiz edilmesi ile daha başarılı ve daha kapsayıcı yeni sınırlayıcı kutular üretilebileceği ilkesine dayanmaktadır. Buradan hareketle, Kombinasyonel Oylama ve Maksimum Olmayan Ortalama adları verilen iki yeni sınırlayıcı kutu belirleme yöntemi önerilmektedir. Önerilen yöntemler literatürdeki sınırlayıcı kutu belirleme yöntemleri ile karşılaştırılmıştır. Kombinasyonel Oylama yönteminin, ürettiği %93,6 doğruluk değeriyle literatürdeki sınırlayıcı kutu belirleme yöntemlerinden daha başarılı sonuçlar ortaya koyduğu görülmüştür. KW - Yüz tespiti KW - yakın kızılötesi görüntüler KW - sınırlayıcı kutu topluluğu birleştirme N2 - Face detection plays a crucial role in various areas such as security, healthcare, industry, biometrics, etc. It is essential to perform face detection accurately and efficiently in the mentioned areas regardless of the lighting conditions. However, detecting faces in poor lighting conditions can be challenging. To address this issue, this study presents an approach focusing on face detection on near infrared images, which can work efficiently even in insufficient lighting conditions. For this purpose, first of all, eight state-of-the-art face detection models in the literature were determined and compared visually and numerically to reveal their performance in near infrared images. Additionally, experiments were carried out in CPU and GPU environments to compare the running times of the face detection methods. The approach proposed in this study is based on the principle that a more successful and inclusive bounding-box can be produced by using a bounding-box ensemble generated by more than one face detection model. Based on this, two new bounding-box ensemble methods called Combinational Voting and Non-Maximum Averaging are proposed. The proposed methods are compared with bounding-box ensemble methods in the literature. Combinational Voting produces more successful results than the other methods in the literature, with an accuracy rate of 93.6%. CR - Kumar, A., Kaur, A., & Kumar, M. Face detection techniques: a review. Artificial Intelligence Review, 52, 927-948. 2019. CR - Cho, S. W., Baek, N. R., Kim, M. C., Koo, J. H., Kim, J. H., & Park, K. R. Face detection in nighttime images using visible-light camera sensors with two-step faster region-based convolutional neural network. Sensors (Switzerland), 18(9). 2018. https://doi.org/10,3390/s18092995 CR - Li, J., Zhang, D., Zhang, K., Hu, K., & Yang, L. Real-time face detection during the night. 2017 4th International Conference on Systems and Informatics, ICSAI 2017, 2018-January. https://doi.org/10,1109/ICSAI.2017.8248358 CR - S Liao, A K Jain and S Z. Li, "A fast and accurate unconstrained face detector[J]", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 38, no. 2, pp. 211-223, 2016. CR - Wang, W., Wang, X., Yang, W., & Liu, J. Unsupervised face detection in the dark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1), 1250–1266. 2023. https://doi.org/10,1109/TPAMI.2022.3152562 CR - Li, J., Wang, Y., Wang, C., Tai, Y., Qian, J., Yang, J., Wang, C., Li, J., & Huang, F. DSFD: Dual shot face detector. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019-June, 5055–5064. https://doi.org/10,1109/CVPR.2019.00520 CR - Dash, P., Kisku, D. R., Gupta, P., & Sing, J. K. Fast face detection using a unified architecture for unconstrained and infrared face images. Cognitive Systems Research, 74, 18-38. 2022. CR - Gao, Z., Zhang, S., Fang, H., Li, L., & Huang, L. Multi-modal image fusion based improved face detection algorithm in poor lighting conditions. ACM International Conference Proceeding Series. 2021. https://doi.org/10,1145/3483207.3483213 CR - King, D. E. Dlib-ml: A machine learning toolkit. Journal of Machine Learning Research, 10, 2009. CR - Zhang, S., Zhu, X., Lei, Z., Shi, H., Wang, X., & Li, S. Z. FaceBoxes: A CPU real-time face detector with high accuracy. IEEE International Joint Conference on Biometrics, IJCB 2017, 2018-January,1–9.2018. https://doi.org/10,1109/BTAS.2017.8272675 CR - Bazarevsky, V., Kartynnik, Y., Vakunov, A., Raveendran, K., & Grundmann, M. BlazeFace: Sub-millisecond neural face detection on mobile GPUs. 2019. http://arxiv.org/abs/1907.05047 CR - Xu, Y., Yan, W., Yang, G., Luo, J., Li, T., & He, J. CenterFace: Joint face detection and alignment using face as point. Scientific Programming, 2020, 1–8. https://doi.org/10,1155/2020/7845384 CR - Deng, J., Guo, J., Ververas, E., Kotsia, I., & Zafeiriou, S. (2020). RetinaFace: Single-shot multi-level face localisation in the wild. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 5202–5211. 2020, https://doi.org/10,1109/CVPR42600,2020,00525 CR - Guo, J., Deng, J., Lattas, A., & Zafeiriou, S. Sample and computation redistribution for efficient face detection. ICLR 2022 - 10th International Conference on Learning Representations.2022. CR - Wu, W., Peng, H., & Yu, S. YuNet: A tiny millisecond-level face detector. Machine Intelligence Research. 2023. https://doi.org/10,1007/s11633-023-1423-y CR - Neubeck, A., & van Gool, L. Efficient non-maximum suppression. Proceedings - International Conference on Pattern Recognition, 3. 2006. https://doi.org/10,1109/ICPR.2006.479 CR - Solovyev, R., Wang, W., & Gabruseva, T. Weighted boxes fusion: Ensembling boxes from different object detection models. 2019. https://doi.org/10,1016/j.imavis.2021.104117 CR - C. Ning, H. Zhou, Y. Song, and J. Tang. Inception single shot multibox detector for object detection. In ICME, 2017. CR - Panetta, Karen, Qianwen Wan, Sos Agaian, Srijith Rajeev, Shreyas Kamath, Rahul Rajendran, Shishir Rao et al. "A comprehensive database for benchmarking imaging systems." IEEE Transactions on Pattern Analysis and Machine Intelligence. 2018. CR - TUFTS Face DB, TUFTS yüz görüntüsü veri seti depolama alanı, http://tdface.ece.tufts.edu/, son erişim 23 Kasım 2023. CR - Zou, Z., Chen, K., Shi, Z., Guo, Y., & Ye, J. Object detection in 20 years: A survey. Proceedings of the IEEE. 2023. CR - Mamieva, D., Abdusalomov, A. B., Mukhiddinov, M., & Whangbo, T. K. Improved face detection method via learning small faces on hard images based on a deep learning approach. Sensors, 23(1), 502. 2023. CR - Adouani, A., Henia, W. M. ben, & Lachiri, Z. A comparison of face detection methods using spontaneous videos. Multimedia Tools and Applications, 81(16), 23163–23191. 2022. https://doi.org/10,1007/s11042-022-12781-8 CR - LabelImg, Tzutalin LabelImg Git code (2015). https://github.com/tzutalin/labelImg, son erişim 24 Kasım 2023. CR - R. Vaillant, C. Monrocq, and Y. Le Cun. Original approach for the localisation of objects in images. IEE Proceedings Vision, Image and Signal Processing, 1994. CR - Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., & Berg, A. C. SSD: Single shot multibox detector. Lecture Notes in Computer Science (Including subseries lecture notes in artificial ıntelligence and lecture notes in bioinformatics), 9905 LNCS. 2016. https://doi.org/10,1007/978-3-319-46448-0_2 CR - Girshick, R. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision (pp. 1440-1448). 2015. CR - Ren, S., He, K., Girshick, R., & Sun, J. Faster R-CNN: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems, 2015-January. CR - Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L. C. MobileNetV2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2018. https://doi.org/10,1109/CVPR.2018.00474 CR - Lin, T. Y., Dollár, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. Feature pyramid networks for object detection. Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, 2017-January. https://doi.org/10,1109/CVPR.2017.106 CR - He, K., Zhang, X., Ren, S., & Sun, J. Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-December. https://doi.org/10,1109/CVPR.2016.90 CR - Radosavovic, I., Kosaraju, R. P., Girshick, R., He, K., & Dollár, P. Designing network design spaces. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.2020, https://doi.org/10,1109/CVPR42600,2020,01044 CR - Liu, S., Qi, L., Qin, H., Shi, J., & Jia, J. Path Aggregation Network for Instance Segmentation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.2018. https://doi.org/10,1109/CVPR.2018.00913 CR - Zhou, H., Li, Z., Ning, C., & Tang, J. CAD: Scale ınvariant framework for real-time object detection. proceedings - 2017 IEEE International Conference on Computer Vision Workshops, ICCVW 2017, 2018-January. https://doi.org/10,1109/ICCVW.2017.95 UR - https://doi.org/10.28948/ngumuh.1373458 L1 - https://dergipark.org.tr/tr/download/article-file/3463846 ER -