Boosted LightFace: A Hybrid DNN and GBM Model for Boosted Facial Recognition

Abstract

Facial recognition technology has seen significant advancements, impacting security, surveillance, and personal identification. Deep neural networks have enhanced accuracy and reliability, with integration into everyday devices further accelerating adoption. Researchers explore combining Deep Neural Networks with Gradient Boosting Machines for improved performance. This paper proposes Boosted LightFace, a hybrid Deep Neural Networks and Gradient Boosting Machines model leveraging robust facial recognition and face detection models. The architecture first integrates predictions from five high-performing DNN models. Their distance metrics and classification outcomes are engineered into a tabular dataset of 6,000 image pairs with 13 features. This dataset is then trained using a highly efficient LightGBM model with a low learning rate of 0.01 and 1000 estimators, incorporating an early stopping mechanism, and employing 10-fold cross-validation to maximize generalization. Recent research identifies FaceNet512d as a robust model, surpassing human recognition on the Labeled Faces in The Wild dataset with 98.4% score. Boosted LightFace achieves 99.1% accuracy, surpassing human recognition by 1.6% and outperforming the best single model in LightFace by 0.7%, underscoring the potential of integrating Deep Neural Networks and Gradient Boosting Machines models in advancing facial recognition technology. Furthermore, Boosted LightFace not only outperforms individual models in terms of accuracy but also surpasses them in precision, recall, F1, and AUC scores, highlighting its comprehensive superiority.

Keywords

• Face recognition
• Ensemble learning
• Deep learning
• Gradient boosting

References

1. [1] LeCun, Y., Bengio, Y., Hinton, G., “Deep learning”, Nature, 521(7553): 436–444, (2015). DOI: 10.1038/nature14539
2. [2] Krizhevsky, A., Sutskever, I., Hinton, G. E., "Imagenet classification with deep convolutional neural networks”, Advances in Neural Information Processing Systems, 25, (2012). DOI: 10.1145/3065386
3. [3] Friedman, J. H., “Greedy function approximation: a gradient boosting machine” Annals of Statistics, 1189–1232, (2001). DOI: 10.1214/aos/1013203451
4. [4] Kumari, P., Toshniwal, D., “Extreme gradient boosting and deep neural network based ensemble learning approach to forecast hourly solar irradiance”, Journal of Cleaner Production, 279, 123285, (2021). DOI: 10.1016/j.jclepro.2020.123285
5. [5] Mohammed, A., Kora, R., “A comprehensive review on ensemble deep learning: Opportunities and challenges”, Journal of King Saud University-Computer and Information Sciences, 35(2): 757–774, (2023). DOI: 10.1016/j.jksuci.2023.01.014
6. [6] Shrivastav, L. K., Kumar, R., “An ensemble of random forest gradient boosting machine and deep learning methods for stock price prediction”, Journal of Information Technology Research (JITR), 15(1): 1–19, (2022). DOI: 10.4018/JITR.2022010102
7. [7] Serengil, S. I., Ozpinar, A., “LightFace: A hybrid deep face recognition framework”, 2020 Innovations in Intelligent Systems and Applications Conference (ASYU), 23–27. IEEE, (2020). DOI: 10.1109/ASYU50717.2020.9259802
8. [8] Parkhi, O., Vedaldi, A., Zisserman, A., “Deep face recognition”, In BMVC 2015-Proceedings of the British Machine Vision Conference 2015. British Machine Vision Association, (2015). DOI: 10.5244/c.29.41

9. [9] Schroff, F., Kalenichenko, D., Philbin, J., “Facenet: A unified embedding for face recognition and clustering”, In Proceedings of the IEEE conference on computer vision and pattern recognition, 815–823, (2015). DOI: 10.1109/CVPR.2015.7298682
10. [10] Amos, B., Ludwiczuk, B., Satyanarayanan, M., “Openface: A general-purpose face recognition library with mobile applications”, CMU School of Computer Science, 6(2): 20, (2016).
11. [11] Taigman, Y., Yang, M., Ranzato, M., Wolf, L., “Deepface: Closing the gap to human-level performance in face verification”, In Proceedings of the IEEE conference on computer vision and pattern recognition, 1701–1708, (2014). DOI: 10.1109/CVPR.2014.220
12. [12] Sun, Y., Wang, X., Tang, X., “Deep learning face representation from predicting 10,000 classes”, In Proceedings of the IEEE conference on computer vision and pattern recognition, 1891–1898, (2014). DOI: 10.1109/CVPR.2014.244
13. [13] Deng, J., Guo, J., Xue, N., Zafeiriou, S., “Arcface: Additive angular margin loss for deep face recognition.”, In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 4690–4699, (2019). DOI: 10.1109/TPAMI.2021.3087709
14. [14] King, D. E., “Dlib-ml: A machine learning toolkit”, The Journal of Machine Learning Research, 10: 1755–1758, (2009). DOI: 10.5555/1577069.1755843
15. [15] Zhong, Y., Deng, W., Hu, J., Zhao, D., Li, X., Wen, D., “Sface: Sigmoid-constrained hypersphere loss for robust face recognition”, IEEE Transactions on Image Processing, 30: 2587–2598, (2021). DOI: 10.1109/TIP.2020.3048632
16. [16] Alansari, M., Abdul Hay, O., Javed, S., Shoufan, A., Zweiri, Y., Werghi, N., “Ghostfacenets: Lightweight face recognition model from cheap operations”, IEEE Access, 11: 35429–35446, (2023). DOI: 10.1109/ACCESS.2023.3266068
17. [17] Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., Liu, T. Y., “Lightgbm: A highly efficient gradient boosting decision tree”, Advances in Neural Information Processing Systems, 30, (2017). DOI: 10.5555/3294996.3295074
18. [18] Chen, T., Guestrin, C., “Xgboost: A scalable tree boosting system”, In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, 785–794, (2016). DOI: 10.1145/2939672.293978
19. [19] Huang, G. B., Mattar, M., Berg, T., Learned-Miller, E., “Labeled faces in the wild: A database for studying face recognition in unconstrained environments”, In Workshop on faces in ‘Real-Life’ Images: detection, alignment, and recognition, (2008). DOI: 10.1007/978-3-319-25958-1_8
20. [20] Deng, J., Guo, J., Ververas, E., Kotsia, I., Zafeiriou, S., “Retinaface: Single-shot multi-level face localisation in the wild”, In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 5203–5212, (2020). DOI: 10.1109/CVPR42600.2020.00525
21. [21] Serengil, S., Ozpinar, A., “A benchmark of facial recognition pipelines and co-usability performances of modules”, Bilisim Teknolojileri Dergisi, 17(2): 95–107 (2024). DOI: 10.17671/gazibtd.1399077
22. [22] Kumar, N., Berg, A. C., Belhumeur, P. N., Nayar, S. K., “Attribute and simile classifiers for face verification”, In 2009 IEEE 12th international conference on computer vision, 365–372. IEEE, (2009). DOI: 10.1109/ICCV.2009.5459250
23. [23] Wu, C., Zhang, Y., “MTCNN and FACENET based access control system for face detection and recognition”, Automatic Control and Computer Sciences, 55(1): 102-112, (2021). DOI: 10.3103/S0146411621010090
24. [24] Nemavhola A., Chibaya, C., Viriri, S., “A Systematic Review of CNN Architectures, Databases, Performance Metrics, and Applications in Face Recognition”, Information, 16(2): 107, (2025). DOI: 10.3390/info16020107
25. [25] Freund, Y., Schapire, R. E., “Experiments with a new boosting algorithm”, In ICML 96: 148–156. Citeseer, (1996). DOI: 10.5555/3091696.3091715