Real-Time Scalable System For Face Tracking In Multi-Camera

Öz

Face detection and tracking have become increasingly popular in recent years. It has critical importance in security, defense, and robotics applications uses encountered in everyday life. For this purpose, many decision support or expert systems have been developed using artificial intelligence and machine learning. Thanks to the developments in the field of deep learning and hardware many effective and reliable face tracking systems have been realized. However there are still very few real-time scalable end-to-end systems. Also, the realization of this system on multiple cameras is a real challenge. In this study, a real-time, multi-camera, deep learning-based face tracking system has been developed. In the realized system, SCRFD model is used for face detection, ArcFace model is used for face recognition, and an updated DeepSORT algorithm is used for more stable face tracking. In addition, Apache Kafka stream processing system and Socket.IO bidirectional communication library were used to process multi-camera data in real-time and scalable. In the proposed system, when an image is input into the system, it can be displayed on the web page after approximately 127 ms

Anahtar Kelimeler

• Face Recognition
• Face Tracking
• Deep Learning
• Multi-Camera

Proje Numarası

YL-2021-2449

Çoklu Kamerada Gerçek Zamanlı Ölçeklenebilir Yüz Takibi Sistemi

Öz

Yüz algılama ve izleme son yıllarda giderek daha popüler hale geldi. Günlük hayatta karşılaşılan güvenlik, savunma ve robotik uygulama kullanımlarında kritik öneme sahiptir. Bu amaçla yapay zeka ve makine öğrenmesi kullanılarak birçok karar destek veya uzman sistem geliştirilmiştir. Derin öğrenme ve donanım alanındaki gelişmeler sayesinde birçok etkin ve güvenilir yüz takip sistemi gerçekleştirilmiştir. Ancak hala çok az gerçek zamanlı ölçeklenebilir uçtan uca sistem var. Ayrıca, böyle bir sistemin birden çok kamerada gerçekleştirilmesi gerçek bir zorluktur. Bu çalışmada gerçek zamanlı, çok kameralı, derin öğrenme tabanlı yüz takip sistemi geliştirilmiştir. Gerçekleştirilen sistemde yüz tespiti için SCRFD modeli, yüz tanıma için ArcFace modeli ve daha kararlı yüz takibi için güncellenmiş DeepSORT algoritması kullanılmıştır. Ayrıca, çoklu kamera verilerini gerçek zamanlı ve ölçeklenebilir olarak işlemek için Apache Kafka akış işleme sistemi ve Socket.IO çift yönlü iletişim kütüphanesi kullanılmıştır. Önerilen sistemde girdi olarak görüntü verildiğinde yaklaşık 127 ms sonra web sayfasında görüntülenebilmektedir.

Anahtar Kelimeler

• Yüz Tanıma
• Yüz Takibi
• Derin Öğrenme
• Çoklu Kamera

Destekleyen Kurum

İnönü Üniversitesi Bilimsel Araştırmalar Proje Birimi

Proje Numarası

YL-2021-2449

Kaynakça

1. [1] Deng J., Guo J., Ververas E., Kotsia I., Zafeiriou S., "Retinaface: Single-shot multi-level face localisation in the wild", Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition , 5202-5211, (2020).
2. [2] Hanbay K., Alpaslan N., Talu M., Hanbay D., Karci A., Kocamaz A., "Continuous rotation invariant features for gradient-based texture classification", Computer Vision and Image Understanding , 132: 87-101, (2015).
3. [3] Liu Y., Tang X., Han J., Liu J., Rui D., Wu X., "HAMBox: Delving Into Mining High-Quality Anchors on Face Detection", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2020).
4. [4] Li J., Wang Y., Wang C., Tai Y., Qian J., Yang J., Wang C., Li J., Huang F., "DSFD: Dual Shot Face Detector", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2019).
5. [5] Üzen H., Hanbay K., "Yaya Özellik Tanıma için LM Filtre Temelli Derin Evrişimsel Sinir Ağı", Politeknik Dergisi , 23: 605–613, (2020).
6. [6] AKYEL C., ARICI N., "U-Net-RCB7: Image Segmentation Algorithm", Politeknik Dergisi , 26: 1555–1562, (2023).
7. [7] KARADAĞ B., ARI A., "Akıllı Mobil Cihazlarda YOLOv7 Modeli ile Nesne Tespiti", Politeknik Dergisi , 26: 1207–1214, (2023).
8. [8] Apache ., "Apache Kafka" , https://kafka.apache.org

9. [9] Zhang K., Zhang Z., Li Z., Qiao Y., "Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks", IEEE Signal Processing Letters , 23: 1499-1503, (2016).
10. [10] Zhu Y., Cai H., Zhang S., Wang C., Xiong Y., "Tinaface: Strong but simple baseline for face detection", arXiv preprint arXiv:2011.13183 , (2020).
11. [11] Guo J., Deng J., Lattas A., Zafeiriou S., "Sample and Computation Redistribution for Efficient Face Detection", (2021).
12. [12] Viola P., Jones M., "Rapid object detection using a boosted cascade of simple features", Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, (2001).
13. [13] Mita T., Kaneko T., Hori O., "Joint haar-like features for face detection", Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, 1619-1626, (2005).
14. [14] Zhang L., Chu R., Xiang S., Liao S., Li S., "Face Detection Based on Multi-Block LBP Representation", Advances in Biometrics, Berlin, Heidelberg, 11-18, (2007).
15. [15] He K., Zhang X., Ren S., Sun J., "Deep residual learning for image recognition", Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition , 2016-Decem: 770-778, (2016).
16. [16] Yang S., Luo P., Loy C., Tang X., "WIDER FACE: A Face Detection Benchmark", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 5525-5533, (2016).
17. [17] Turk M., Pentland A., "Eigenfaces for recognition", Journal of cognitive neuroscience , 3: 71-86, (1991).
18. [18] Liu W., Wen Y., Yu Z., Li M., Raj B., Song L., "SphereFace: Deep Hypersphere Embedding for Face Recognition", 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 6738-6746, (2017).
19. [19] Wang H., Wang Y., Zhou Z., Ji X., Gong D., Zhou J., Li Z., Liu W., "CosFace: Large Margin Cosine Loss for Deep Face Recognition", 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5265-5274, (2018).
20. [20] Deng J., Guo J., Xue N., Zafeiriou S., "ArcFace: Additive angular margin loss for deep face recognition", Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition , 2019-June: 4685-4694, (2019).
21. [21] Boutros F., Damer N., Kirchbuchner F., Kuijper A., "ElasticFace: Elastic Margin Loss for Deep Face Recognition", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 1578-1587, (2022).
22. [22] Huang G., Mattar M., Berg T., Learned-Miller E., "Labeled faces in the wild: A database forstudying face recognition in unconstrained environments", Workshop on faces in'Real-Life'Images: detection, alignment, and recognition, (2008).
23. [23] Wolf L., Hassner T., Maoz I., "Face recognition in unconstrained videos with matched background similarity", CVPR 2011, 529-534, (2011).
24. [24] Kemelmacher-Shlizerman I., Seitz S., Miller D., Brossard E., "The MegaFace Benchmark: 1 Million Faces for Recognition at Scale", 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4873-4882, (2016).
25. [25] Bewley A., Ge Z., Ott L., Ramos F., Upcroft B., "Simple online and realtime tracking", Proceedings - International Conference on Image Processing, ICIP , 2016-Augus: 3464-3468, (2016).
26. [26] Wojke N., Bewley A., Paulus D., "Simple online and realtime tracking with a deep association metric", Proceedings - International Conference on Image Processing, ICIP , 2017-Septe: 3645-3649, (2018).
27. [27] Zhang Y., Sun P., Jiang Y., Yu D., Weng F., Yuan Z., Luo P., Liu W., Wang X., "ByteTrack: Multi-Object Tracking by Associating Every Detection Box", , (2022).
28. [28] Cao J., Pang J., Weng X., Khirodkar R., Kitani K., "Observation-centric sort: Rethinking sort for robust multi-object tracking", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9686-9696, (2023).
29. [29] Rambach J., Huber M., Balthasar M., Zoubir A., "Collaborative multi-camera face recognition and tracking", 2015 12th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 1-6, (2015).
30. [30] Lian Z., Shao S., Huang C., "A Real Time Face Tracking System based on Multiple Information Fusion", Multimedia Tools and Applications , 79: 16751-16769, (2020).
31. [31] Welch G., Bishop G., Others ., "An introduction to the Kalman filter", (1995).
32. [32] Badave H., Kuber M., "Head Pose Estimation Based Robust Multicamera Face Recognition", 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS), 492-495, (2021).
33. [33] Schroff F., Kalenichenko D., Philbin J., "FaceNet: A Unified Embedding for Face Recognition and Clustering", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2015).
34. [34] Deng J., Zhou Y., Zafeiriou S., "Marginal loss for deep face recognition", Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 60--68, (2017).
35. [35] Liu W., Lin R., Liu Z., Liu L., Yu Z., Dai B., Song L., "Learning towards minimum hyperspherical energy", Advances in neural information processing systems , 31:, (2018).
36. [36] Rauch G., "Socket.IO" , https://socket.io
37. [37] Stonebraker M., Rowe L., "The Design of POSTGRES", ACM SIGMOD Record , 15:, (1986).