Pupil Center Localization Based on Mini U-Net

Abstract

Many methods have been used from past to present to determine the location of the pupil center, which has an important place in eye tracking algorithms. These methods are usually shape-feature and appearance-based. Shape-feature-based methods use morphological image processing techniques, invariant geometric features of the eye, and infrared light to locate the iris and pupil. These methods are affected by real world conditions such as light, low resolution. In contrast, appearance-based methods are less sensitive to these conditions. In this study, Mini U-Net network, which is one of the appearance-based methods that automatically learns eye features and performs pupil center localization, is proposed. The proposed network was evaluated using the publicly available GI4E dataset for pupil center localization. In the test results of the network, measurements were made according to the maximum normalized error criterion. Accordingly, the center of the pupil was determined with an accuracy of 98.40%. The proposed network is compared with the latest technological methods and the performance of the proposed network is shown.

Keywords

• U-Net
• mini U-Net
• pupil center localization
• gi4e.

Pupil Center Localization Based on Mini U-Net

Abstract

Göz takip algoritmalarında önemli bir yere sahip olan göz bebeği merkezinin yerini belirlemek için geçmişten günümüze birçok yöntem kullanılmıştır. Bu yöntemler genellikle şekil-özellik ve görünüm temellidir. Şekil-özellik tabanlı yöntemler, iris ve göz bebeğinin yerini belirlemek için morfolojik görüntü işleme tekniklerini, gözün değişmez geometrik özelliklerini ve kızılötesi ışığı kullanır. Bu yöntemler ışık, düşük çözünürlük gibi gerçek dünya koşullarından etkilenir. Buna karşılık, görünüm temelli yöntemler bu koşullara daha az duyarlıdır. Bu çalışmada, göz özelliklerini otomatik olarak öğrenen ve göz bebeği merkezi lokalizasyonu gerçekleştiren görünüm tabanlı yöntemlerden biri olan Mini U-Net ağı önerilmiştir. Önerilen ağ, göz bebeği merkezi yerelleştirmesi için halka açık GI4E veri seti kullanılarak değerlendirildi. Ağın test sonuçlarında maksimum normalize edilmiş hata kriterine göre ölçümler yapılmıştır. Buna göre göz bebeğinin merkezi %98,40 doğrulukla belirlendi. Önerilen ağ, en son teknolojik yöntemlerle karşılaştırılmış ve önerilen ağın performansı ortaya konmuştur.

Keywords

• U-Net
• mini U-Net
• Göz bebeği merkezi yerelleştirmesi
• gi4e

References

1. Cai H, Liu B, Ju Z, Thill S, Belpaeme T, Vanderborght B, Liu H. (2018) Accurate Eye Center Localization via Hierarchical Adaptive Convolution. In Proceedings of the 29th British Machine Vision Conference, BMVC, pp.284.
2. Choi J. H, il Lee K, Kim Y. C, Cheol Song B. (2019) Accurate Eye Pupil Localization Using Heterogeneous CNN Models. Proceedings - International Conference on Image Processing, ICIP, pp.2179-2183.
3. Dlib C++ Library (2022). http://www.dlib.net. Accessed 25 July 2022
4. Gou C, Wu Y, Wang K, Wang F. Y, Ji Q. (2016) Learning-by-synthesis for accurate eye detection. Proceedings - International Conference on Pattern Recognition, pp.3362-3367.
5. Gou C, Wu Y, Wang K, Wang K, Wang F. Y, Ji Q (2017) A joint cascaded framework for simultaneous eye detection and eye state estimation. Pattern Recognition 67:23–31.
6. Jesorsky O, Kirchberg K. J, Frischholz R. W (2001) Robust face detection using the Hausdorff distance. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 2091:90-95.
7. Kim S, Jeong M, Ko B. C (2020) Energy Efficient Pupil Tracking Based on Rule Distillation of Cascade Regression Forest. Sensors 20(18):5141.
8. Kitazumi K, Nakazawa A. (2019) Robust Pupil Segmentation and Center Detection from Visible Light Images Using Convolutional Neural Network. Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC, pp.862–868.

9. Larumbe-Bergera A, Garde G, Porta S, Cabeza R, Villanueva A (2021) Accurate pupil center detection in off-the-shelf eye tracking systems using convolutional neural networks. Sensors 21(20).
10. Lee K. Il, Jeon J. H, Song B. C (2020) Deep Learning-Based Pupil Center Detection for Fast and Accurate Eye Tracking System. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12364 LNCS, 36-52.
11. Levinshtein A, Phung E, Aarabi P (2018) Hybrid eye center localization using cascaded regression and hand-crafted model fitting. Image and Vision Computing 71:17–24.
12. OpenCV (2022). https://www.opencv.org. Accessed 25 July 2022
13. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 9351:234-241.
14. Villanueva A, Ponz V, Sesma-Sanchez L, Ariz M, Porta S, Cabeza R (2013) Hybrid method based on topography for robust detection of iris center and eye corners. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 9(4).
15. Xia Y, Yu H, Wang F. Y (2019) Accurate and robust eye center localization via fully convolutional networks. IEEE/CAA Journal of Automatica Sinica 6(5):1127–1138.
16. Xiao F, Huang K, Qiu Y, Shen H (2018) Accurate iris center localization method using facial landmark, snakuscule, circle fitting and binary connected component. Multimedia Tools and Applications 77(19):25333-25353.
17. Zhang W, Smith M. L, Smith L. N, Farooq A (2016) Eye center localization and gaze gesture recognition for human-computer interaction. Journal of the Optical Society of America. A, Optics, Image Science, and Vision 33(3):314-325.