H-DrowsyNet: Sürücü Yorgunluk Tespiti için Göz Açıklık Oranı (EAR) ve Evrişimli Sinir Ağlarını (CNN) Esas Alan Çift Kollu Hibrit Bir Mimari

Abstract

Sürücü yorgunluğu, dünya genelinde trafik kazalarının başlıca nedenlerinden biri olup, çok sayıda yaralanma ve can kaybına yol açmaktadır. Bunu azaltmak için, tespit yöntemleri Geometrik Teknikler (örneğin Göz Oran Endeksi- EAR) veya Derin Öğrenme Yaklaşımları (örneğin Evrişimli Sinir Ağları - CNN’ler) gibi yöntemler kullanır ve her birinin kendine özgü sınırlamaları vardır. EAR, hesaplama açısından verimlilik sunarken değişen aydınlatma, gözlük ve baş pozisyonlarında yetersiz kalabilir; CNN’ler ise yüksek doğruluk sağlar ancak önemli miktarda kaynak gerektirir ve gölgeler ya da engellemeler tarafından yanıltılabilir. Bu çalışma, bu zayıf yönleri gidermek amacıyla, EAR analizini bir CNN sınıflandırıcıyla birleştirip çıktıları bir karar füzyon modülüyle birleştiren, çift dallı hibrit bir mimari olan H-DrowsyNet’i tanıtmaktadır. Sistem, 9.120 yüz görüntüsünden oluşan ve Uyanık ile Uykulu sınıflarına ayrılmış Kaggle Sürücü Uykululuk Veri seti (DDD) kullanılarak değerlendirilmiştir. Sonuçlar, H-DrowsyNet’in tek başına kullanılan EAR ya da CNN modellerini geride bırakarak, daha yüksek doğruluk ve daha düşük yanlış negatif (FN) oranı elde ettiğini; böylece yolda uykulu sürücü kaçırma ihtimalini azaltarak trafik güvenliğini artırdığını göstermektedir. Her iki analizden de uykulu sınıflandırması için karar füzyon modülü, hata oranını düşürmektedir. Bu hibrit yaklaşım, gerçek zamanlı sürücü izleme sistemleri için uygundur. Gelecekteki çalışmalar, H-DrowsyNet’in genellenebilirliğini artırmak amacıyla, farklı gerçek dünya koşullarında testler yapacaktır. H-DrowsyNet’in başarısı, geometrik ve derin öğrenme yöntemlerini birleştirerek trafik güvenliğini ileriye taşımaktadır.

Keywords

• Sürücü uykululuk tespiti
• Göz Açıklık Oranı
• Karar Birleştirme Modülü
• CNN sınıflandırıcı
• Uykululuk sınıflandırması

H-DrowsyNet: A Dual-Branch Hybrid Architecture Based on Eye Aspect Ratio (EAR) and Convolutional Neural Networks (CNN) for Driver Fatigue Detection

Abstract

Driver fatigue is a leading cause of traffic accidents worldwide, resulting in numerous injuries and fatalities. To mitigate this, detection methods use geometric techniques, such as the Eye Aspect Ratio (EAR), or deep learning approaches, such as Convolutional Neural Networks (CNNs), each with limitations. EAR offers computational efficiency but struggles with varying lighting, eyewear, and head poses, whereas CNNs provide high accuracy but require significant resources and can be misled by shadows or occlusions. This study introduces H-DrowsyNet, a dual-branch hybrid architecture that addresses these weaknesses by combining EAR analysis with a CNN classifier and fusing their outputs using a decision fusion module. The system was evaluated using the Kaggle Driver Drowsiness Dataset (DDD), which contains 9,120 facial images divided into Awake and Drowsy classes. The results show that H-DrowsyNet outperforms standalone EAR and CNN models, achieving higher accuracy and a reduced false-negative (FN) rate, thereby enhancing road safety by minimizing missed drowsy drivers. The decision fusion module, which requires agreement from both analyses for drowsiness classification, reduces errors. This hybrid approach is suitable for real-time driver-monitoring systems. Future work will test H-DrowsyNet under diverse real-world conditions to improve its generalization. The success of H-DrowsyNet stems from the combination of geometric and deep learning methods, thereby advancing traffic safety.

Keywords

• Driver drowsiness detection
• Eye Aspect Ratio
• Decision Fusion Module
• CNN classifier
• drowsiness classification.

References

1. Ahmed, M. I. B., Alabdulkarem, H., Alomair, F., Aldossary, D., Alahmari, M., Alhumaidan, M., Alrassan, S., Rahman, A., Youldash, M., & Zaman, G. (2023). A deep-learning approach to driver drowsiness detection. Safety, 9(3), Article 65. https://doi.org/10.3390/safety9030065
2. Ahmed, M., Masood, S., Ahmad, M., & Abd El-Latif, A. A. (2022). Intelligent driver drowsiness detection for traffic safety based on multi-CNN deep model and facial subsampling. IEEE Transactions on Intelligent Transportation Systems, 23(10), 19743–19752. https://doi.org/10.1109/TITS.2021.3134222
3. Ahmed, N., Romiz, S. N., Amin, R., Shan, M. S., & Mahmud, M. (2024). Drivers’ real-time drowsiness and attention detection and alarm system using eye aspect ratio (EAR) analysis. In Proceedings of the 2024 3rd International Conference on Advancement in Electrical and Electronic Engineering (ICAEEE) (pp. 1–4). IEEE. https://doi.org/10.1109/ICAEEE62219.2024.10561659
4. Ayachi, R., Afif, M., Said, Y., & Ben Abdelali, A. (2021). Driver fatigue detection using EfficientDet in advanced driver assistance systems. Proceedings of the 18th International Multi-Conference on Systems, Signals & Devices (SSD) (pp. 738–742). IEEE. https://doi.org/10.1109/SSD52085.2021.9429294
5. Chadha, R. S., Jugesh, & Singh, J. (2024). Enhancing road safety: A driver fatigue detection and behavior monitoring system using advanced computer vision techniques. Journal of Ubiquitous Computing and Communication Technologies, 6(2), 122–134. https://doi.org/10.36548/jucct.2024.2.004
6. Chirra, V., Reddy Uyyala, S., & Kishore Kolli, V. (2019). Deep CNN: A machine learning approach for driver drowsiness detection based on eye state. Revue d’Intelligence Artificielle, 33(6), 461–466. https://doi.org/10.18280/ria.330609
7. Choi, J. W., Koo, D. L., Kim, D. H., Nam, H., Lee, J. H., Hong, S.-N., & Kim, B. (2024). A novel deep learning model for obstructive sleep apnea diagnosis: Hybrid CNN–Transformer approach for radar-based detection of apnea–hypopnea events. SLEEP, 47(12), zsae184. https://doi.org/10.1093/sleep/zsae184
8. Das, S., Pratihar, S., Pradhan, B., Jhaveri, R. H., & Benedetto, F. (2024). IoT-assisted automatic driver drowsiness detection through facial movement analysis using deep learning and a U-Net-based architecture. Information, 15(1), Article 30. https://doi.org/10.3390/info15010030

9. Dewi, C., Chen, R.-C., Chang, C.-W., Wu, S.-H., Jiang, X., & Yu, H. (2022). Eye aspect ratio for real-time drowsiness detection to improve driver safety. Electronics, 11(19), Article 3183. https://doi.org/10.3390/electronics11193183
10. Florez, R., Palomino-Quispe, F., Coaquira-Castillo, R. J., Herrera-Levano, J. C., Paixão, T., & Alvarez, A. B. (2023). A CNN-based approach for driver drowsiness detection by real-time eye state identification. Applied Sciences, 13(13), Article 7849. https://doi.org/10.3390/app13137849
11. Kamti, M. K., & Iqbal, R. (2022). Evolution of driver fatigue detection techniques—A review from 2007 to 2021. Transportation Research Record: Journal of the Transportation Research Board, 2676(12), 485–507. https://doi.org/10.1177/03611981221096118
12. Kazemi, V., & Sullivan, J. (2014). One millisecond face alignment with an ensemble of regression trees. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1867–1874).
13. King, D. E. (2009). Dlib-ml: A machine learning toolkit. The Journal of Machine Learning Research, 10, 1755-1758.
14. Liu, F., Li, X., Lv, T., & Xu, F. (2019). A review of driver fatigue detection: Progress and prospect. In Proceedings of the IEEE International Conference on Consumer Electronics (ICCE) (pp. 1–6). IEEE. https://doi.org/10.1109/ICCE.2019.8662098
15. Peivandi, M., Ardabili, S. Z., Sheykhivand, S., & Danishvar, S. (2023). Deep learning for detecting multi-level driver fatigue using physiological signals: A comprehensive approach. Sensors, 23(19), Article 8171. https://doi.org/10.3390/s23198171
16. Rajasekaran, R., M, N., Solanki, R., Sanghavi, V., & S, Y. (2024). Enhancing driver safety: Real-time drowsiness detection through eye aspect ratio and CNN-based eye state analysis. In Proceedings of the 2024 International Conference on Intelligent and Innovative Technologies in Computing, Electrical and Electronics (IITCEE) (pp. 1–7). IEEE. https://doi.org/10.1109/IITCEE59897.2024.10467769
17. Rakibul, E. R. (2020). Drowsiness prediction dataset. Kaggle. https://www.kaggle.com/datasets/rakibuleceruet/drowsiness-prediction-dataset
18. Reddy, B., Kim, Y. H., Yun, S., Seo, C., & Jang, J. (2017). Real-time driver drowsiness detection for embedded systems using model compression of deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 121–128).
19. Saleem, S. (2022). Risk assessment of road traffic accidents related to sleepiness during driving: A systematic review. Eastern Mediterranean Health Journal, 28(9), 695–700. https://doi.org/10.26719/emhj.22.055
20. Sedik, A., Marey, M., & Mostafa, H. (2023). An adaptive fatigue detection system based on 3D CNNs and ensemble models. Symmetry, 15(6), Article 1274. https://doi.org/10.3390/sym15061274
21. Shin, H.-C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., & Summers, R. M. (2016). Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics, and transfer learning. IEEE Transactions on Medical Imaging, 35(5), 1285–1298. https://doi.org/10.1109/TMI.2016.2528162
22. Sikander, G., & Anwar, S. (2019). Driver fatigue detection systems: A review. IEEE Transactions on Intelligent Transportation Systems, 20(6), 2339–2352. https://doi.org/10.1109/TITS.2018.2868499
23. Soukupova, T., & Cech, J. (2016). Eye blink detection using facial landmarks. In Proceedings of the 21st Computer Vision Winter Workshop (Vol. 2, p. 4).
24. Vicente, J., Laguna, P., Bailón, R., & Bartra, A. (2016). Drowsiness detection using heart rate variability. Medical & Biological Engineering & Computing, 54(6), 927–937. https://doi.org/10.1007/s11517-015-1448-7
25. Wang, Q., & Mu, Z. (2021). Heterogeneous signal fusion method in driving fatigue detection signals. Journal of Advanced Transportation, 2021, Article 4464890. https://doi.org/10.1155/2021/4464890
26. Zhao, L., Niu, X., Wang, L., Niu, J., Zhu, X., & Dai, Z. (2023). Stress detection via multimodal multitemporal-scale fusion: A hybrid of deep learning and handcrafted feature approach. IEEE Sensors Journal, 23(22), 27817–27827. https://doi.org/10.1109/JSEN.2023.3314718