Research Article
PDF EndNote BibTex RIS Cite

ESA tabanlı göz durumu tespitinde görüntü önişlem yöntemlerinin etkisi

Year 2022, Volume 11, Issue 3, 496 - 505, 18.07.2022
https://doi.org/10.28948/ngumuh.1086414

Abstract

Bu çalışma göz kırpma, göz yorgunluğu, sürücü uyuşukluğu gibi temel problemlerin çözümlenmesinde önemli olan göz durumu tespitine odaklanmaktadır. Bu çalışmada, göz durumu tespiti için görüntü önişlem yöntemleri ve derin öğrenme tabanlı evrişimsel sinir ağına (ESA) dayanan bir yöntem önerilmiş ve önerilen yöntem ZJU veri seti üzerinde performansı test edilmiştir. Ayrıca, önerilen ESA modelinde farklı havuzlama katmanları değerlendirilmiş ve ZJU veriseti üzerinde elde edilen bulgularda ortalama havuzlama kullanılan önerilen ESA modelinin en iyi performansı elde ettiği görülmüştür. Sonrasında, ZJU veri setine görüntü ön işlem yöntemleri uygulanmış ve işlenmiş ZJU veri seti, önerilen ESA modelinde eğitilerek performansları karşılaştırılmıştır. Elde edilen sonuçlara göre histogram eşitleme yöntemi kullanılarak eğitimi gerçekleştirilen ESA modelinin ZJU veri setinde %94.32 doğruluk, %94.95 duyarlılık, %92.42 özgüllük, %97.41 kesinlik ve %96.16 F1 skor performans metrikleri ile üstün bir başarı elde ettiği görülmüştür. Bu çalışmada elde edilen sonuçlar, ZJU veri setinde yapılan önceki çalışmalarda sunulan performans metrikleri ile karşılaştırılmıştır. Önerilen yöntemin literatür ile karşılaştırıldığında, göz durumu tespitinde güçlü sınıflandırma performansına sahip olduğu tespit edilmiştir.

References

  • M. H. Yang, D. J. Kriegman and N. Ahuja, Detecting faces in images: A survey. IEEE Transactions on pattern analysis and machine intelligence, 24(1), 34-58, 2002. https://doi.org/10.1109/34.982883
  • T. Soukupova and J. Cech, Eye blink detection using facial landmarks. In 21st computer vision winter workshop, Rimske Toplice, Slovenia, 3-5 February 2006.
  • R. Huang, Y. Wang and L. Guo, P-FDCN based eye state analysis for fatigue detection. In 2018 IEEE 18th International Conference on Communication Technology (ICCT), pp. 1174-1178, Chongqing, China, 8-11 October 2018. https://doi.org/10.1109/ICCT.2018.8599947
  • D. M. Joshi, N.K. Rana and V. Misra, Classification of brain cancer using artificial neural network. In 2010 2nd international conference on electronic computer technology, pp. 112-116, 19-21 March 2010. https://doi.org/10.1109/ICECTECH.2010.5479975M
  • W. Zhao, R. Chellappa, P.J. Phillips and A. Rosenfeld, Face recognition: A literature survey. ACM computing surveys (CSUR), 35(4), 399-458, 2003. https://doi.org/10.1145/954339.954342
  • M. Divjak and H. Bischof, Eye Blink Based Fatigue Detection for Prevention of Computer Vision Syndrome. In MVA, pp. 350-353, 2009.
  • C. L. Chou, Y. H. Huang and S. C. Ho, Blink Detection Using Facial Landmark Blink Detector and Multi-Layer Perceptron. In NCS 2019, pp. 542-545, 2019.
  • M. Lalonde, D. Byrns, L Gagnon, N. Teasdale and D. Laurendeau, Real-time eye blink detection with GPU-based SIFT tracking. In Fourth Canadian Conference on Computer and Robot Vision (CRV'07), Montreal, Quebec, Canada, pp. 481-487, Montreal, Quebec, Canada, 28-30 May 2007. https://doi.org/ 10.1109/CRV.2007.54
  • J. He, W. Choi, Y. Yang, J. Lu, X. Wu and K. Peng, Detection of driver drowsiness using wearable devices: A feasibility study of the proximity sensor. Applied ergonomics, 65, 473-480, 2017. https://doi.org/10.1016/j.apergo.2017.02.016
  • M. Su, C. Yeh, S. Lin, P. Wang and S. Hou, An implementation of an eye-blink-based communication aid for people with severe disabilities. In 2008 International Conference on Audio, Language and Image Processing, pp. 351-356, Shanghai, China, 7-9 July, 2008. https://doi.org/ 10.1109/ICALIP.2008.4590090
  • S. Soltani and A. Mahnam, A practical efficient human computer interface based on saccadic eye movements for people with disabilities. Computers in biology and medicine, 70, 163-173, 2016. https://doi.org/10.1016/j.compbiomed.2016.01.012
  • Y. Dong, Y. Zhang, J. Yue and Z. Hu, Comparison of random forest, random ferns and support vector machine for eye state classification. Multimedia Tools and Applications, 75(19), 11763-11783, 2016. https://doi.org/10.1007/s11042-015-2635-0
  • L. Zhao, Z. Wang, G. Zhang, Y. Qi and X. Wang, Eye state recognition based on deep integrated neural network and transfer learning. Multimedia Tools and Applications, 77(15), 19415-19438, 2018. https://doi.org/10.1007/s11042-017-5380-8
  • Y. LeCun, Y. Bengio and G. Hinton, Deep learning. nature, 521(7553), 436-444, 2015. https://doi.org/10.1038/nature14539
  • L. Pauly and D. Sankar, Non intrusive eye blink detection from low resolution images using HOG-SVM classifier. International Journal of Image, Graphics and Signal Processing, 8(10), 11, 2016. https://doi.org/ 10.5815/ijigsp.2016.10.02
  • L. Pauly and D. Sankar, A novel method for eye tracking and blink detection in video frames. In 2015 IEEE International Conference on Computer Graphics, Vision and Information Security (CGVIS), pp. 252-257, Bhubaneswar, India, 2-3 November 2015. https://doi.org/ 10.1109/CGVIS.2015.7449931
  • Y. J. Han, W. Kim and J. S. Park, Efficient eye-blinking detection on smartphones: A hybrid approach based on deep learning. Mobile Information Systems, 2018, 2018. https://doi.org/10.1155/2018/6929762
  • W. O. Lee, E. C. Lee and K. R. Park, Blink detection robust to various facial poses. Journal of neuroscience methods, 193(2), 356-372, 2010. https://doi.org/10.1016/j.jneumeth.2010.08.034
  • Y. S. Wu, T. W. Lee, Q. Z. Wu and H. S. Liu, An eye state recognition method for drowsiness detection. In 2010 IEEE 71st Vehicular Technology Conference, pp. 1-5, Taipei, Taiwan, 16-19 May 2010. https://doi.org/ 10.1109/VETECS.2010.5493951
  • B. Wu and R. Nevatia, Cluster boosted tree classifier for multi-view, multi-pose object detection. In 2007 IEEE 11th International Conference on Computer Vision, pp. 1-8, Rio de Janeiro, Brazil, 14-21 October 2007. https://doi.org/10.1109/ICCV.2007.4409006
  • L. Cadena, A. Zotin, F. Cadena, A. Korneeva, A. Legalov and B. Morales, Noise reduction techniques for processing of medical images. In Proceedings of the World Congress on Engineering Vol. 1, pp. 5-9, London, U.K., 5-7 July, 2017.
  • C. Saravanan, Color image to grayscale image conversion. In 2010 Second International Conference on Computer Engineering and Applications Vol. 2, pp. 196-199, Bali Island, 19-21 March 2010. https://doi.org/ 10.1109/ICCEA.2010.192
  • C. Munteanu and V. Lazarescu, Evolutionary contrast stretching and detail enhancement of satellite images. In Proc. Mendel Vol. 99, pp. 94-99, 1999.
  • M. Abdullah-Al-Wadud, M. H. Kabir, M. A. A. Dewan and O. Chae, A dynamic histogram equalization for image contrast enhancement. IEEE Transactions on Consumer Electronics, 53(2), 593-600, 2007.
  • S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, T. Greer and K. Zuiderveld, Adaptive histogram equalization and its variations. Computer vision, graphics, and image processing, 39(3), 355-368, 1987. https://doi.org/10.1016/S0734-189X(87)80186-X
  • G. Deng and L. W. Cahill, An adaptive Gaussian filter for noise reduction and edge detection. In 1993 IEEE conference record nuclear science symposium and medical imaging conference, pp. 1615-1619, San Francisco, CA, USA, 31 Oct-6 Nov 1993. https://doi.org/ 10.1109/NSSMIC.1993.373563
  • T. Chen, K. K. Ma and L. H. Chen, Tri-state median filter for image denoising. IEEE Transactions on Image processing, 8(12), 1834-1838, 1999.
  • Y. Guo, Y. Liu, A. Oerlemans, S. Lao, S. Wu and M.S. Lew, Deep learning for visual understanding: A review. Neurocomputing, 187, 27-48, 2016. https://doi.org/10.1016/j.neucom.2015.09.116
  • I. Goodfellow, Y. Bengio and A. Courville, Deep learning. MIT press, London, England, 2016.
  • V. Suárez-Paniagua and I. Segura-Bedmar, Evaluation of pooling operations in convolutional architectures for drug-drug interaction extraction. BMC bioinformatics, 19(8), 39-47, 2018. https://doi.org/10.1186/s12859-018-2195-1

The effect of image preprocessing methods in CNN-based eye state detection

Year 2022, Volume 11, Issue 3, 496 - 505, 18.07.2022
https://doi.org/10.28948/ngumuh.1086414

Abstract

This research investigates how eye state detection is used to overcome simple problems including blinking, eyestrain, and driving drowsiness. The study utilized the ZJU dataset, and it suggested an eye state recognition approach based on image preprocessing methods and a deep learning-based convolutional neural network (CNN). First, multiple pooling layers were tested in the suggested CNN model, and it was discovered that the average pooling performed the best in the results. The image preprocessing methods applied on the ZJU dataset were then trained on the proposed CNN model and their results have been compared. The CNN model performed exceptionally well on the ZJU dataset, according to a comparison of the results obtained using the histogram equalization method, with 94.32% accuracy, 94.95% sensitivity, 92.42% specificity, 97.41% precision, and 96.16% F1 score performance metrics. The results of this investigation were compared to prior studies on the ZJU dataset, which had provided performance measures. When compared to the literature, it was obtained that the proposed technique has a high classification performance in detecting vision problems.

References

  • M. H. Yang, D. J. Kriegman and N. Ahuja, Detecting faces in images: A survey. IEEE Transactions on pattern analysis and machine intelligence, 24(1), 34-58, 2002. https://doi.org/10.1109/34.982883
  • T. Soukupova and J. Cech, Eye blink detection using facial landmarks. In 21st computer vision winter workshop, Rimske Toplice, Slovenia, 3-5 February 2006.
  • R. Huang, Y. Wang and L. Guo, P-FDCN based eye state analysis for fatigue detection. In 2018 IEEE 18th International Conference on Communication Technology (ICCT), pp. 1174-1178, Chongqing, China, 8-11 October 2018. https://doi.org/10.1109/ICCT.2018.8599947
  • D. M. Joshi, N.K. Rana and V. Misra, Classification of brain cancer using artificial neural network. In 2010 2nd international conference on electronic computer technology, pp. 112-116, 19-21 March 2010. https://doi.org/10.1109/ICECTECH.2010.5479975M
  • W. Zhao, R. Chellappa, P.J. Phillips and A. Rosenfeld, Face recognition: A literature survey. ACM computing surveys (CSUR), 35(4), 399-458, 2003. https://doi.org/10.1145/954339.954342
  • M. Divjak and H. Bischof, Eye Blink Based Fatigue Detection for Prevention of Computer Vision Syndrome. In MVA, pp. 350-353, 2009.
  • C. L. Chou, Y. H. Huang and S. C. Ho, Blink Detection Using Facial Landmark Blink Detector and Multi-Layer Perceptron. In NCS 2019, pp. 542-545, 2019.
  • M. Lalonde, D. Byrns, L Gagnon, N. Teasdale and D. Laurendeau, Real-time eye blink detection with GPU-based SIFT tracking. In Fourth Canadian Conference on Computer and Robot Vision (CRV'07), Montreal, Quebec, Canada, pp. 481-487, Montreal, Quebec, Canada, 28-30 May 2007. https://doi.org/ 10.1109/CRV.2007.54
  • J. He, W. Choi, Y. Yang, J. Lu, X. Wu and K. Peng, Detection of driver drowsiness using wearable devices: A feasibility study of the proximity sensor. Applied ergonomics, 65, 473-480, 2017. https://doi.org/10.1016/j.apergo.2017.02.016
  • M. Su, C. Yeh, S. Lin, P. Wang and S. Hou, An implementation of an eye-blink-based communication aid for people with severe disabilities. In 2008 International Conference on Audio, Language and Image Processing, pp. 351-356, Shanghai, China, 7-9 July, 2008. https://doi.org/ 10.1109/ICALIP.2008.4590090
  • S. Soltani and A. Mahnam, A practical efficient human computer interface based on saccadic eye movements for people with disabilities. Computers in biology and medicine, 70, 163-173, 2016. https://doi.org/10.1016/j.compbiomed.2016.01.012
  • Y. Dong, Y. Zhang, J. Yue and Z. Hu, Comparison of random forest, random ferns and support vector machine for eye state classification. Multimedia Tools and Applications, 75(19), 11763-11783, 2016. https://doi.org/10.1007/s11042-015-2635-0
  • L. Zhao, Z. Wang, G. Zhang, Y. Qi and X. Wang, Eye state recognition based on deep integrated neural network and transfer learning. Multimedia Tools and Applications, 77(15), 19415-19438, 2018. https://doi.org/10.1007/s11042-017-5380-8
  • Y. LeCun, Y. Bengio and G. Hinton, Deep learning. nature, 521(7553), 436-444, 2015. https://doi.org/10.1038/nature14539
  • L. Pauly and D. Sankar, Non intrusive eye blink detection from low resolution images using HOG-SVM classifier. International Journal of Image, Graphics and Signal Processing, 8(10), 11, 2016. https://doi.org/ 10.5815/ijigsp.2016.10.02
  • L. Pauly and D. Sankar, A novel method for eye tracking and blink detection in video frames. In 2015 IEEE International Conference on Computer Graphics, Vision and Information Security (CGVIS), pp. 252-257, Bhubaneswar, India, 2-3 November 2015. https://doi.org/ 10.1109/CGVIS.2015.7449931
  • Y. J. Han, W. Kim and J. S. Park, Efficient eye-blinking detection on smartphones: A hybrid approach based on deep learning. Mobile Information Systems, 2018, 2018. https://doi.org/10.1155/2018/6929762
  • W. O. Lee, E. C. Lee and K. R. Park, Blink detection robust to various facial poses. Journal of neuroscience methods, 193(2), 356-372, 2010. https://doi.org/10.1016/j.jneumeth.2010.08.034
  • Y. S. Wu, T. W. Lee, Q. Z. Wu and H. S. Liu, An eye state recognition method for drowsiness detection. In 2010 IEEE 71st Vehicular Technology Conference, pp. 1-5, Taipei, Taiwan, 16-19 May 2010. https://doi.org/ 10.1109/VETECS.2010.5493951
  • B. Wu and R. Nevatia, Cluster boosted tree classifier for multi-view, multi-pose object detection. In 2007 IEEE 11th International Conference on Computer Vision, pp. 1-8, Rio de Janeiro, Brazil, 14-21 October 2007. https://doi.org/10.1109/ICCV.2007.4409006
  • L. Cadena, A. Zotin, F. Cadena, A. Korneeva, A. Legalov and B. Morales, Noise reduction techniques for processing of medical images. In Proceedings of the World Congress on Engineering Vol. 1, pp. 5-9, London, U.K., 5-7 July, 2017.
  • C. Saravanan, Color image to grayscale image conversion. In 2010 Second International Conference on Computer Engineering and Applications Vol. 2, pp. 196-199, Bali Island, 19-21 March 2010. https://doi.org/ 10.1109/ICCEA.2010.192
  • C. Munteanu and V. Lazarescu, Evolutionary contrast stretching and detail enhancement of satellite images. In Proc. Mendel Vol. 99, pp. 94-99, 1999.
  • M. Abdullah-Al-Wadud, M. H. Kabir, M. A. A. Dewan and O. Chae, A dynamic histogram equalization for image contrast enhancement. IEEE Transactions on Consumer Electronics, 53(2), 593-600, 2007.
  • S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, T. Greer and K. Zuiderveld, Adaptive histogram equalization and its variations. Computer vision, graphics, and image processing, 39(3), 355-368, 1987. https://doi.org/10.1016/S0734-189X(87)80186-X
  • G. Deng and L. W. Cahill, An adaptive Gaussian filter for noise reduction and edge detection. In 1993 IEEE conference record nuclear science symposium and medical imaging conference, pp. 1615-1619, San Francisco, CA, USA, 31 Oct-6 Nov 1993. https://doi.org/ 10.1109/NSSMIC.1993.373563
  • T. Chen, K. K. Ma and L. H. Chen, Tri-state median filter for image denoising. IEEE Transactions on Image processing, 8(12), 1834-1838, 1999.
  • Y. Guo, Y. Liu, A. Oerlemans, S. Lao, S. Wu and M.S. Lew, Deep learning for visual understanding: A review. Neurocomputing, 187, 27-48, 2016. https://doi.org/10.1016/j.neucom.2015.09.116
  • I. Goodfellow, Y. Bengio and A. Courville, Deep learning. MIT press, London, England, 2016.
  • V. Suárez-Paniagua and I. Segura-Bedmar, Evaluation of pooling operations in convolutional architectures for drug-drug interaction extraction. BMC bioinformatics, 19(8), 39-47, 2018. https://doi.org/10.1186/s12859-018-2195-1

Details

Primary Language Turkish
Subjects Computer Science, Information System
Journal Section Computer Engineering
Authors

İsmail KAYADİBİ> (Primary Author)
AFYON KOCATEPE ÜNİVERSİTESİ, MÜHENDİSLİK FAKÜLTESİ, BİYOMEDİKAL MÜHENDİSLİĞİ BÖLÜMÜ
0000-0002-1949-8211
Türkiye


Gür Emre GÜRAKSIN>
AFYON KOCATEPE ÜNİVERSİTESİ, MÜHENDİSLİK FAKÜLTESİ, BİLGİSAYAR MÜHENDİSLİĞİ BÖLÜMÜ
0000-0002-1935-2781
Türkiye


Uçman ERGÜN>
AFYON KOCATEPE ÜNİVERSİTESİ, MÜHENDİSLİK FAKÜLTESİ, BİYOMEDİKAL MÜHENDİSLİĞİ BÖLÜMÜ
0000-0002-9218-2192
Türkiye

Publication Date July 18, 2022
Submission Date March 11, 2022
Acceptance Date April 29, 2022
Published in Issue Year 2022, Volume 11, Issue 3

Cite

Bibtex @research article { ngumuh1086414, journal = {Niğde Ömer Halisdemir Üniversitesi Mühendislik Bilimleri Dergisi}, eissn = {2564-6605}, address = {Niğde Üniversitesi Mühendislik Fakültesi, Merkez Yerleşke, Niğde}, publisher = {Nigde Omer Halisdemir University}, year = {2022}, volume = {11}, number = {3}, pages = {496 - 505}, doi = {10.28948/ngumuh.1086414}, title = {ESA tabanlı göz durumu tespitinde görüntü önişlem yöntemlerinin etkisi}, key = {cite}, author = {Kayadibi, İsmail and Güraksın, Gür Emre and Ergün, Uçman} }
APA Kayadibi, İ. , Güraksın, G. E. & Ergün, U. (2022). ESA tabanlı göz durumu tespitinde görüntü önişlem yöntemlerinin etkisi . Niğde Ömer Halisdemir Üniversitesi Mühendislik Bilimleri Dergisi , 11 (3) , 496-505 . DOI: 10.28948/ngumuh.1086414
MLA Kayadibi, İ. , Güraksın, G. E. , Ergün, U. "ESA tabanlı göz durumu tespitinde görüntü önişlem yöntemlerinin etkisi" . Niğde Ömer Halisdemir Üniversitesi Mühendislik Bilimleri Dergisi 11 (2022 ): 496-505 <https://dergipark.org.tr/en/pub/ngumuh/issue/71284/1086414>
Chicago Kayadibi, İ. , Güraksın, G. E. , Ergün, U. "ESA tabanlı göz durumu tespitinde görüntü önişlem yöntemlerinin etkisi". Niğde Ömer Halisdemir Üniversitesi Mühendislik Bilimleri Dergisi 11 (2022 ): 496-505
RIS TY - JOUR T1 - The effect of image preprocessing methods in CNN-based eye state detection AU - İsmailKayadibi, Gür EmreGüraksın, UçmanErgün Y1 - 2022 PY - 2022 N1 - doi: 10.28948/ngumuh.1086414 DO - 10.28948/ngumuh.1086414 T2 - Niğde Ömer Halisdemir Üniversitesi Mühendislik Bilimleri Dergisi JF - Journal JO - JOR SP - 496 EP - 505 VL - 11 IS - 3 SN - -2564-6605 M3 - doi: 10.28948/ngumuh.1086414 UR - https://doi.org/10.28948/ngumuh.1086414 Y2 - 2022 ER -
EndNote %0 Nigde Omer Halisdemir University Journal of Engineering Sciences ESA tabanlı göz durumu tespitinde görüntü önişlem yöntemlerinin etkisi %A İsmail Kayadibi , Gür Emre Güraksın , Uçman Ergün %T ESA tabanlı göz durumu tespitinde görüntü önişlem yöntemlerinin etkisi %D 2022 %J Niğde Ömer Halisdemir Üniversitesi Mühendislik Bilimleri Dergisi %P -2564-6605 %V 11 %N 3 %R doi: 10.28948/ngumuh.1086414 %U 10.28948/ngumuh.1086414
ISNAD Kayadibi, İsmail , Güraksın, Gür Emre , Ergün, Uçman . "ESA tabanlı göz durumu tespitinde görüntü önişlem yöntemlerinin etkisi". Niğde Ömer Halisdemir Üniversitesi Mühendislik Bilimleri Dergisi 11 / 3 (July 2022): 496-505 . https://doi.org/10.28948/ngumuh.1086414
AMA Kayadibi İ. , Güraksın G. E. , Ergün U. ESA tabanlı göz durumu tespitinde görüntü önişlem yöntemlerinin etkisi. NOHU J. Eng. Sci.. 2022; 11(3): 496-505.
Vancouver Kayadibi İ. , Güraksın G. E. , Ergün U. ESA tabanlı göz durumu tespitinde görüntü önişlem yöntemlerinin etkisi. Niğde Ömer Halisdemir Üniversitesi Mühendislik Bilimleri Dergisi. 2022; 11(3): 496-505.
IEEE İ. Kayadibi , G. E. Güraksın and U. Ergün , "ESA tabanlı göz durumu tespitinde görüntü önişlem yöntemlerinin etkisi", Niğde Ömer Halisdemir Üniversitesi Mühendislik Bilimleri Dergisi, vol. 11, no. 3, pp. 496-505, Jul. 2022, doi:10.28948/ngumuh.1086414

23135