FACIAL EXPRESSION CLASSIFICATION WITH HAAR FEATURES, GEOMETRIC FEATURES AND CUBIC BÉZIER CURVES

Year 2013, Volume: 13 Issue: 2, 1667 - 1673, 25.12.2013

Rembiye Kandemir , Gonca Özmen

Abstract

Facial expressions are nonverbal communication channels to interact with other people.  Computer recognition of human emotions based on facial expression is  an interesting and difficult problem. In this study, images were analyzed based on facial expressions and tried to identify different emotions, such as smile, surprise, sadness, fear, disgust, anger and  neutral. In practice, it was used Viola-Jones face detector  used  AdaBoost  algorithm for  finding the location of the face. Haar filters were used in finding the eyes and mouth. In cases where erroneous detection of the mouth and eyes, facial geometric ratios were used. Cubic Bézier curves were used in determining emotion. FEEDTUM facial expression database were used for training and testing. The seven different emotions used for the study, the recognition success rates ranged from 97%  to 60%.

Keywords

Emotion Recognition, Facial Expression Recognition, AdaBoost, Haar Features, Bézier curves.

References

• Ekman, P., “About brows: emotional and conversational signals. In: Human ethology: claims and limits of a new discipline: contributions to the Colloquium”, pp. 169– 2 Cambridge University Press, England,1979
• B. Fasel and J. Luettin, “Automatic facial expression analysis: a survey,” Pattern Recognition, vol. 1, no. 30, pp. 259–275, 2003.
• Mehrabian A., “Communication without Words”, Psychology Today, vol. 2,no. 4, pp. 53-56, 1968.
• L. Ma and K. Khorasani, “Facial Expression Recognition Using Constructive Feedforward and Neural Networks”, IEEE transactions on systems, man and cybernetics- part B: Cybernetics, vol.34, No. 3, June 2004.
• Ekman, P., Rosenberg, E.L., “What the face reveals: basic and applied studies of spontaneous expression using the facial action coding system (FACS)”, Oxford University Press, US, 2005
• Russell, J.A., Fernez-Dols, J.M. (eds.), “The Psychology of Facial Expression”, Cambridge University Press, Cambridge, 1997
• Darwin, C., “ The expression of the emotions in man and animals”, University of Chicago Press, Chicago, 1965
• Faigin, G., “The Artist’s Complete Guide to Facial Expression”, Watson-Guptill, 2008
• Bartlett M, Littlewort G, Frank M, et al., “ Automatic recognition of facial actions in spontaneous expressions [J]”, Journal of Multimedia, 1(6): 22–35, 2006.
• R. Lienhart, J. Maydt. “An extended set of haar-like features for rapid object detection”, Proceedings of the IEEE International Conference on Image Processing, Rochester, New York, vol. 1,pp. 900-903, 2002.
• Ryan, A., et al.,”Automated Facial Expression Recognition System”, Proc. Int. Carnahan Conf. on Security Technology, 2009, p. 172-177.
• Ashraf, A.B., et al., “The painful face-Pain expression recognition using active appearance models”, Image and Vision Computing, 2009. 27(12): p. 1788-1796.
• Vural, E., et al., “Automated Drowsiness Detection For Improved Driving Safety”, in Proc. of the Int. Conf. on Automotive Technologies ,2008.
• P. Ekman, D. Matsumoto, and W.V. Friesen, “Facial Expression in Affective Disorders, What the Face Reveals”, P. Ekman and E.L. Rosenberg, eds., pp. 429-439, 2005.
• OpenCv,http://www.cognotics.com/opencv/servo_2007_ series/part_2/sidebar.html#fig_1(Access: 13.07.2012)
• Jones, M. ve Viola P., “Rapid Object Detection using a Boosted Cascade of Simple Features”, 2001
• P. Viola and M. J. Jones, “Robust real-time object detection”, International Journal of Computer Vision, Vol. 57, No. 2, p.137–154, 2004.
• Cohen, I., N. Sebe, F. G. Gozman, M. C. Cirelo, T. S. Huang, “Learning Bayesian network classifiers for facial expression recognition both labeled and unlabeled data”, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. I-595 - I-601, 2003
• Khan, M., İ., Bhuiyan A., “Facial Features Approximation for Expression Detection in Human-Robot Interface”, Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, 2010
• Y.Wang, H.Ai, B.Wu, and C.Hung, “Real time facial expression recognition with adabost”, Proceengs of the 17 th International Conference on Pattern Recognition, vol 3, pp 926-929,2004
• J.Whitehil, C.W.Omlin, “Haar Features for FACS AU Recognition”, Proceengs of the 7 th International Conference on Automatic Face and Gasture Recognition, 2006 http://www.mmk.ei.tum.de/~waf/fgnet/feedtum.pdf (Access: 02012)
• S. U. Jung, D.H. Kim, K. H. An, M. J. Chung, “Efficient rectangle feature extraction for real-time facial expression recognition based on adaboost”, IEEE/RSJ International Conference on Intelligent Robotics and Systems (IROS), 200 C.Shan, S.Gong, P.W. McOwan, “Facial expression recognition based on Local Binary Patterns:A comprehensive study”, Image and Vision Computing 27 (2009) 803–816
• N.Sarode, S.Bhatia, “Facial Expression Recognition”, (IJCSE) International Journal on Computer Science and Engineering ,Vol. 02, No. 05, 2010, 1552-1557
• C. Papageorgiu, M. Oren, T. Poggio, “A general framework for Object Detection”, Proceedings of the International Conference on Computer Vision, Bombay, India, pp. 555562, 1998.
• Kurt, B., “Bilgisayar ile psikolojik durum değerlendirmesi”, Yüksek Lisans Tezi, KTÜ Fen Bilimleri Enstitüsü Bilgisayar Mühendisliği Anabilim Dalı, 2007
• G.Özmen, R.Kandemir, “Haar Dalgacıkları ve Kübik Bezier Eğrileri İle Yüz İfadesi Tespiti”, ELECO’2012, Bursa/Türkiye, 2012
• Y. Tian, T. Kanade, J. Cohn, “Recognizing action units for facial expression analysis”, IEEE Transactions on Pattern Analysis and Machine Intelligence 23 (2) (2001) 97–
• M. Pantic, L.J.M. Rothkrantz, “Facial action recognition for facial expression analysis from static face images”, IEEE Transactions on Systems, Man, and Cybernetics 34 (3) (2004) 1449–1461.
• Z. Zhang, M.J. Lyons, M. Schuster, S. Akamatsu, “Comparison between geometry-based and Gaborwavelets-based facial expression recognition using multilayer perceptron”, in: IEEE International Conference on Automatic Face & Gesture Recognition (FG), 1998.
• Z. Zeng, Y. Fu, G. I. Roisman, Z. Wen, Y. Hu and T. S. Huang, “Spontaneous Emotional Facial Expression Detection”, Journal of Multimedia, Vol. 1, No. 5, p. 1-8, 200 G.C. Littlewort, M.S. Bartlett, J. Chenu, I. Fasel, T. Kanda, H. Ishiguro, J.R. Movellan, “Towards social robots: Automatic evaluation of human-robot interaction by face detection and expression classification”, Advances in Neural Information Processing Systems, Vol 16, p. 1563-1570, 200
• P. Visutsak, “Emotion Classification using Adaptive SVMs”, World Academy of Science, Engineering and Technology 63, 2012
• K. Mase, “Recognition of facial expression from optical flow”, IEICE Transactions, pp. 3474-3483, 1991
• J.F. Cohn, A.J. Zlochower, J.J. Lien, and T. Kanade, “Feature-Point Tracking by Optical Flow Discriminates Subtle Differences in Facial Expression”, Proc. Int'l Conf. Automatic Face and Gesture Recognition, p. 396401, 1998
• I. Essa and A. Pentland, “Coding, Analysis Interpretation, Recognition of Facial Expressions”, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol.19, No. 7, p. 757-763, July 1997 Rembiye KANDEMİR, received the Ph.D. degree in Computer Engineering from the University of Trakya, Turkey. She is currently an assistant professor in the Department of Computer Engineering, University of Trakya. Her research interests include image analysis, computer vision and machine learning applied to face and body gesture recognition, multimodal human-computer interaction HCI). Gonca ÖZMEN, received B.S. degree in Computer Engineering Department at Kocaeli University, Turkey, and M.S. degree in Computer Engineering Department at Trakya University, Turkey. She is currently working in Computer Programming Department at Technical Science Vocational School, Kırklareli
• University, Turkey, as a lecturer.