Emotion Analysis using Facial Expressions in Video

Abstract

The topic of human computer interaction is one of the increasingly popular topics in recent times. Human facial expression and emotion analysis with the computer is one of the complex problems as well as interesting. In this paper, emotion analysis was made on human images. In the study, 5 different emotional states, being happy, angry, sad, surprised and neutral, were analyzed. The proposed algorithm basically consists of 3 steps. The first is the preprocessing of the images required for the SVM model. The second is the creation of the SVM model that could perform emotion analysis. The final step is to assign facial expressions to the relevant emotion class. In this study, JAFFE dataset and many images available from Google were used. The recognition success rates of 5 different emotions determined for the study were found between 80% and 100%.

Keywords

• Emotion Analysis
• Support Vector Machines (SVM)
• Facial Expressions.

Videodaki Yüz İfadeleri Üzerinden Duygu Analizi

Öz

İnsan ve bilgisayar etkileşimi konusu, son zamanlarda giderek daha popüler hale gelen konulardan biridir. Bilgisayar ile insan yüz ifadesi ve duygu analizi ilginç olduğu kadar karmaşık sorunlardan biridir. Bu çalışmada insan görüntüleri üzerinden duygu analizi yapılmıştır. Çalışmada mutlu, kızgın, üzgün, şaşırmış ve nötr olmak üzere 5 farklı duygu durumu analiz edilmiştir. Önerilen algoritma temelde 3 adımdan oluşmaktadır. Birincisi, SVM modeli için gerekli görüntülerin ön işlemesidir. İkincisi, duygu analizi yapabilen SVM modelinin oluşturulmasıdır. Son adım,yüz ifadelerin ilgili duygu sınıfa yönlendirilmesidir. Bu çalışmada, JAFFE veri seti ve Google'da bulunan birçok görsel kullanılmıştır. Çalışma için belirlenen 5 farklı duygu için başarı oranları % 80 ile % 100 arasında değişmektedir.

Anahtar Kelimeler

• Duygu Analizi
• Destek Vektör Makineleri (SVM)
• Yüz İfadeleri

Kaynakça

1. Romero, M. , Pears N. (2009). Landmark localisation in 3D face data. 6th IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS 2009, 73-78.
2. Valstar, M.F. & Pantic, M. (2012). Fully automatic recognition of the temporal phases of facial actions. IEEE Trans. Syst. Man Cybern. Part B Cybern. 42(1), 28-43.
3. Gudi, A. , Tasli, H.E. , Den Uyl, T. M. & Maroulis A. (2015). Deep learning based FACS action unit occurence and intensity estimation.
4. Khorromi, P.K. , Paine, T.L. , Brady, K. , Dagli, C. & Huang, T.S. (2016). How deep neural networks can improve emotion recognition on video data. 1-5.
5. Nguyen's Yearn, HD., Lee, G-S. , Yang, H-J. , Na, L. & Kim, H. (2018). Facial emotion recognition using an ensemble of multilevel convolutional neural networks. International Journal of Pattern Recognition and Artificial Intelligence.
6. Cao, T. & Li, M. (2019). Facial expression recognition algorithm based on the combination of CNN and K-Means. 11th International Conference on Machine Learning and Computing.
7. Oztel, I. (2018). Facial expression detection with machine learning methods on partial and full face images. PhD Thesis, Sakarya University
8. Tenekeci, M. E. , Gumuscu, A., Baytak, A., Aslan, E. (2014). Emotion analysis from image with OpenCV. Academic Informatics'14 - XVI. Academic Informatics Conference Proceedings.

9. Ozmen, G. (2012). Facial expression recognition with cubic bezier curves. Master's Thesis, Trakya University.
10. Martinez, J. C. "https://livecodestream.dev/post/detecting-face-features-with-python/ 21.02.2021".
11. Lyons. M. , Akamatsu, S. , Kamachi, M. & Gyoba, J. (1998). Coding facial expressions with Gabor wavelets. Third IEEE International Conference on Automatic Face and Gesture Recognition, 200-205.