Real-time Facial Emotion Classification Using Deep Learning

Year 2019, Volume: 2 Issue: 1, 13 - 17, 15.07.2019

Emre Dandıl , Rıdvan Özdemir

Abstract

Facial emotion recognition has an important position in the computer vision and artificial intelligence field. In addition, real-time face recognition applications have to be able to be performed at high speed and accuracy rate in order to make human-computer interaction successful in increasing artificial intelligence and humanoid robot applications. In this study, we detected the faces on real-time video data to recognize the anger, fear, happy, surprise, sad and neutral emotions upon these detected faces using deep learning methods. We created our own dataset to use in this study for six different facial emotions. At first stage, we created a convolutional neural network and trained it over our dataset by scratching method and we achieved 50% accuracy rate. Then, we increased the number of images in our database by 3 times, and get better accuracy which is 62%. Thanks to transfer training method and AlexNet's pre-trained networks, we reached 74% accuracy rate after increasing the number of images 80% in the dataset. In addition, we achieved 72% accuracy rate when we test our network which is trained with our own dataset with the Compound Emotion dataset. The basic reason of this decrease can be angry emotion because there are differences poses between our dataset and Compound Emotion dataset for angry emotion images. However, we obtained 100% accuracy rate for happy emotion and 89% for sad emotion. It has been seen that the work we are doing gives successful results when tested with different people in different ambient and light conditions.

Keywords

classification, convolutional neural network, deep learning, emotion recognition, face recognition

References

• [1] Shan, C., Shaogang, G., & Peter, W. M. (2009). Facial expression recognition based on local binary patterns: A comprehensive study (Image and Vision Computing, 27(6), p. 803-816).
• [2] Chul, Ko, B. (2018). A Brief Review of Facial Emotion Recognition Based on Visual Information (Sensors, 18, 401), p. 1-2.
• [3] Kołakowska, A., Landowska, A., Szwoch, M., Szwoch, W., & Wróbel, M. R. (2014). Human-Computer Systems Interaction: Backgrounds and Applications (Cham: Springer International Publishing), chapter 3, p. 51–62.
• [4] Hoang, Le T. (2011). Applying Artificial Neural Networks For Face Recognition (Advances in Artificial Neural Systems, vol. 2011, Article ID 673016), p. 15.
• [5] Breuer, R., & Kimmel, R. (2017). A Deep Learning Perspective on the Origin of Facial Expressions (arXiv 1705.01842).
• [6] Jung, H., Lee, S., Yim, J., Park, S., & Kim, J. (2015). Fine-Tuning in Deep Neural Networks For Facial Expression Recognition (In Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile), p. 2983–2991.
• [7] Kahou, S. E., Pal, C., Bouthillier, X., Froumenty, P., Gülçehre, C., Memisevic, R., Vincent, P., Courville, A., Bengio, Y., & Ferrari, R. C., et al. (2013). Combining modality specific deep neural networks for emotion recognition in video (In Proceedings of the 15th ACM on International conference on multimodal interaction, ACM), p. 543–550.
• [8] https://stock.adobe.com/search/images?load_type=search&native_visual_search=&similar_content_id=&is_recent_search=&k=facial+emotions
• [9] Viola, P., & Jones, M. J. (2004). Robust Real-Time Face Detection (International Journal of Computer Vision, vol. 57, no. 2), p. 137-154.
• [10] Aydilek, İ. B. (2017). Derin Öğrenme ile Tüketilen Besin İçeriklerinin Yaklaşık Kestirimi (International Conference on Computer Science and Engineering), p. 2.
• [11] Savoiu, A., & Wong, J. (2017). Recognizing Facial Expressions Using Deep Learning (Stanford University), p. 4.
• [12] Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet Classification with Deep Convolutional Neural Networks (NIPS), p. 5.