Emotion Recognition Based on Interval Type-2 Fuzzy Logic from Facial Expression

Abstract

Automatic recognition of facial emotion plays an effective and important role in Human–Computer Interaction (HCI). There are various emotion recognition approaches have been proposed in the literature. The analytic face model consisted of a 26-dimensional geometric feature vector. These properties are used effectively to identify facial changes resulting from different expressions. The variation and uncertainties of these features make the emotion recognition problem more complicated. For decreasing these complications, we propose a distance-based clustering and uncertainty measures of the base new method for Emotion Recognition from Facial Expression using automatically selects 19 diagnostics of Action Units (AUs) in a 2D facial image using Type-2 Fuzzy inference system. The proposed system includes an automated generation scheme of the geometric facial feature vector. The proposed system has classified six facial expressions using the MUG Facial Expression database. The experimental results show that the proposed model is very efficient in uncertainty management policy and recognizes six basic emotions with an average precision rate of 86.175%.

Keywords

• Action Unit,Emotion Recognition,Facial Expression Recognition,Human–Computer Interaction,Interval Type-2 Fuzzy System

References

1. C. Silva, A. Sobral, and R. T. Vieira, “An automatic facial expression recognition system evaluated by different classifiers,” no. October, 2014.
2. R. Contreras, O. Starostenko, and V. Alarcon-aquino, “Facial Feature Model for Emotion Recognition,” pp. 11–21, 2010.
3. O. Starostenko, R. Contreras, V. A. Aquino, and L. F. Pulido, “A Fuzzy Reasoning Model for Recognition of Facial Expressions,” vol. 15, no. 2, pp. 163–180, 2011.
4. A. J. Calder, A. M. Burton, P. Miller, A. W. Young, and S. Akamatsu, “A principal component analysis of facial expressions,” Vision Res., 2001.
5. M. H. Siddiqi, R. Ali, A. M. Khan, E. S. Kim, G. J. Kim, and S. Lee, “Facial expression recognition using active contour-based face detection, facial movement-based feature extraction, and non-linear feature selection,” Multimed. Syst., 2015.
6. S. Ulukaya and Ç. Eroǧlu Erdem, “Estimation of the neutral face shape using Gaussian mixture models,” in ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, 2012.
7. D. Filko and G. Martinovic, “Emotion Recognition System by a Neural Network Based Facial Expression Analysis,” Autom. ‒ J. Control. Meas. Electron. Comput. Commun., 2013.
8. S. Jirayucharoensak, S. Pan-Ngum, and P. Israsena, “EEG-Based Emotion Recognition Using Deep Learning Network with Principal Component Based Covariate Shift Adaptation,” Sci. World J., 2014.

9. Z. Yin, M. Zhao, Y. Wang, J. Yang, and J. Zhang, “Recognition of emotions using multimodal physiological signals and an ensemble deep learning model,” Comput. Methods Programs Biomed., 2017.
10. A. T. Lopes, E. de Aguiar, A. F. De Souza, and T. Oliveira-Santos, “Facial expression recognition with Convolutional Neural Networks: Coping with few data and the training sample order,” Pattern Recognit., 2017.
11. V. K. N. K. Kamlesh Pai, M. Balrai, S. Mogaveera, and D. Aeloor, “Face Recognition Using Convolutional Neural Networks,” in Proceedings of the 2nd International Conference on Trends in Electronics and Informatics, ICOEI 2018, 2018.
12. J. J. Lee, M. Zia Uddin, and T.-S. Kim, “spatiotemporal human facial expression recognition using fisher independent component analysis and Hidden Markov Model,” 2009.
13. J. Ou, X. B. Bai, Y. Pei, L. Ma, and W. Liu, “Automatic facial expression recognition using Gabor filter and expression analysis,” in ICCMS 2010 - 2010 International Conference on Computer Modeling and Simulation, 2010.
14. C. F. Chuang and F. Y. Shih, “Recognizing facial action units using independent component analysis and support vector machine,” Pattern Recognit., 2006.
15. A. Halder et al., “Reducing uncertainty in interval type-2 fuzzy sets for qualitative improvement in emotion recognition from facial expressions,” in IEEE International Conference on Fuzzy Systems, 2012.
16. A. Konar, A. Chakraborty, A. Halder, R. Mandal, and R. Janarthanan, “Interval type-2 fuzzy model for emotion recognition from facial expression,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2012, vol. 7143 LNCS, pp. 114–121.
17. A. Chakraborty, A. Konar, U. K. Chakraborty, and A. Chatterjee, “Emotion recognition from facial expressions and its control using fuzzy logic,” IEEE Trans. Syst. Man, Cybern. Part ASystems Humans, vol. 39, no. 4, pp. 726–743, 2009.
18. M. Ilbeygi and H. Shah-Hosseini, “A novel fuzzy facial expression recognition system based on facial feature extraction from color face images,” Eng. Appl. Artif. Intell., vol. 25, no. 1, pp. 130–146, Feb. 2012.
19. A. Halder, R. Mandal, A. Chakraborty, A. Konar, and R. Janarthanan, “Application of general type-2 fuzzy set in emotion recognition from facial expression,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2011, vol. 7076 LNCS, no. PART 1, pp. 460–468.
20. A. Halder, R. Mandal, A. Konar, A. Chakraborty, and R. Janarthanan, “Emotion recognition from facial expression using general type-2 fuzzy set,” in 2011 International Conference on Recent Trends in Information Systems, ReTIS 2011 - Proceedings, 2011, pp. 263–268.
21. A. Halder, R. Mandal, A. Konar, A. Chakraborty, and R. Janarthanan, “Emotion Recognition from Facial Expression using General Type-2 Fuzzy Set,” pp. 263–268, 2011.
22. N. Esau, E. Wetzel, L. Kleinjohann, and B. Kleinjohann, “Real-Time Facial Expression Recognition Using Model Fuzzy Emotion,” 2007.
23. S. Ghosh and G. Paul, “A Type-2 Approach in Emotion Recognition and an Extended Type-2 Approach for Emotion Detection,” Fuzzy Inf. Eng., vol. 7, no. 4, pp. 475–498, Dec. 2015.
24. N. Esau, E. Wetzel, L. Kleinjohann, and B. Kleinjohann, “Real-time facial expression recognition using a fuzzy emotion model,” in IEEE International Conference on Fuzzy Systems, 2007.
25. D. Y. Liliana, B. T., and R. W. M., “Fuzzy Emotion Recognition Using Semantic Facial Features and Knowledge-based Fuzzy,” Int. J. Eng. Technol., vol. 11, no. 2, pp. 177–186, Apr. 2019.
26. J. M. Mendel, “Fuzzy sets for words: a new beginning,” 2004.
27. J. Mendel, “Type-2 Fuzzy Sets, A Tribal Parody [Discussion Forum,” IEEE Comput. Intell. Mag., 2010.
28. R. J. Davidson et al., “How are emotions distinguished from moods, temperament, and other related affective constructs?,” Nat. Emot. Fundam. Quest. Ser. Affect. Sci., 1994.
29. P. Ekman, “All Empotions Are Basic,” in The Nature of Emotion, 1994.
30. P. Ekman, “Moods, Emotions, and Traits,” The Nature of Emotion: Fundamental Questions. 1994.
31. P. Ekman, R. J. Davidson, M. Ricard, and B. A. Wallace, “Buddhist and psychological perspectives on emotions and well-being,” Curr. Dir. Psychol. Sci., 2005.
32. P. Ekman and W. V Friesen, “Facial action coding system: A technique for the measurement of facial movement.,” CA Consult. Psychol. Press. Ellsworth, PC, Smith, CA (1988). From Apprais. to Emot. Differ. among unpleasant Feel. Motiv. Emot., 1978.
33. P. Ekman and W. V. Friesen, The Facial Action Coding System. 1978.
34. P. Ekman and W. V Friesen, Unmasking the face: A guide to recognizing emotions from facial clues. 2003.
35. A. Majumder, L. Behera, and V. K. Subramanian, “Emotion recognition from geometric facial features using self-organizing map,” Pattern Recognit., vol. 47, no. 3, pp. 1282–1293, 2014.
36. P. Ekman, “Facial expression and emotion,” Am. Psychol., 1993.
37. P. Viola and M. J. Jones, “Robust Real-time Object Detection,” 2001.
38. P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” 2005.
39. K. W. Wan, K. M. Lam, and K. C. Ng, “An accurate active shape model for facial feature extraction,” Pattern Recognit. Lett., 2005.
40. Y. H. Lee, Y. Lee, D. S. Yang, J. H. Park, and Y. Kim, “Facial feature extraction using enhanced active shape model,” in 2013 International Conference on Information Science and Applications, ICISA 2013, 2013.
41. L. A. Zadeh, “Soft computing and fuzzy logic,” IEEE Softw., vol. 11, no. 6, pp. 48–56, 1994.
42. L.A.Zadeh, “A fuzzy -algorithmic approach to the definition of complex or imprecise concepts,” no. 8, pp. 249–291, 1976.
43. M. Dirik, O. Castillo, A. F. Kocamaz, M. Dirik, O. Castillo, and A. F. Kocamaz, “Visual-Servoing Based Global Path Planning Using Interval Type-2 Fuzzy Logic Control,” Axioms 2019, Vol. 8, Page 58, vol. 8, no. 2, p. 58, May 2019.
44. The MUG Facial Expression Database | Multimedia Understanding Group.” [Online]. Available: https://mug.ee.auth.gr/fed/. [Accessed: 06-Jul-2019].
45. O. Castillo, P. Melin, J. Kacprzyk, and W. Pedrycz, “Type-2 Fuzzy Logic: Theory and Applications,” in 2007 IEEE International Conference on Granular Computing (GRC 2007), 2007, pp. 145–145.
46. O. Castillo and P. Melin, “Hybrid intelligent systems for time series prediction using neural networks, fuzzy logic, and fractal theory,” IEEE Trans. Neural Networks, 2002.
47. J. M. Mendel, “Advances in type-2 fuzzy sets and systems,” Inf. Sci. (Ny)., 2007.