Mixed texture descriptors for facial expression recognition

Abstract

This article proposes a novel approach to recognize facial expression using a combination of three local texture descriptors such as Local Binary Pattern (LBP), Gradient Direction Pattern (GDP) and Local Directional Pattern (LDP). Firstly, we need to perform the pre-process of face image, which includes face detection and face image cropping. Next, we calculate texture descriptors based on LBP, GDP and LDP operators. Histogram sequence concatenation is then applied to these descriptors after dividing the facial image into a number of non-overlapping blocks. Finally, the concatenated histograms represent an input to a Support Vector Machine (SVM), which will be used to classify facial images in one of the six universal expressions. Experimental results using CK+ and JAFFE database show that the proposed approach achieves superior recognition performance compared to the existing studies with classification accuracy of 97.62% and 93.84% respectively.

Keywords

• Facial expressions
• Human Face
• Emotional recognition
• Local Binary Pattern
• Gradient Direction Pattern
• Local Directional Pattern
• Support Vector Machine

References

1. Albert Mehrabian. Communication without words. In Communication theory, pages 193–200. Routledge, 2017.
2. Paul Ekman and WV Friesen. Facial Action Coding System: A Technique for Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto, California, USA, 1978.
3. Timo Ojala, Matti Pietikäinen, and David Harwood. A comparative study of texture measures with classification based on featured distributions. Pattern recognition, 29(1):51–59, 1996.
4. Taskeed Jabid, Md Hasanul Kabir, and Oksam Chae. Local directional pattern (ldp) for face recognition. In 2010 digest of technical papers international conference on consumer electronics (ICCE), pages 329–330. IEEE, 2010.
5. Taskeed Jabid, Md Hasanul Kabir, and Oksam Chae. Robust facial expression recognition based on local directional pattern. ETRI journal, 32(5):784–794, 2010.
6. Xiaoyang Tan and William Triggs. Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE transactions on image processing, 19(6):1635–1650, 2010.
7. F Ahmed. Gradient directional pattern: a robust feature descriptor for facial expression recognition. Electronics letters, 48(19):1203–1204, 2012.
8. Faisal Ahmed and Emam Hossain. Automated facial expression recognition using gradient-based ternary texture patterns. Chinese Journal of Engineering, 2013(1):831747, 2013.

9. Matthew A Turk, Alex Pentland, et al. Face recognition using eigenfaces. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), volume 91, pages 586–591, 1991.
10. Curtis Padgett, Garrison W Cottrell, and Ralph Adolphs. Categorical perception in facial emotion classification. In Proceedings of the 18th Annual Conference of the Cognitive Science Society, pages 249–253, University of California, San Diego, 1996. Routledge.
11. Marian Stewart Bartlett, Javier R Movellan, and Terrence J Sejnowski. Face recognition by independent component analysis. IEEE Transactions on neural networks, 13(6):1450–1464, 2002.
12. Peter N. Belhumeur, Joao P Hespanha, and David J. Kriegman. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Transactions on pattern analysis and machine intelligence, 19(7):711–720, 2002.
13. Hong-Bo Deng, Lian-Wen Jin, Li-Xin Zhen, Jian-Cheng Huang, et al. A new facial expression recognition method based on local gabor filter bank and pca plus lda. International Journal of Information Technology, 11(11):86–96, 2005.
14. Michael Lyons, Shigeru Akamatsu, Miyuki Kamachi, and Jiro Gyoba. Coding facial expressions with gabor wavelets. In Proceedings Third IEEE international conference on automatic face and gesture recognition, pages 200–205. IEEE, 1998.
15. Zhengyou Zhang, Michael Lyons, Michael Schuster, and Shigeru Akamatsu. Comparison between geometry-based and gabor-wavelets-based facial expression recognition using multi-layer perceptron. In Proceedings Third IEEE International Conference on Automatic face and gesture recognition, pages 454–459. IEEE, 1998.
16. Caifeng Shan, Shaogang Gong, and Peter W McOwan. Facial expression recognition based on local binary patterns: A comprehensive study. Image and vision Computing, 27(6):803–816, 2009.
17. Seyed Mehdi Lajevardi and Zahir M Hussain. Facial expression recognition using log-gabor filters and local binary pattern operators. In Proceedings of the International Conference on Communication, Computer and Power, pages 349–353, 2009.
18. Byungyong Ryu, Adín Ramírez Rivera, Jaemyun Kim, and Oksam Chae. Local directional ternary pattern for facial expression recognition. IEEE Transactions on Image Processing, 26(12):6006–6018, 2017.
19. Faisal Ahmed and Md Hasanul Kabir. Facial expression recognition under difficult conditions: A comprehensive study on edge directional texture patterns. International Journal of Applied Mathematics and Computer Science, 28(2):399–409, 2018.
20. Ying Tong and Rui Chen. Local dominant directional symmetrical coding patterns for facial expression recognition. Computational intelligence and neuroscience, 2019(1):3587036, 2019.
21. F Abdat, C Maaoui, and A Pruski. Suivi du gradient pour la localisation descaractéristiques faciales dans des images statiques. In Colloque GRETSI, pages 11–14, 2007.
22. Frank Y Shih and Chao-Fa Chuang. Automatic extraction of head and face boundaries and facial features. Information Sciences, 158:117–130, 2004.
23. Takeo Kanade, Jeffrey F Cohn, and Yingli Tian. Comprehensive database for facial expression analysis. In Proceedings fourth IEEE international conference on automatic face and gesture recognition (cat. No. PR00580), pages 46–53. IEEE, 2000.
24. Michael J Lyons, Julien Budynek, and Shigeru Akamatsu. Automatic classification of single facial images. IEEE transactions on pattern analysis and machine intelligence, 21(12):1357–1362, 1999.
25. Thorsten Joachims. Making large-scale svm learning practical. In B. Schölkopf, C. Burges, and A. Smola, editors, Advances in Kernel Methods- Support Vector Learning. MIT Press, 1998.