Yüz ifadelerinin otomatik analizi üzerine bir literatür çalışması

Yıl 2016, , 383 - 398, 01.08.2016

Sümeyye Bayrakdar , Devrim Akgün , İbrahim Yücedağ

https://doi.org/10.16984/saufenbilder.92940

Cited By: 3

Öz

Yüz ifadeleri, duygulara kişilerarası iletişimde anlam bütünlüğü kazandıran evrensel işaretlerdir. Aynı zamanda yüz ifadelerindeki değişimler, duygu psikolojisinde en önemli ipuçları olarak kabul edilmektedir. Yüz ifade analizinin; insan davranışlarının analizi, insan-insan etkileşimi ve insan-bilgisayar etkileşimi gibi çok geniş bir uygulama yelpazesi vardır. İnsanlar yüz ifadelerini tanımada ve yorumlamada herhangi bir sorun yaşamazken bu işlem bilgisayarlarda zorlu bir problem olarak karşımıza çıkmaktadır. Fakat özellikle insan-bilgisayar etkileşiminde yaşanan gelişmelerle birlikte, insanlara ait duyguların bilgisayarlar tarafından anlaşılması elzem bir konu haline gelmiştir. Bunun yanında güvenlik, psikoloji, sağlık, robotik ve sanal gerçeklilik gibi birçok çalışma alanında da yüz ifadelerinin analizi ve tanıması giderek yaygınlaşmıştır. Bu nedenlerden dolayı, yüz ifadelerinin hızlı bir şekilde analiz edilmesi ve analiz edilen yüz ifadelerinin doğru olarak tanınması farklı uygulama alanlarında birçok yazılım sistemi için kritik bir rol oynamaktadır. Bu çalışmada, yüz ifadelerinin tanıması ile ilgili literatürde yapılan çalışmalar incelenmiş ve yüz ifadelerinin analizinde kullanılan bazı yöntemlere yer verilmiştir.

Anahtar Kelimeler

Yüz ifade analizi, Yüz ifadesi tanıma; Görüntü işleme

A survey on automatic analysis of facial expressions

Öz

Facial expressions are universal symbols of emotions that provide cohesion to interpersonal communication. At the same time, the changes in facial expressions are considered to be the most important hints in the psychology of emotion. Facial expression analysis has widespread range of application in areas such as analysis of human behaviors, human-human interaction and human-computer interaction. While humans do not have any difficulty for recognizing and interpreting facial expressions, it is challenging problem for the computers. However, especially with advances in human-computer interaction, understanding the human emotions by computer has become an indispensable issue. Furthermore, analysis and recognition of facial expressions has pervaded in various areas such as security, psychology, health, robotics and virtual reality. For these reasons, the analyzing of facial expressions quickly and the recognition of facial expressions correctly according to the analyzed facial expressions play a critical role for many software systems in different applications. In this study, the methods used in the analysis of facial expressions and the studies in the literature on emotion recognition were investigated.

Anahtar Kelimeler

facial expression analysis, facial expression recognition, image processing

Kaynakça

• C. Darwin, Expression of the Emotions in Man and Animals, London: John Murray, 1872.
• A. Mehriban, «Communication without Words,» Psychology Today, cilt 2, no. 4, pp. 53-56, 1968.
• P. Ekman ve W. V. Friesen, «Constants Across Cultures in the Face and Emotion,» Journal of Personality and Social Psychology, cilt 17, no. 2, pp. 124-129, 1971.
• P. Ekman, «Universals and Cultural Differences in Facial Expressions of Emotion,» Nebraska Symposium on Motivation , Lincoln, 1972.
• M. Suwa, N. Sugie ve K. Fujimora, «A Preliminary Note on Pattern Recognition of Human Emotional Expression,» Proceedings of the Fourth International Joint Conference on Pattern Recognition, Kyoto, 1978.
• B. Fasel ve J. Luettin, «Automatic Facial Expression Analysis: A Survey,» Pattern Recognition, cilt 36, pp. 259-275, 2003.
• J. Kumaria, R. Rajesh ve K. Poojaa, «Facial Expression Recognition: A survey,» Second International Symposium on Computer Vision and the Internet (VisionNet’15), Kochi, 2015.
• S. K. Gupta, S. Agrwal, Y. K. M. ve N. Nain, «A Hybrid Method of Feature Extraction for Facial Expression Recognition,» Seventh International Conference on Signal Image Technology & Internet-Based Systems, Dijon, 2011.
• L. Zhang, S. Chen, T. Wang ve Z. Liu, «Automatic Facial Expression Recognition Based on Hybrid Features,» International Conference on Future Electrical Power and Energy Systems, 2012.
• W. Gaebel ve W. Wölwer, «Facial Expression and Emotional Face Recognition in Schizophrenia and Depression,» European Archives of Psychiatry and Clinical Neuroscience, cilt 242, no. 1, pp. 46-52, 1992.
• M. K. Mandal, R. Pandey ve A. B. Prasad, «Facial Expressions of Emotions and Schizophrenia: A Review,» chizophrenia Bulletin, cilt 24, no. 3, pp. 399-412, 1998.
• C. Alvino, C. Kohler, F. Barrett, R. E. Gur, R. C. Gur ve R. Verma, «Computerized Measurement of Facial Expression of Emotions in Schizophrenia,» Journal of Neuroscience Methods, cilt 163, no. 2, pp. 350-361, 2007.
• A. Butalia, M. Ingle ve P. Kulkarni, «Facial Expression Recognition for Security,» International Journal of Modern Engineering Research (IJMER), cilt 2, no. 4, pp. 1449-1453, 2012.
• A. A. M. Al-modwahi, O. Sebetela, L. N. Batleng, B. Parhizkar ve A. H. Lashkari, «Facial Expression Recognition Intelligent Security System for Real Time Surveillance,» International Conference on Computer Graphics and Virtual Reality, Las Vegas Nevada, 2012.
• G. Lu, L. Yuan, X. Li ve H. Li, «Facial Expression Recognition of Pain in Neonates,» International Conference on Computer Science and Software Engineering, Wuhan, 2008.
• G. Lu, X. Li ve H. Li, «Facial expression recognition for neonatal pain assessment,» International Conference on Neural Networks and Signal Processing, Nanjing, 2008.
• S. Hachisuka, «Human and Vehicle Drowsiness Detection by Facial Expression,» International Conference on Biometrics and Kansei Engineering (ICBAKE), Tokyo, 2013.
• M. E. Jabon, J. N. Bailenson, E. Pontikakis, L. Takayama ve C. Nass, «Facial Expression Analysis for Predicting Unsafe Driving Behavior,» IEEE Pervasive Computing, cilt 10, no. 4, pp. 84-95, 2011.
• A. Dureha, «An Accurate Algorithm for Generating a Music Playlist based on Facial Expressions,» International Journal of Computer Applications, cilt 100, no. 9, pp. 33-39, 2014.
• H. Kabani, S. Khan, O. Khan ve S. Tadvi, «Emotion Based Music Player,» International Journal of Engineering Research and General Science, cilt 3, no. 1, pp. 750-756, 2015.
• P. Ekman ve W. V. Friesen, «Facial Action Coding System: A Technique for the Measurement of Facial Movement,» Consulting Psychology Press., 1978.
• K. Scherer ve P. Ekman, «Methods for Measuring Facial Action,» Handbook of Methods in Nonverbal Behavior Research , New York, Cambridge University Press, 1982, pp. 45-135.
• M. S. Bartlett, P. A. Viola, T. J. Sejnowski, B. A. Golomb, J. Larsen, J. C. Hager ve P. Ekman, «Classifying Facial Action,» Advances in Neural Information Processing Systems 8, pp. 823-829, 1996.
• Y. Yacoob ve L. S. Davis, «Recognizing Human Facial Expressions from Long Image Sequences using Optical Flow,» IEEE Transactions on Pattern Analysis and Machine Intelligence , cilt 18, no. 6, pp. 636 - 642, 1996.
• N. Tsapatsoulis, M. Leonidou ve S. Kollias, «Facial Expression Recognition using HMM with Observation Dependent Transition Matrix,,» IEEE Second Workshop on Multimedia Signal Processing, Redondo Beach, 1998.
• Z. Zhang, «Feature-Based Facial Expression Recognition: Sensitivity Analysis and Experiments with a Multi-Layer Perceptron,» International Journal of Pattern Recognition and Artificial Interlligence, cilt 13, no. 6, pp. 893-911, 1999.
• I. Cohen, A. Garg ve T. S. Huang, «Emotion Recognition from Facial Expressions using Multilevel HMM,» Neural Information Processing Systems (NIPS), 2000.
• S. B. Gokturk, J.-Y. Bouguet, C. Tomasi ve B. Girod, «Model-Based Face Tracking for View-Independent Facial Expression Recognition,» Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition (FGR'02) , 2002.
• L. C. D. Silva ve S. C. Hui, «Real-time Facial Feature Extraction and Emotion Recognition,» Fourth Pacific Rim Conference on Multimedia, Information, Communications and Signal Processing, Singapore, 2003.
• X. Feng, M. Pietikäinen ve A. Hadid, «Facial Expression Recognition Based on Local Binary Patterns and Linear Programming,» Pattern Recognition and Image Analysis, cilt 15, no. 2, p. 546–548, 2005.
• S. M. Lajevardi ve M. Lech, «Facial Expression Recognition from Image Sequences Using Optimized Feature Selection,» 23rd International Conference Image and Vision Computing (IVCNZ), New Zealand, 2008.
• P. Martins, J. Sampaio ve J. Batista, «Facial Expression Recognition Using Active Appearance Models,» International Conference on Computer Vision Theory and Applications (VISAPP), Funchal, 2008.
• M. Song, D. Tao, Z. Liu, X. Li ve M. Zhou, «Image Ratio Features for Facial Expression Recognition Application,» IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics, cilt 40, no. 3, pp. 779-788, 2010.
• L. Oliveira, M. Mansano, A. Koerich ve J. Alceu de Souza Britto, «2D Principal Component Analysis for Face and Facial-Expression Recognition,» Computing in Science & Engineering , cilt 13, no. 3, pp. 9-13, 2011.
• S. B. Kazmi, Qurat-ul-Ain ve M. A. Jaffar, «Wavelets-based Facial Expression Recognition using a Bank of Support Vector Machines,» Soft Computing, cilt 16, no. 3, pp. 369-379, 2012.
• F. S. Farahani, M. Sheikhan ve A. Farrokhi, «A Fuzzy Approach for Facial Emotion Recognition,» 13th Iranian Conference on Fuzzy Systems (IFSC), Qazvin, 2013.
• X. Chen ve W. Cheng, «Facial Expression Recognition Based on Edge Detection,» International Journal of Computer Science & Engineering Survey (IJCSES), cilt 6, no. 2, pp. 1-9, 2015.
• S. C. Neoh, L. Zhang, K. Mistry, M. A. Hossain, C. P. Lim ve P. K. Nauman Aslam, «Intelligent Facial Emotion Recognition using a Layered Encoding Cascade Optimization Model,» Elsevier Applied Soft Computing, cilt 34, pp. 72-93, 2015.
• D. Ibtesham, «A Survey on Facial Expression Recognition,» New Mexico, 2012.
• M. Murtaza, M. Sharif, M. Raza ve J. H. Shah, «Analysis of Face Recognition under Varying Facial Expression: A Survey,» The International Arab Journal of Information Technology, cilt 10, no. 4, pp. 378-388, 2013.
• N. N. Khatri, Z. H. Shah ve S. A. Patel, «Facial Expression Recognition: A Survey,» International Journal of Computer Science and Information Technologies (IJCSIT), cilt 5, no. 1, pp. 149-152, 2014.
• H. A. Rowley, S. Baluja ve T. Kanade, «Neural Network-Based Face Detection,» IEEE Transactions on Pattern Analysis and Machine Intelligence, cilt 20, no. 1, pp. 23-38, 1998.
• Y. Li, S. Gong, J. Sherrah ve H. Liddell, «Multi-view Face Detection Using Support Vector Machines and Eigenspace Modelling,» Fourth International Conference on knowledge-Based Intelligent Engineering Systems & Allied Technologies, Brighton, 2000.
• W. Widjojo ve K. C. Yow, «A Color and Feature-Based Approach to Human Face Detection,» Seventh lotcrnational Conference on Control, Automation, Rabotics And Vision (ICARCV'02), Singapore, 2002.
• J. Chen, S. Shan, P. Yang, S. Yan, X. Chen ve W. Gao, «Novel Face Detection Method Based on Gabor Features,» Advances in Biometric Person Authentication, Springer-Verlag Berlin Heidelberg, 2004, pp. 90-99.
• L. Lang ve W. Gu, «Study of Face Detection Algorithm for Real-time Face Detection System,» IEEE Second International Symposium on Electronic Commerce and Security (ISECS'09), Nanchang, 2009.
• S. K. Pal, U. Chourasia ve M. Ahirwar, «A Method for Face Detection based on Wavelet Transform and Optimised Feature Selection using Ant Colony Optimisation in Support Vector Machine,» International Journal of Innovative Research in Computer and Communication Engineering, cilt 1, no. 2, pp. 358-363, 2013.
• H. Bing, H. Xianfeng ve H. Ruizhen, «Research of Face Detection Based on AdaBoost and ASM,» The Open Cybernetics & Systemics Journal , cilt 8, no. 1, pp. 183-190, 2014.
• L. Shuang, «A Novel Face Detection Algorithm based on PCA and Adaboost,» IEEE Sixth International Conference on Measuring Technology and Mechatronics Automation (ICMTMA), Zhangjiajie, 2014.
• M.-H. Yang, D. J. Kriegman ve N. Ahuja, «Detecting Faces in Images: A Survey,» EEE Transactions on Pattern Analysis and Machine Intelligence, cilt 24, no. 1, pp. 34-58, 2002.
• P. Viola ve M. J. Jones, «Robust Real-Time Face Detection,» International Journal of Computer Vision, cilt 57, no. 2, pp. 137-154, 2004.
• G. Özmen, Kübil Bezier Eğrileri ile Yüz İfadesi Tanıma, Edirne: Trakya Üniversitesi Fen Bilimleri Enstitüsü Yüksek Lisans Tezi, 2012.
• W. Zhao ve R. Chellappa, Face Processing Advanced Modeling and Methods, Elsevier Academic Press , 2006.
• M. J. Black ve Y. Yacoob, «Tracking and Recognizing Rigid and Non-Rigid Facial Motions using Local Parametric Models of Image Motion,» IEEE Fifth International Conference on Computer Vision, Cambridge, 1995.
• J.-G. Ko, K.-N. Kim ve R. S. Ramakrishna, «Facial Feature Tracking for Eye-Head Controlled Human Computer Interface,» IEEE TENCON'99, Cheju, 1999.
• R. S. Feris, T. E. d. Campos ve R. M. C. Junior, «Detection and Tracking of Facial Features in Video Sequences,» Mexican International Conference on Artificial Intelligence (MICAI), Acapulco, 2000.
• F. Dornaika ve F. Davoine, «Head and Facial Animation Tracking using Appearance-Adaptive Models and Particle Filters,» IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'04), 2004.
• F. Dornaika ve J. Orozco, «Real Time 3D Face and Facial Feature Tracking,» Journal of Real-Time Image Processing, cilt 2, no. 1, p. 35–44, 2007.
• E. R. Gast, A Framework for Real-Time Face and Facial Feature Tracking using Optical Flow Pre-estimation and Template Tracking, Leiden: Leiden University LIACS Master Thesis, 2010.
• P. R. Borude, S. T. Gandhe, P. A. Dhulekar ve G. M. Phade, «Identification and Tracking of Facial Features,» Procedia Computer Science, cilt 49, no. 1, pp. 2-10, 2015.
• N. U. Khan, «A Comparative Analysis of Facial Expression Recognition Techniques,» 3rd IEEE International Advance Computing Conference (IACC), Ghaziabad, 2013.
• İ. O. Sığırcı, Facial Expression Recognition System, İstanbul: Fatih University The Graduate School of Sciences and Engineering Master Thesis, 2013.
• Y.-L. Tian, T. Kanade ve J. f. Cohn, «Facial Expression Analysis,» Handbook of Face Recognition, New York, Springer Science+Business Media, Inc., 2005, pp. 247-275.
• I. Kotsia ve I. Pitas, «Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines,» IEEE Transactions on Image Processing, cilt 16, no. 1, pp. 172-187, 2007.
• G. L. Libralon ve R. A. F. Romero, «Geometrical Facial Modeling for Emotion Recognition,» IEEE International Joint Conference on Neural Networks (IJCNN), Dallas, 2013.
• K. Lin, W. Cheng ve J. Li, «Facial Expression Recognition Based on Geometric Features and Geodesic Distance,» International Journal of Signal Processing, Image Processing and Pattern Recognition, cilt 7, no. 1, pp. 323-330, 2014.
• A. Saeed, A. Al-Hamadi, R. Niese ve M. Elzobi, «Frame-Based Facial Expression Recognition Using Geometrical Features,» Advances in Human-Computer Interaction, cilt 2014, no. Article ID: 408953, p. 13 pages.
• R. A. Patil, V. Sahula ve A. S. Mandal, «Facial Expression Recognition In Image Sequences Using Active Shape Model and SVM,» IEEE Fifth UKSim European Symposium on Computer Modeling and Simulation (EMS), Madrid, 2011.
• R. Shbib ve S. Zhou, «Facial Expression Analysis using Active Shape Model,» International Journal of Signal Processing, Image Processing and Pattern Recognition, cilt 8, no. 1, pp. 9-22, 2015.
• C.-K. Hsieh, Shang-Hong ve Y.-C. Chen, «An Optical Flow-Based Approach to Robust Face Recognition Under Expression Variations,» IEEE Transactions on Image Processing, cilt 19, no. 1, pp. 233-240, 2010.
• R. Niese, A. Al-Hamadi, A. Farag, H. Neumann ve B. Michaelis, «Facial Expression Recognition based on Geometric and Optical Flow Features in Colour Image Sequences,» IET Computer Vision, cilt 6, no. 2, p. 79–89, 2012.
• U. Prabhu ve K. Seshadri, «Facial Recognition Using Active Shape Models, Local Patches and Support Vector Machines,» [Çevrimiçi]. Available: http://www.contrib.andrew.cmu.edu/~kseshadr/ML_Paper.pdf. [Erişildi: 5 Kasım 2015].
• N. Rose, «Facial Expression Classification using Gabor and Log-Gabor Filters,» IEEE 7th International Conference on Automatic Face and Gesture Recognition (FGR’06), Southampton, 2006.
• J. Ou, X.-B. Bai, Y. Pei, L. Ma ve W. Liu, «Automatic Facial Expression Recognition Using Gabor Filter,» IEEE Second International Conference on Computer Modeling and Simulation (ICCMS), Sanya, 2010.
• T. Wu, M. S. Bartlett ve J. R. Movellan, «Facial Expression Recognition Using Gabor Motion Energy Filters,» IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), San Francisco, 2010 .
• J. Ruan, J. Yin, Q. Chen ve G. Chen, «Facial Expression Recognition Based on Gabor Wavelet Transform and Relevance Vector Machine,» Journal of Information & Computational Science, cilt 11, no. 1, pp. 295-302, 2014.
• C. Shan, S. Gong ve P. W. McOwan, «Facial Expression Recognition based on Local Binary Patterns: A Comprehensive Study,» Image and Vision Computing, cilt 27, p. 803–816, 2009.
• R. Hablani, N. Chaudhari ve S. Tanwani, «Recognition of Facial Expressions using Local Binary Patterns of Important Facial Parts,» International Journal of Image Processing (IJIP), cilt 7, no. 2, pp. 163-170, 2013.
• Y. M. Wais, Mood Detection According to Facial Expressions, Ankara: Çankaya University Graduate School of Natural and Applied Sciences, Master Thesis, 2015.
• M. Kılınç, Yüz Resimlerinden Yaş Bilgisinin Tespit Edilmesi, Gebze: Gebze Yüksek Teknoloji Enstitüsü Mühendislik ve Fen Bilimleri Enstitüsü, Yüksek Lisans Tezi, 2012.
• «"Gabor Dönüşümü için OpenCV Yüz Tanıma Kodu",» [Çevrimiçi]. Available: http://guoming.me/opencv-gabor. [Erişildi: 5 Kasım 2015].
• A. A. A. Youssif, «Automatic Facial Expression Recognition System Based on Geometric and Appearance Features,» Computer and Information Science, cilt 4, no. 2, pp. 115-124, 2011.
• A. K. K. Bermani, A. Z. Ghalwash ve A. A. A. Youssif, «Automatic Facial Expression Recognition Based on Hybrid Approach,» International Journal of Advanced Computer Science and Applications (IJACSA), cilt 3, no. 11, pp. 102-107, 2012.
• S. Ulukaya ve Ç. E. Erdem, «A Hybrid Facial Expression Recognition Method based on Neutral Face Shape Estimation,» IEEE 20th Signal Processing and Communications Applications Conference (SIU), Mugla, 2012.
• S. L. Happy ve A. Routray, «Robust Facial Expression Classification Using Shape and Appearance Features,» Eighth International Conference on Advances in Pattern Recognition (ICAPR), Kolkata, 2015.
• B. S. Akkoca, Durgun Görüntülerden Yüz İfadelerinin Tanınması, İstanbul: ˙İstanbul Teknik Üniversitesi Fen Bilimleri Enstitüsü, Yüksek Lisans Tezi, 2014.
• T. Kaur ve J. Kaur, «Facial Expression Recognition with PCA And LDA,» International Journal of Computer Science and Information Technologies (IJCSIT), cilt 5, no. 6, pp. 6996-6998, 2014.
• M. Lyons ve S. Akamatsu, «Coding Facial Expressions with Gabor Wavelets,» IEEE Third IEEE International Conference on Automatic Face and Gesture Recognition, Nara, 1998.
• T. Kanade, J. F. Cohn ve Y. Tian, «Comprehensive Database for Facial Expression Analysis,» IEEE Fourth International Conference on Automatic Face and Gesture Recognition, Grenoble, 2000.
• P. Lucey, J. F. Cohn, T. Kanade, J. Saragih ve Z. Ambadar, «The Extended Cohn-Kanade Dataset (CK+): A Complete Dataset for Action Unit,» IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) , San Francisco, 2010.
• N. Aifanti, C. Papachristou ve A. Delopoulos, «The MUG Facial Expression Database,» IEEE 11th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS), Desenzano del Garda, 2010.