Yüz Duygu Tanıma Veri Setleri ve Metodolojilerinin Kapsamlı İncelemesi

Öz

Otomatik yüz duygu tanıma, diğer adıyla yüz duygu tanıma (FER), bilgisayar görme ve yapay zeka alanlarında devam eden çalışmaların temel bir özelliğidir. FER'in temel amacı, mutluluk, üzüntü, öfke, korku, tiksinti, şaşkınlık ve tarafsızlık gibi duyguları tanımaktır. Bu inceleme makalesi, 11 popüler FER veri seti, yani CK+, FER-2013, JAFFE, SAVEE, AffectNet, KDEF, RAF-DB, RAVDESS, RFD, Oulu-Casia NIR & VIS ve SFEW 2.0 arasında kapsamlı bir karşılaştırma sunmaktadır. Karşılaştırma, veri kümesi boyutları, duygu sınıfları, veri türleri, katılımcıların yaş aralıkları ve duygu dağılımı gibi bir dizi ölçüt üzerinden gerçekleştirilmiştir. Çalışma, bu tür veri kümelerinde klasik yöntemlerin (ör. SVM, HOG) doğruluğunu, yeni derin öğrenme modellerinin (ör. CNN, LSTM) doğruluğuyla karşılaştırmaktadır. Laboratuvar ortamında mükemmel doğruluk elde edilmiştir (ör. CK+ üzerinde SCNN ile %99,68), ancak gerçek dünyada zorluklar devam etmektedir (ör. SFEW 2.0 üzerinde CNN ile %61,29). Çalışma, bunun temel zorluklarını, yani duyguların çarpıklığı, demografik önyargılar ve gerçek dünyadaki heterojenliği açıklamaktadır. Çalışma, mevcut veri kümelerinin ve yaklaşımların eksikliklerinden bahsetmekte ve veri kümelerinin çeşitliliğinin artırılması, çok modlu füzyon ve kültürel olarak uyarlanabilir model oluşturma açısından gelecekteki araştırma yönlerini önermektedir. Alan incelemesi, FER'in bilimsel temelini ve uygulama kapsamını güçlendirmekte ve araştırmacılar ve uygulayıcılar için kapsamlı bir kılavuz sunmaya çalışmaktadır.

Anahtar Kelimeler

• AffectNet
• evrişimli sinir ağları (CNN)
• derin öğrenme
• duygu tanıma
• yüz duygu tanıma (FER)
• insan-bilgisayar etkileşimi
• JAFFE
• makine öğrenimi
• SFEW
• görsel veri kümeleri

Comprehensive Review of Facial Emotion Recognition Datasets and Methodologies

Abstract

Automatic face emotion recognition, which is otherwise referred to as facial emotion recognition (FER), is a basic feature of ongoing work in computer vision and AI. The principal aim of FER is the emotion recognition involving happiness, sadness, anger, fear, disgust, surprise and neutrality. This review paper presents a comprehensive comparison between 11 popular FER datasets, i.e., CK+, FER-2013, JAFFE, SAVEE, AffectNet, KDEF, RAF-DB, RAVDESS, RFD, Oulu-Casia NIR & VIS and SFEW 2.0. The comparison is conducted across a range of metrics including dataset sizes, emotion classes, data types, age ranges of participants, and emotion distribution. The study contrasts the accuracy of classical methods (e.g., SVM, HOG) against newer deep learning models (e.g., CNN, LSTM) on such datasets, with excellent lab accuracy (e.g., 99.68% with SCNN on CK+) and continued difficulty in the real world (e.g., 61.29% with CNN on SFEW 2.0). The study explains the key challenges thereof, i.e., the skewness of emotions, demographic biases and heterogeneity in the real world. The study mentions shortcomings of existing datasets and approaches, and suggests future research directions in terms of diversity addition of datasets, multimodal fusion, and culturally adaptive model building. Field review strengthens the scientific basis and scope of application of FER and attempts to provide a comprehensive guidebook for researchers and practitioners.

Keywords

• AffectNet
• convolutional neural networks (CNN)
• deep learning
• emotion recognition
• facial emotion recognition (FER)
• human–computer interaction
• JAFFE
• machine learning
• SFEW
• visual datasets

Kaynakça

1. [1] Fig. 1: The six universal emotional expressions. ResearchGate 2025 Apr 22. [Online]. Available: https://www.researchgate.net/figure/The-six-universal-emotional-expressions-26_fig1_261250781.
2. [2] Gautam C, Seeja KR. Facial emotion recognition using Handcrafted features and CNN. Procedia Comput Sci 2023;218:1295-1303.
3. [3] Zou W, Zhang D, Lee D-J. A new multi-feature fusion based convolutional neural network for facial expression recognition. Appl Intell 2022;52(3):2918–2929.
4. [4] Yan L, Sheng M, Wang C, et al. Hybrid neural networks based facial expression recognition for smart city. Multimed Tools Appl 2022;81(1):319–342.
5. [5] Singh R, Saurav S, Kumar T, et al. Facial expression recognition in videos using hybrid CNN & ConvLSTM. Int J Inf Technol 2023;15(4):1819–1830.
6. [6] Li D, Liu J, Yang Z, et al. Speech emotion recognition using recurrent neural networks with directional self-attention. Expert Syst Appl 2021;173:114683.
7. [7] Du L, Hu H. Weighted Patch-based Manifold Regularization Dictionary Pair Learning model for facial expression recognition using Iterative Optimization Classification Strategy. Comput Vis Image Underst 2019;186:13–24.
8. [8] Ouellet S. Real-time emotion recognition for gaming using deep convolutional network features. arXiv 2014.

9. [9] Li J, Nie J, Guo D, et al. Emotion Separation and Recognition from a Facial Expression by Generating the Poker Face with Vision Transformers. 2022.
10. [10] Liu P, Han S, Meng Z, Tong Y. Facial Expression Recognition via a Boosted Deep Belief Network. 2014 IEEE Conf Comput Vis Pattern Recognit, Columbus, OH, USA, 2014;p. 1805–1812.
11. [11] Liu M, Li S, Shan S, Chen X. AU-inspired Deep Networks for Facial Expression Feature Learning. Neurocomputing 2015;159:126–136.
12. [12] Dalrymple KA, Gomez J, Duchaine B. The Dartmouth Database of Children’s Faces: Acquisition and Validation of a New Face Stimulus Set. PLoS ONE 2013;8(11):e79131.
13. [13] Ding H, Zhou SK, Chellappa R. FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognition. 2017 12th IEEE Int Conf Autom Face Gesture Recognit (FG 2017), Washington, DC, USA, 2017;p. 118–126.
14. [14] Cai J, Meng Z, Khan AS, et al. Island Loss for Learning Discriminative Features in Facial Expression Recognition. 2018 13th IEEE Int Conf Autom Face Gesture Recognit (FG 2018), Xi’an, China, 2018;p. 302–309.
15. [15] Liu X, Kumar BVKV, You J, et al. Adaptive Deep Metric Learning for Identity-Aware Facial Expression Recognition. 2017 IEEE Conf Comput Vis Pattern Recognit Workshops (CVPRW), Honolulu, HI, USA, 2017;p. 522–531.
16. [16] Yang H, Ciftci U, Yin L. Facial Expression Recognition by De-expression Residue Learning. 2018 IEEE/CVF Conf Comput Vis Pattern Recognit, Salt Lake City, UT, USA, 2018;p. 2168–2177.
17. [17] Lucey P, Cohn JF, Kanade T, et al. The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression. 2010 IEEE Comput Soc Conf Comput Vis Pattern Recognit - Workshops, San Francisco, CA, USA, 2010;p. 94–101.
18. [18] Khorrami P, Paine TL, Huang TS. Do Deep Neural Networks Learn Facial Action Units When Doing Expression Recognition? arXiv 2015.
19. [19] LoBue V, Thrasher C. The Child Affective Facial Expression (CAFE) set: validity and reliability from untrained adults. Front Psychol 2015;5.
20. [20] Nojavanasghari B, Baltrušaitis T, Hughes CE, Morency L-P. EmoReact: a multimodal approach and dataset for recognizing emotional responses in children. Proc 18th ACM Int Conf Multimodal Interact, Tokyo, Japan, 2016;p. 137–144.
21. [21] Zhang Z, Luo P, Loy CC, Tang X. From Facial Expression Recognition to Interpersonal Relation Prediction. arXiv 2016.
22. [22] Meng Z, Liu P, Cai J, et al. Identity-Aware Convolutional Neural Network for Facial Expression Recognition. 2017 12th IEEE Int Conf Autom Face Gesture Recognit (FG 2017), Washington, DC, USA, 2017;p. 558–565.
23. [23] Zeng N, Zhang H, Song B, et al. Facial expression recognition via learning deep sparse autoencoders. Neurocomputing 2018;273:643–649.
24. [24] Jaiswal A, Raju AK, Deb S. Facial Emotion Detection Using Deep Learning. 2020 Int Conf Emerg Technol (INCET), Belgaum, India, 2020;p. 1–5.
25. [25] Tang Y. Deep Learning using Linear Support Vector Machines. arXiv 2013.
26. [26] Devries T, Biswaranjan K, Taylor GW. Multi-task Learning of Facial Landmarks and Expression. 2014 Can Conf Comput Robot Vis, Montreal, QC, Canada, 2014;p. 98–103.
27. [27] Zhang Z, Luo P, Loy CC, Tang X. Learning Social Relation Traits from Face Images. arXiv 2015.
28. [28] Guo Y, Tao D, Yu J, et al. Deep Neural Networks with Relativity Learning for facial expression recognition. 2016 IEEE Int Conf Multimedia Expo Workshops (ICMEW), Seattle, WA, USA, 2016;p. 1–6.
29. [29] Kim B-K, Dong S-Y, Roh J, et al. Fusing Aligned and Non-aligned Face Information for Automatic Affect Recognition in the Wild: A Deep Learning Approach. 2016 IEEE Conf Comput Vis Pattern Recognit Workshops (CVPRW), Las Vegas, NV, USA, 2016;p. 1499–1508.
30. [30] Pramerdorfer C, Kampel M. Facial Expression Recognition using Convolutional Neural Networks: State of the Art. arXiv 2016.
31. [31] Dufourq E. A Survey on Factors Affecting Facial Expression Recognition based on Convolutional Neural Networks. Conf South Afr Inst Comput Sci Inf Technol 2020, Cape Town, South Africa, 2020;p. 168–179.
32. [32] Adjabi I, Ouahabi A, Benzaoui A, Taleb-Ahmed A. Past, Present, and Future of Face Recognition: A Review. Electronics 2020;9(8):1188.
33. [33] Bayoudh K, Knani R, Hamdaoui F, Mtibaa A. A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets. Vis Comput 2022;38(8):2939–2970.
34. [34] Khan RA, Crenn A, Meyer A, Bouakaz S. A novel database of children’s spontaneous facial expressions (LIRIS-CSE). Image Vis Comput 2019;83–84:61–69.
35. [35] A. Uçar, Y. Demir, and C. Güzeliş, "A new facial expression recognition based on curvelet transform and online sequential extreme learning machine initialized with spherical clustering," Neural Comput. Appl., vol. 27, no. 1, pp. 131–142, 2016.
36. [36] Egger HL, et al. The NIMH Child Emotional Faces Picture Set (NIMH‐ChEFS): a new set of children’s facial emotion stimuli. Int J Methods Psychiatr Res 2011;20(3):145–156.
37. [37] Jaiswal A, Raju AK, Deb S. Facial Emotion Detection Using Deep Learning. 2020 Int Conf Emerg Technol (INCET), Belgaum, India, 2020;p. 1–5.
38. [38] Gautam C, Seeja KR. Facial emotion recognition using Handcrafted features and CNN. Procedia Comput Sci 2023;218:1295–1303.
39. [39] Mohana M, Subashini P. Emotion Recognition using Deep Stacked Autoencoder with Softmax Classifier. 2023 Third Int Conf Artif Intell Smart Energy (ICAIS), Coimbatore, India, 2023;p. 864–872.
40. [40] Akhand MAH, Roy S, Siddique N, et al. Facial Emotion Recognition Using Transfer Learning in the Deep CNN. Electronics 2021;10(9):1036.
41. [41] Hamester D, Barros P, Wermter S. Face expression recognition with a 2-channel Convolutional Neural Network. 2015 Int J Conf Neural Networks (IJCNN), Killarney, Ireland, 2015;p. 1–8.
42. [42] Levi G, Hassner T. Emotion Recognition in the Wild via Convolutional Neural Networks and Mapped Binary Patterns. Proc 2015 ACM Int Conf Multimodal Interact, Seattle, WA, USA, 2015;p. 503–510.
43. [43] Ng H-W, Nguyen VD, Vonikakis V, Winkler S. Deep Learning for Emotion Recognition on Small Datasets using Transfer Learning. Proc 2015 ACM Int Conf Multimodal Interact, Seattle, WA, USA, 2015;p. 443–449.
44. [44] Li S, Deng W, Du J. Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild. 2017 IEEE Conf Comput Vis Pattern Recognit (CVPR), Honolulu, HI, USA, 2017;p. 2584–2593.
45. [45] Kim B-K, Roh J, Dong S-Y, Lee S-Y. Hierarchical committee of deep convolutional neural networks for robust facial expression recognition. J Multimodal User Interfaces 2016;10(2):173–189.
46. [46] Yu Z, Zhang C. Image based Static Facial Expression Recognition with Multiple Deep Network Learning. Proc 2015 ACM Int Conf Multimodal Interact, Seattle, WA, USA, 2015;p. 435–442.
47. [47] Singh R, Saurav S, Kumar T, et al. Facial expression recognition in videos using hybrid CNN & ConvLSTM. Int J Inf Technol 2023;15(4):1819–1830.
48. [48] Salunke VV, Patil CG. A New Approach for Automatic Face Emotion Recognition and Classification Based on Deep Networks. 2017 Int Conf Comput, Commun, Control Autom (ICCUBEA), Pune, India, 2017;p. 1–5.
49. [49] Mahmud F, Islam B, Hossain A, Goala PB. Facial Region Segmentation Based Emotion Recognition Using K-Nearest Neighbors. 2018 Int Conf Innov Eng Technol (ICIET), Dhaka, Bangladesh, 2018;p. 1–5.
50. [50] Rasamoelina AD, Adjailia F, Sincak P. Deep Convolutional Neural Network for Robust Facial Emotion Recognition. 2019 IEEE Int Symp INnovations Intell SysTems Appl (INISTA), Sofia, Bulgaria, 2019;p. 1–6.
51. [51] Yaddaden Y, Adda M, Bouzouane A, et al. Facial Sub-regions for Automatic Emotion Recognition using Local Binary Patterns. 2018 Int Conf Signal, Image, Vis Appl (SIVA), Guelma, Algeria, 2018;p. 1–6.
52. [52] Yaddaden Y, Adda M, Bouzouane A. Facial Expression Recognition using Locally Linear Embedding with LBP and HOG Descriptors. 2020 2nd Int Workshop Hum Centric Smart Environ Health Well-being (IHSH), Boumerdes, Algeria, 2021;p. 221–226.
53. [53] Shokrani S, Moallem P, Habibi M. Facial emotion recognition method based on Pyramid Histogram of Oriented Gradient over three direction of head. 2014 4th Int Conf Comput Knowledge Eng (ICCKE), Mashhad, Iran, 2014;p. 215–220.
54. [54] Mavani V, Raman S, Miyapuram KP. Facial Expression Recognition Using Visual Saliency and Deep Learning. 2017 IEEE Int Conf Comput Vis Workshops (ICCVW), Venice, Italy, 2017;p. 2783–2788.
55. [55] Yaddaden Y, Adda M, Bouzouane A, et al. One-Class and Bi-Class SVM Classifier Comparison for Automatic Facial Expression Recognition. 2018 Int Conf Appl Smart Syst (ICASS), Medea, Algeria, 2018;p. 1–6.
56. [56] Jiang C-S, Liu Z-T, Wu M, et al. Efficient Facial Expression Recognition With Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction. IEEE Trans Ind Inform 2023;19(9):9943–9952.
57. [57] Yaddaden Y, Adda M, Bouzouane A, et al. Hybrid-Based Facial Expression Recognition Approach for Human-Computer Interaction. 2018 IEEE 20th Int Workshop Multimedia Signal Process (MMSP), Vancouver, BC, Canada, 2018;p. 1–6.
58. [58] Jiang C-S, Liu Z-T, She J. Facial Expression Recognition Based on Zero-Addition Pretext Training and Feature Conjunction-Selection Network in Human–Robot Interaction. IEEE Sens J 2023;23(18):22093–22102.
59. [59] Zavarez MV, Berriel RF, Oliveira-Santos T. Cross-Database Facial Expression Recognition Based on Fine-Tuned Deep Convolutional Network. 2017 30th SIBGRAPI Conf Graphics, Patterns Images (SIBGRAPI), Niteroi, Brazil, 2017;p. 405–412.
60. [60] Shin Y, Bum J, Son C-H, Choo H. BCGAN: Facial Expression Synthesis by Bottleneck-Layered Conditional Generative Adversarial Networks. 2021 15th Int Conf Ubiquitous Inf Manag Commun (IMCOM), Seoul, South Korea, 2021;p. 1–4.
61. [61] Shams WK, Wahab A, Qadwi U, et al. Emotion Recognition Model Using Source-Temporal Features and Fuzzy. Neural Inf Process 2013;8226:577–584.
62. [62] González-Hernández F, Zatarain-Cabada R, Barrón-Estrada ML, Rodríguez-Rangel H. Recognition of learning-centered emotions using a convolutional neural network. J Intell Fuzzy Syst 2018;34(5):3325–3336.
63. [63] Yaddaden Y. An efficient facial expression recognition system with appearance-based fused descriptors. Intell Syst Appl 2023;17:200166.
64. [64] Ozdemir MA, Elagoz B, Alaybeyoglu A, et al. Real Time Emotion Recognition from Facial Expressions Using CNN Architecture. 2019 Med Technol Congr (TIPTEKNO), Izmir, Türkiye, 2019;p. 1–4.
65. [65] Gupta R, Sharma SJ. Emotion Recognition from Facial Expressions Using Deep Learning Model. 2024 IEEE Int Conf Intell Signal Process Eff Commun Technol (INSPECT), Gwalior, India, 2024;p. 1–6.
66. [66] Timothy J, Prasetio R, Tandi D, et al. Facial Emotion Recognition with InceptionV3 CNN Architecture. 2023 Int Conf Inform, Multimedia, Cyber Inf Syst (ICIMCIS), Jakarta Selatan, Indonesia, 2023;p. 1–5.
67. [67] Devi HB, Sandhan T. Regularized Fusion of CNN with Transfer Learning for Facial Emotion Recognition. 2023 14th Int Conf Comput Commun Netw Technol (ICCCNT), Delhi, India, 2023;p. 1–6.
68. [68] Ghadami O, Rezvanian A, Shakuri S. Scalable Real-time Emotion Recognition using EfficientNetV2 and Resolution Scaling. 2024 10th Int Conf Web Res (ICWR), Tehran, Iran, 2024;p. 7–12.
69. [69] Yanç I, İpek A, Kayaarma SY. Facial Emotion Recognition for Imitation in Human-Robot Interaction. 2024 9th Int Conf Comput Sci Eng (UBMK), Antalya, Türkiye, 2024;p. 654–659.
70. [70] Palaniswamy S, Suchitra. A Robust Pose & Illumination Invariant Emotion Recognition from Facial Images using Deep Learning for Human-Machine Interface. 2019 4th Int Conf Comput Syst Inf Technol Sustain Solution (CSITSS), Bengaluru, India, 2019;p. 1–6.
71. [71] Punuri SB, Kuanar SK, Mishra TK, et al. Decoding Human Facial Emotions: A Ranking Approach Using Explainable AI. IEEE Access 2024;12:186229–186245.
72. [72] Kayaoğlu B, Toktaş T, Kırbız S. CNN-Based Emotion Recognition using Data Augmentation and Preprocessing Methods. 2023 Innov Intell Syst Appl Conf (ASYU), Sivas, Türkiye, 2023;p. 1–4.
73. [73] Nafi’An MHI, Bachtiar FA, Setiawan BD. Modified MobileNetV2 with Convolutional Block Attention Module for Facial Expression Recognition. 2023 Int Workshop Artif Intell Image Process (IWAIIP), Yogyakarta, Indonesia, 2023;p. 22–27.
74. [74] Azizi FN, Kurniawardhani A, Paputungan IV. Facial Expression Image based Emotion Detection using Convolutional Neural Network. 2022 IEEE 20th Student Conf Res Dev (SCOReD), Bangi, Malaysia, 2022;p. 157–162.
75. [75] Sahoo GK, Das SK, Singh P. Deep Learning-Based Facial Emotion Recognition for Driver Healthcare. 2022 National Conf Commun (NCC), Mumbai, India, 2022;p. 154–159.
76. [76] Waref D, Salem M. Split Federated Learning for Emotion Detection. 2022 4th Novel Intell Leading Emerg Sci Conf (NILES), Giza, Egypt, 2022;p. 112–115.
77. [77] Webb N, Ruiz-Garcia A, Elshaw M, Palade V. Emotion Recognition from Face Images in an Unconstrained Environment for usage on Social Robots. 2020 Int J Conf Neural Networks (IJCNN), Glasgow, UK, 2020;p. 1–8.
78. [78] Torcate AS, Santana MAD, Santos WPD. Emotion Recognition to Support Personalized Therapy: An Approach Based on a Hybrid Architecture of CNN and Random Forest. 2023 IEEE Lat Am Conf Comput Intell (LA-CCI), Recife-Pe, Brazil, 2023;p. 1–6.
79. [79] Sadiyah R, Fariza A, Kusumaningtyas EM. Emotion Recognition Based on Facial Expression by Exploring Batch Normalization Convolutional Neural Network. 2022 Int Electron Symp (IES), Surabaya, Indonesia, 2022;p. 511–516.
80. [80] Gursesli MC, Lombardi S, Duradoni M, et al. Facial Emotion Recognition (FER) Through Custom Lightweight CNN Model: Performance Evaluation in Public Datasets. IEEE Access 2024;12:45543–45559.
81. [81] Tomar A, Gera S, Kumar S, Pant B. HHFER: A Hybrid Framework for Human Facial Expression Recognition. 2021 Int Conf Data Anal Bus Ind (ICDABI), Sakheer, Bahrain, 2021;p. 537–541.
82. [82] Win KC, Akhtar Z, Mohan CK. Robust Facial Emotion Recognition System via De-Pooling Feature Enhancement and Weighted Exponential Moving Average. TENCON 2024 - 2024 IEEE Reg 10 Conf (TENCON), Singapore, 2024;p. 116–119.
83. [83] Casas-Ortiz A, Echeverria J, Jimenez-Tellez N, Santos OC. Exploring the Impact of Partial Occlusion on Emotion Classification From Facial Expressions: A Comparative Study of XR Headsets and Face Masks. IEEE Access 2024;12:44613–44627.
84. [84] Srinivas P, Khamar SN, Borusu N, et al. Identification of Facial Emotions in Hitech Modern Era. 2023 2nd Int Conf Edge Comput Appl (ICECAA), Namakkal, India, 2023;p. 1202–1208.
85. [85] Zhang Y, Shi Y, Sun Z. Multi-scale fusion-based expression recognition algorithm. 2023 IEEE 6th Inf Technol, Netw, Electron Autom Control Conf (ITNEC), Chongqing, China, 2023;p. 955–958.
86. [86] Shang Y, Yan F, Liu Y, et al. Facial Emotion Recognition Combining Auxiliary Classifiers and Multiscale CBAM Attention Mechanisms. IEEE Access 2024;12:57356–57365.
87. [87] Khoeun R, Yookwan W, Chophuk P, et al. Emotion Recognition of Partial Face Using Star-Like Particle Polygon Estimation. IEEE Access 2023;11:87558–87570.
88. [88] Tamilselvi M. Human Facial Emotion Recognition Using Graphical Cascaded Convolutional Neural Network. 2024 Ninth Int Conf Sci Technol Eng Math (ICONSTEM), Chennai, India, 2024;p. 1–8.
89. [89] Lv Z. Facial expression recognition method based on dual-branch fusion network with noisy labels. 2024 IEEE 7th Adv Inf Technol, Electron Autom Control Conf (IAEAC), Chongqing, China, 2024;p. 1608–1612.
90. [90] Chinta U, Atyabi A. Facial Expression Recognition Using Robust Attention-Based CNN. 2023 Int Conf Mach Learn Appl (ICMLA), Jacksonville, FL, USA, 2023;p. 858–863.
91. [91] Roy AK, Kathania HK, Sharma A, et al. ResEmoteNet: Bridging Accuracy and Loss Reduction in Facial Emotion Recognition. IEEE Signal Process Lett 2025;32:491–495.
92. [92] Bai Y, Chen L, Li M, et al. Partially Occluded Face Expression Recognition with CBAM-Based Residual Network for Teaching Scene. 2023 China Autom Congr (CAC), Chongqing, China, 2023;p. 6052–6057.
93. [93] Aly M, Ghallab A, Fathi IS. Enhancing Facial Expression Recognition System in Online Learning Context Using Efficient Deep Learning Model. IEEE Access 2023;11:121419–121433.
94. [94] Gao J. Research on face expression recognition with subdivision structure based on depth learning. 2023 IEEE 3rd Int Conf Data Sci Comput Appl (ICDSCA), Dalian, China, 2023;p. 799–803.
95. [95] Yu B, Zhang G, Xu W, Wei M. Face Expression Recognition Based on Lightweight Fused Attention Mechanism. 2022 4th Int Conf Robot Comput Vis (ICRCV), Wuhan, China, 2022;p. 85–89.
96. [96] Choi JY, Lee B. Combining Deep Convolutional Neural Networks With Stochastic Ensemble Weight Optimization for Facial Expression Recognition in the Wild. IEEE Trans Multimed 2023;25:100–111.
97. [97] Liu H, Deng L, Liu T, et al. ACAForms: Learning Adaptive Context-Aware Feature for Facial Expression Recognition in Human-Robot Interaction. 2024 9th Int Conf Robot Autom Eng (ICRAE), Singapore, 2024;p. 175–180.
98. [98] Gou R, et al. Facial Expression Recognition Based on TripletLoss and Attention Mechanism. 2023 28th Int Conf Autom Comput (ICAC), Birmingham, UK, 2023;p. 1–6.
99. [99] Liu H, Cai H, Lin Q, et al. Adaptive Multilayer Perceptual Attention Network for Facial Expression Recognition. IEEE Trans Circuits Syst Video Technol 2022;32(9):6253–6266.
100. [100] Kamada S, Ichimura T. Multimodal Adaptive Structural Deep Belief Network for Emotion Recognition on RAVDESS Dataset. 2024 16th IIAI Int Congr Adv Appl Inf (IIAI-AAI), Takamatsu, Japan, 2024;p. 292–298.
101. [101] Kurniawardhani A, Paputungan IV, Respati RG. Improving Voice-Based Emotion Recognition Using Convolutional Neural Networks: Integrating RAVDESS and JL Corpus Datasets. 2024 11th Int Conf Soft Comput Mach Intell (ISCMI), Melbourne, Australia, 2024;p. 252–256.
102. [102] Gavali MP, Verma A. Automatic Recognition of Emotions in Speech With Large Self-Supervised Learning Transformer Models. 2023 IEEE Int Conf Artif Intell, Blockch, IoT (AIBThings), Mount Pleasant, MI, USA, 2023;p. 1–7.
103. [103] Krisnandi D, Pardede HF. Exploration of Human Emotion Recognition in Speech Data Based on Convolutional Neural Network with Mel and Delta Scale Features. 2023 Int Conf Inf Technol Res Innov (ICITRI), Jakarta, Indonesia, 2023;p. 99–102.
104. [104] Ruijie H, Yuetian W. Leveraging Librosa for Speech Emotion Recognition: Techniques and Applications. 2024 IEEE 4th Int Conf Data Sci Comput Appl (ICDSCA), Dalian, China, 2024;p. 245–249.
105. [105] Cioroiu G, Radoi A. Emotion Recognition from Contextualized Speech Representations using Fine-tuned Transformers. 2024 15th Int Conf Commun (COMM), Bucharest, Romania, 2024;p. 1–5.
106. [106] Shinde AS, et al. ML Based Speech Emotion Recognition Framework For Music Therapy Suggestion System. 2022 6th Int Conf Comput, Commun, Control Autom (ICCUBEA, Pune, India, 2022;p. 1–5.
107. [107] Abakarim F, Abenaou A. Speech Emotion Recognition System Using Discrete Wavelet Transform and Support Vector Machine. 2024 Int Conf Intell Syst Comput Vis (ISCV), Fez, Morocco, 2024;p. 1–5.
108. [108] Singh V, Prasad S. Speech Emotion Recognition using Fully Convolutional Network and Augmented RAVDESS Dataset. 2023 Int Conf Adv Comput Technol Appl (ICACTA), Mumbai, India, 2023;p. 1–7.
109. [109] Zhang L-M, Ng GW, Leau Y-B, Yan H. A Parallel-Model Speech Emotion Recognition Network Based on Feature Clustering. IEEE Access 2023;11:71224–71234.
110. [110] Adi DP, Yuniarno EM, Wulandari DP. Emotion Recognition from Video Frame Sequence using Face Mesh and Pre-Trained Models of Convolutional Neural Network. 2023 Int Semin Intell Technol Appl (ISITIA), Surabaya, Indonesia, 2023;p. 353–358.
111. [111] K. K. V, S. L. R, J. J. S. A Study on Human Emotion Recognition Techniques. 2023 Int Conf Innov Eng Technol (ICIET), Muvattupuzha, India, 2023;p. 1–6.
112. [112] Joshi D, Dhok A, Khandelwal A, et al. Real Time Emotion Analysis (RTEA). 2021 Int Conf Artif Intell Mach Vis (AIMV), Gandhinagar, India, 2021;p. 1–5.
113. [113] Ding SY, Tang TB, Lu C-K. Investigation on Light-Weight Deep Learning Model for Emotion Recognition Using Facial Expressions. TENCON 2023 - 2023 IEEE Reg 10 Conf (TENCON), Chiang Mai, Thailand, 2023;p. 204–208.
114. [114] Garaiman FE, Radoi A. Multimodal Emotion Recognition System based on X-Vector Embeddings and Convolutional Neural Networks. 2024 15th Int Conf Commun (COMM), Bucharest, Romania, 2024;p. 1–6.
115. [115] Vardhan JV, Chakravarti YK, Chand AJ. Deep Learning-Based Emotion Recognition by Fusion of Facial Expressions and Speech Features. 2024 2nd World Conf Commun Comput (WCONF), Raipur, India, 2024;p. 1–6.
116. [116] Mocanu B, Tapu R. Facial Emotion Recognition using Video Visual Transformer and Attention Dropping. 2023 Int Symp Signals, Circuits Syst (ISSCS), Iasi, Romania, 2023;p. 1–4.
117. [117] Maham S, Tariq A, Tayyaba B, et al. MMER: Mid-Level Fusion Strategy for Multimodal Emotion Recognition using Speech and Video Data. 2024 18th Int Conf Open Source Syst Technol (ICOSST), Lahore, Pakistan, 2024;p. 1–6.
118. [118] Song Y, Cai Y, Tan L. Video-Audio Emotion Recognition Based on Feature Fusion Deep Learning Method. 2021 IEEE Int Midwest Symp Circuits Syst (MWSCAS), Lansing, MI, USA, 2021;p. 611–616.
119. [119] Gavade PA, Bhat V, Pujari J. Facial Expression Recognition in Videos by learning Spatio-Temporal Features with Deep Neural Networks. 2021 Sixth Int Conf Image Inf Process (ICIIP), Shimla, India, 2021;p. 359–363.
120. [120] Arabian H, Wagner-Hartl V, Chase JG, Möller K. Facial Emotion Recognition Focused on Descriptive Region Segmentation. 2021 43rd Annu Int Conf IEEE Eng Med Biol Soc (EMBC), Mexico, 2021;p. 3415–3418.
121. [121] Jin X, Xiao Y. An Enhanced Expression Recognition Based on Sparse Representation Using Data Normalization. 2021 8th Int Conf Comput Sci/Intell Appl Inf (CSII), Zhuhai, China, 2021;p. 43–48.
122. [122] Nagrani HR, Patel DK, Shah KD, et al. Facial Expression Recognition using Convolutional Neural Network through Region-based Patch Generation: Harnessing Subtle Facial Cues. 2023 Int Conf Mach Learn Appl (ICMLA), Jacksonville, FL, USA, 2023;p. 973–979.
123. [123] Liu J, Feng Y, Wang H. Facial Expression Recognition Using Pose-Guided Face Alignment and Discriminative Features Based on Deep Learning. IEEE Access 2021;9:69267–69277.
124. [124] Chandrasekharan S, Arabian H, Moeller K. Decoding Emotions: How Temporal Modelling Enhances Recognition Accuracy. IFAC-Pap 2024;58(24):439–442.
125. [125] Zhang Y. DCAlexNet: Deep coupled AlexNet for micro facial expression recognition based on double face images. Comput Biol Med 2025;189:109986.
126. [126] Hsu C-F, Mudiyanselage SPK, Agustina R, Lin M-F. Basic emotion detection accuracy using artificial intelligence approaches in facial emotions recognition system: A systematic review. Appl Soft Comput 2025;172:112867.
127. [127] Arabian H, Wagner-Hartl V, Moeller K. Image Pre-processing Effects on Attention Modules in Facial Emotion Recognition. IFAC-Pap 2024;58(24):433–438.
128. [128] Arabian H, Wagner-Hartl V, Chase JG, Möller K. Image Pre-processing Significance on Regions of Impact in a Trained Network for Facial Emotion Recognition. IFAC-Pap 2021;54(15):299–303.
129. [129] Alphonse S, Verma H. Facial Expression Recognition with High Response-Based Local Directional Pattern (HR-LDP) Network. Comput Mater Contin 2024;78(2):2067–2086.
130. [130] Jabbooree AI, Khanli LM, Salehpour P, et al. A novel facial expression recognition algorithm using geometry –skeleton in fusion based on deep CNN. Image Vis Comput 2023;134:104677.
131. [131] Chen J, Shi J, Xu R. Dual subspace manifold learning based on GCN for intensity-invariant facial expression recognition. Pattern Recognit 2024;148:110157.
132. [132] Alphonse AS, Dharma D. A novel Monogenic Directional Pattern (MDP) and pseudo-Voigt kernel for facilitating the identification of facial emotions. J Vis Commun Image Represent 2017;49:459–470.
133. [133] Zhang H, Huang B, Tian G. Facial expression recognition based on deep convolution long short-term memory networks of double-channel weighted mixture. Pattern Recognit Lett 2020;131:128–134.
134. [134] Al-Sumaidaee SAM, Abdullah MAM, Al-Nima RRO, et al. Spatio-temporal modelling with multi-gradient features and elongated quinary pattern descriptor for dynamic facial expression recognition. Pattern Recognit 2023;142:109647.
135. [135] Alphonse AS, Dharma D. Enhanced Gabor (E-Gabor), Hypersphere-based normalization and Pearson General Kernel-based discriminant analysis for dimension reduction and classification of facial emotions. Expert Syst Appl 2017;90:127–145.
136. [136] Sultana A, Dey SK, Rahman MA. Facial emotion recognition based on deep transfer learning approach. Multimed Tools Appl 2023;82(28):44175–44189.