Comprehensive Analysis of Emotion Recognition Algorithms in Convolutional Neural Networks with Hyperparameters

Year 2024, Volume: 12 Issue: 2, 159 - 168, 30.12.2024

Süha Gökalp , İlhan Aydın

https://doi.org/10.18586/msufbd.1480255

Abstract

Emotions emerge from expressions, which are phenomena of human behavior that can help us gain insight into human nature and sometimes even feel what they are experiencing. People express their behavioral characteristics through emotions. In other words, expressions are emotional expressions of human behavioral characteristics. The face carries these expressions to the most fundamental point in human-human interaction. Thanks to these interactions, feedback is received from people, but it must be studied so that machines can perceive this interaction. Achieving higher performance on the FERG dataset, which allows focusing only on emotions with cartoonish human appearances, will also increase the performance rate in real images. For this purpose, this study focused on faster and more accurate prediction of emotions on the human face with the help of artificial intelligence. In this context, 7 emotion expressions, Confused, Sad, Normal, Happy, Fear, Disgust and Angry, were tried to be predicted accurately. 100% performance was achieved in ResNet 50, ResNet 50 32x4d, Vit_b_16, Vit_b32, EfficientNet B0, EfficientNet B1, EfficientNet B2 models run with the FERG dataset. Performance metrics of the models are presented comparatively. The results were compared with other studies in the literature using the FERG dataset.

Keywords

Deep Learning Methods, Facial Emotion Recognition, Convolutional Neural Networks, FERG, ResNet, EfficientNet

Evrişimsel Sinir Ağlarında Duygu Tanıma Algoritmalarının Hiperparametreler ile Kapsamlı Analizi

Abstract

İnsan doğasına dair fikir edinmemize ve hatta bazen yaşadıklarını hissetmemize yardımcı olabilecek insan davranışı fenomenleri olan ifadelerden duygular açığa çıkar. İnsanlar, davranışsal özelliklerini duygular aracılığıyla ifade ederler. Bir diğer deyişle ifadeler, insan davranış özelliklerinin duygusal dışavurumudur. Yüz ise bu ifadeleri insan-insan etkileşiminde en temel noktaya taşır. Bu etkileşimler sayesinde insanlardan geri tepkiler alınır, fakat makinelerin bu etkileşimi algılayabilmesi için üzerinde çalışılmalıdır. Karikatürize insan görünümleriyle yalnızca duygulara odaklanılmasına olanak sağlayan FERG veri setinde daha yüksek performans elde edilmesi, gerçek görüntülerdeki başarım oranını da arttıracaktır. Bu amaçla, bu çalışmada, insan yüzündeki duyguların yapay zekâ yardımıyla daha hızlı ve isabetli tahmin edilmesi üzerine yoğunlaşılmıştır. Bu bağlamda 7 duygu ifadesi olan Şaşkın, Üzgün, Normal, Mutlu, Korku, İğrenme, Kızgın isabetli bir şekilde tahmin edilmeye çalışılmıştır. FERG veri seti ile çalıştırılan ResNet 50, ResNet 50 32x4d, Vit_b_16, Vit_b32, EfficientNet B0, EfficientNet B1, EfficientNet B2 modellerinde %100 başarım elde edilmiştir. Modellerin performans metrikleri karşılaştırmalı olarak sunulmuştur. FERG veri seti ile yapılan literatürdeki diğer çalışmalarla sonuçlar kıyas edilmiştir.

Keywords

Derin Öğrenme Yöntemleri, Yüz Duygu Tanıma, Evrişimsel Sinir Ağları, FERG, ResNet, EfficientNet

References

• [1] Liu, M., Raj, A. N. J., Rajangam, V., Ma, K., Zhuang, Z., Zhuang, S. 2024. Multiscale-multichannel feature extraction and classification through one-dimensional convolutional neural network for Speech emotion recognition. Speech Communication, 156, 103010.
• [2] Gong, W., Qian, Y., Zhou, W., Leng, H. 2024. Enhanced spatial-temporal learning network for dynamic facial expression recognition. Biomedical Signal Processing and Control, 88, 105316.
• [3] Canal, F. Z., Müller, T. R., Matias, J. C., Scotton, G. G., de Sa Junior, A. R., Pozzebon, E., Sobieranski, A. C. 2022. A survey on facial emotion recognition techniques: A state-of-the-art literature review. Information Sciences, 582, 593-617.
• [4] Cheng, J., Deng, Y., Meng, H., Wang, Z. 2013. A facial expression based continuous emotional state monitoring system with gpu acceleration, in. 10th IEEE İnternational Conference And Workshops On Automatic Face And Gesture Recognition (FG), IEEE, 22-26 Nisan,Shanghai, China, 1–6.
• [5] Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I. 2010. The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression, in. IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, IEEE, 13-18 Haziran,San Francisco, USA, 94–101.
• [6] Akgül, İ., Kaya, V., Baran, A. 2021. Examination of facial mask detection using deep learning methods against coronavirus. 4. Uluslararası İpek Yolu Akademik Çalışmalar Sempozyumu, 17-18 Aralık, Nevşehir, Türkiye, 149-154.
• [7] Hasimah, A., Hariharan, M., Yaacob, S., Adom, A.H. 2015. Facial emotion recognition using empirical mode decomposition, Expert Systems with Applications,42,1261–1277.
• [8] Hossain, S., Umer, S., Asari, V., Rout, R. K. 2021. A unified framework of deep learning-based facial expression recognition system for diversified applications. Applied Sciences, 11(19), 9174.
• [9] Mehendale, N. 2020. Facial emotion recognition using convolutional neural networks (FERC). SN Applied Sciences, 2(3), 446.
• [10] Fakhar, S., Baber, J., Bazai, S. U., Marjan, S., Jasinski, M., Jasinska, E., ... Hussain, S. (022. Smart classroom monitoring using novel real-time facial expression recognition system. Applied Sciences, 12(23), 12134.
• [11] Khattak, A., Asghar, M. Z., Ali, M., Batool, U. 2022. An efficient deep learning technique for facial emotion recognition. Multimedia Tools and Applications, 81(2), 1649-1683.
• [12] Qazi, A. S., Farooq, M. S., Rustam, F., Villar, M. G., Rodríguez, C. L., Ashraf, I. 2022. Emotion Detection Using Facial Expression Involving Occlusions and Tilt. Applied Sciences, 12(22), 11797.
• [13] Mukhiddinov, M., Djuraev, O., Akhmedov, F., Mukhamadiyev, A., Cho, J. 2023. Masked face emotion recognition based on facial landmarks and deep learning approaches for visually impaired people. Sensors, 23(3), 1080.
• [14] Chaudhari, A., Bhatt, C., Krishna, A., Travieso-González, C. M. 2023. Facial emotion recognition with inter-modality-attention-transformer-based self-supervised learning. Electronics, 12(2), 288.
• [15] Aneja, D., Colburn, A., Faigin, G., Shapiro, L., Mones, B. 2016. Modeling stylized character expressions via deep learning. In Computer Vision–ACCV 2016: 13th Asian Conference on Computer Vision, 20-24 Kasım, Taipei, Taiwan, 136-153.
• [16] Soman, G., Vivek, M. V., Judy, M. V., Papageorgiou, E., Gerogiannis, V. C. 2022. Precision-based weighted blending distributed ensemble model for emotion classification. Algorithms, 15(2), 55.
• [17] Khan, N., Singh, A. V., Agrawal, R. 2023. Enhancing feature extraction technique through spatial deep learning model for facial emotion detection. Annals of Emerging Technologies in Computing (AETiC), 7(2), 9-22.
• [18] Muthamilselvan, T., Brindha, K., Senthilkumar, S., Saransh, Chatterjee, J. M., Hu, Y. C. 2023. Optimized face-emotion learning using convolutional neural network and binary whale optimization. Multimedia Tools and Applications, 82(13), 19945-19968.
• [19] Kalsum, T., Mehmood, Z. 2023. A novel lightweight deep convolutional neural network model for human emotions recognition in diverse environments. Journal of Sensors.
• [20] Kola, D. G. R., Samayamantula, S. K. 2021. Facial expression recognition using singular values and wavelet‐based LGC‐HD operator. IET Biometrics, 10(2), 207-218.
• [21] Wasi, A. T., Šerbetar, K., Islam, R., Rafi, T. H., Chae, D. K. 2023. ARBEx: Attentive feature extraction with reliability balancing for robust facial expression learning. Arxiv Preprint Arxiv,2305.01486.
• [22] Albraikan, A. A., Alzahrani, J. S., Alshahrani, R., Yafoz, A., Alsini, R., Hilal, A. M., ... Gupta, D. 2022. Intelligent facial expression recognition and classification using optimal deep transfer learning model. Image and Vision Computing, 128, 104583.
• [23] Tan, M., Le, Q. 2019, May. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning (pp. 6105-6114). PMLR.
• [24] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., ... Houlsby, N. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929.
• [25] Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., ... Rush, A. M. 2019. Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771.
• [26] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... Polosukhin, I. 2017. Attention is all you need. Advances in Neural İnformation Processing Systems, 30.
• [27] He, K., Zhang, X., Ren, S., Sun, J. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
• [28] Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K. 2017. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1492-1500).