Derin Öğrenme Tekniklerinde Duygu Analizinin Uygulanması

Öz

Duygu tanıma, sağlık, eğlence ve müşteri deneyimi gibi alanları kapsayan uygulamalarla insan-bilgisayar etkileşimini ilerletmede önemli bir teknoloji haline gelmiştir. Bu makale, yüz ifadelerinden duyguları tespit etmede beş derin öğrenme modelinin (YOLOv8m-cls, ResNet50, EfficientNetB5, MobileNetV2 ve DenseNet121) performansını değerlendirmektedir. Başlangıçta sekiz duygu kategorisi içeren AffectNet veri setinden yararlanarak, düşük veri kullanılabilirliği ve benzerlik nedeniyle üçünü hariç tuttuktan sonra beş duyguya odaklandık. İşlenen duygular arasında öfke, mutluluk, üzüntü, şaşkınlık ve korku yer almaktadır. Modeller, transfer öğrenmesi yoluyla ince ayar yapılarak YOLOv8m-cls'nin doğruluk, hız ve genellemeyi dengeleyerek en iyi performansı gösterdiği ve bu sayede gerçek zamanlı uygulamalar için uygun hale geldiği gösterilmiştir. ResNet50 ve EfficientNetB5 de iyi performans göstermiş, ResNet50 karmaşık yüz özelliklerini işlemede mükemmel performans gösterirken, EfficientNetB5 yüksek doğrulukla hesaplama verimliliği sunmuştur. Çalışma ayrıca, model performansını etkilemeye devam eden sınıf içi değişkenlik ve sınıflar arası benzerlik gibi zorluklara da dikkat çekmektedir. Bu bulgular, model mimarilerinin belirli uygulama gereksinimlerine göre seçilmesinin önemini vurgulamakta ve gelecekteki araştırmaların, duygu tanıma sistemlerini geliştirmek için çok modlu verilerin entegrasyonunu incelemesi gerektiğini göstermektedir.

Anahtar Kelimeler

• Duygu tanıma
• Derin öğrenme
• Transfer öğrenme
• Yüz ifadesi
• YOLOv8m-cls

Application of Emotion Analysis in Deep Learning Techniques

Abstract

Emotion recognition has become a pivotal technology in advancing human-computer interaction with applications spanning fields such as healthcare, entertainment, and customer experience. This paper evaluates the performance of five deep learning models—YOLOv8m-cls, ResNet50, EfficientNetB5, MobileNetV2, and DenseNet121—in detecting emotions from facial expressions. Leveraging the AffectNet dataset, which initially contained eight emotional categories, we focused on five emotions after excluding three due to low data availability and similarity. The emotions processed include anger, happiness, sadness, surprise, and fear. The models were fine-tuned through transfer learning, demonstrating that YOLOv8m-cls performed best, balancing accuracy, speed, and generalization, making it suitable for real-time applications. ResNet50 and EfficientNetB5 also performed well, with ResNet50 excelling in handling complex facial features and EfficientNetB5 offering computational efficiency with high accuracy. The study also highlights challenges such as intra-class variability and inter-class similarity, which continue to affect model performance. These findings underscore the importance of selecting model architectures based on specific application requirements and suggest that future research should explore integrating multimodal data to enhance emotion recognition systems.

Keywords

• Emotion recognition
• Deep learning
• Transfer learning
• Facial expression
• YOLOv8m-cls

References

1. [1] P. Ekman, “Facial expression and emotion,” Am. Psychol, vol. 48, no. 4, pp. 384–392, 1993. DOI:10.1037/0003-066X.48.4.384
2. [2] M. Pantic and L. J. Rothkrantz, “An expert system for recognition of facial actions and their intensity,” AAAI/IAAI, pp. 1026–1033, 2000.
3. [3] Z. Zeng, M. Pantic, G. I. Roisman and T. S. Huang, “A survey of affect recognition methods: Audio, visual and spontaneous expressions,” Proceedings of the 9th International Conference on Multimodal Interfaces, pp. 126–133, 2007.
4. [4] J. Zhang, Z. Yin, P. Cheng and S. Nichele, “Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review,” Inf Fusion, 2020. DOI:10.1016/j.inffus.2020.01.011
5. [5] Y. Tian, T. Kanade and J. F. Cohn, “Facial expression recognition,” in Handbook of Face Recognition, S. Z. Li and A. K. Jain, Eds., Springer, 2011, pp. 487–519. DOI:10.1007/978-0-85729-932-1_19
6. [6] P. Ekman and W. V. Friesen, Facial Action Coding System. Consulting Psychologists Press, 1978.
7. [7] P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar and I. Matthews, “The extended cohn-kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression,” 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition – Workshops (CVPRW), pp. 94–101, 2010. DOI:10.1109/CVPRW.2010. 5543262
8. [8] G. Tolias and O. Chum, “Asymmetric feature maps with application to sketch-based retrieval,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3155–3164, 2017.

9. [9] A. Krizhevsky, I. Sutskever and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in Neural Information Processing Systems (NeurIPS), pp. 1097–1105, 2012.
10. [10] D. Zilyas and A. Yılmaz, “Prediction model of educational success with machine learning methods,” DUMF Engineering Journal, vol. 14, no. 3, pp. 437–447, 2023. DOI:10.24012/dumf.1322273
11. [11] I. Goodfellow, Y. Bengio and A. Courville, Deep Learning. MIT Press, 2016.
12. [12] M. Wöllmer, A. Metallinou, F. Eyben, B. Schuller, and S. Narayanan, “Context-sensitive multimodal emotion recognition from speech and facial expression using bidirectional LSTM modeling,” IEEE Transactions on Affective Computing, vol. 2, no. 1, pp. 42–55, 2010.
13. [13] A. G. Howard et al., “MobileNets: Efficient convolutional neural networks for mobile vision applications,” arXiv preprint, arXiv:1704.04861, 2017.
14. [14] F. Eyben et al., “The geneva minimalistic acoustic parameter set (GeMAPS) for voice research and affective computing,” IEEE Transactions on Affective Computing, vol. 7, no. 2, pp. 190–202, 2015.
15. [15] J. Devlin, “BERT: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
16. [16] A. Zadeh, M. Chen, S. Poria, E. Cambria and L. P. Morency, “Tensor fusion network for multimodal sentiment analysis,” arXiv preprint, arXiv:1707.07250, 2017.
17. [17] R. T. Shawi, A. A. A. Abdulkadhim, and F. N. Abbas, “Facial Emotion Recognition Based on Improved ResNet50 Using Hybrid Pooling and Adaptive Leaky ReLU,” International Journal of Scientific Research in Science, Engineering and Technology, vol. 12, no. 3, pp. 728–737, 2025.
18. [18] A. Alshammari and M. E. Alshammari, “Emotional facial expression detection using YOLOv8,” Engineering, Technology & Applied Science Research, vol. 14, no. 5, pp. 16619–16623, 2024.
19. [19] A. A. Istiqomah, C. A. Sari, A. Susanto, and E. H. Rachmawanto, “Facial expression recognition using convolutional neural networks with transfer learning ResNet-50,” Journal of Applied Informatics and Computing, vol. 8, no. 2, pp. 257–264, 2024.
20. [20] W. Du, “Facial emotion recognition based on improved ResNet,” Applied and Computational Engineering, vol. 21, pp. 242–248, 2023.
21. [21] L. L. X. Wei and N. S. Sani, “Enhanced facial expression recognition based on ResNet50 with a convolutional block attention module,” International Journal of Advanced Computer Science & Applications, vol. 16, no. 1, 2025.
22. [22] M. B. Sutar and A. Ambhaikar, “A Comparative Study on Deep Facial Expression Recognition,” in 2023 7th International Conference on Intelligent Computing and Control Systems (ICICCS), pp. 903–911, May 2023.
23. [23] N. T. Singh, R. Ritu, C. Kaur, and A. Chaudhary, “Comparative analysis of traditional machine learning and deep learning techniques for facial expression recognition,” in 2023 14th International Conference on Computing Communication and Networking Technologies (ICCCNT), pp. 1–7, Jul. 2023.
24. [24] A. G. Vural, B. B. Cambazoğlu, P. Şenkul, and Z. O. Tokgöz, “A framework for sentiment analysis in Turkish: Application to polarity detection of movie reviews in Turkish,” in Computer and Information Sciences III, E. Gelenbe and R. Lent, Eds., Springer, 2013, pp. 437–445. DOI:10.1007/978-1-4471-4594-3_42
25. [25] M. Bilgin and İ. F. Şentürk, “Sentiment analysis of document vectors based tweets using supervised and semi-supervised learning,” Journal of Balıkesir University Institute of Science, vol. 21, no. 2, pp. 822–839, 2019.
26. [26] G. M. Demirci, S. R. Keskin, and G. Doğan, “Sentiment analysis in Turkish with deep learning,” 2019 IEEE International Conference on Big Data (Big Data). DOI:10.1109/bigdata47090.2019. 9006066
27. [27] M. Seyfioğlu and M. Demirezen, “A hierarchical approach for sentiment analysis and categorization of Turkish written customer relationship management data,” Proceedings of the 2017 Federated Conference on Computer Science and Information Systems (FedCSIS), IEEE, 2017.
28. [28] B. Ayan, B. Kuyumcu and B. Ceylan, “Detection of Islamophobic tweets on Twitter using sentiment analysis,” Gazi Univ. J. Sci. Part C Des. Technol., vol. 7, no. 2, pp. 495–502, 2019.
29. [29] F. N. Uyaroğlu Akdeniz and H. I. Cebeci, “Sentiment analysis approach in evaluation of municipal services: Sakarya province example,” J. Intell. Syst. Theory Appl., vol. 4, no. 2, pp. 127–135, 2021.
30. [30] S. Tuzcu, “Classification of online user comments with sentiment analysis,” Eskişehir Turkish World App. Res. Center Informat. J., vol. 1, no. 2, pp. 1–5, 2020.
31. [31] D. Yohanes, J. S. Putra, K. Filbert, K. M. Suryaningrum, and H. A. Saputri, “Emotion detection in textual data using deep learning,” Procedia Comput. Sci., vol. 227, pp. 464-473, 2023.
32. [32] G. Zhang and S. Ananiadou, “Examining and mitigating gender bias in text emotion detection task,” Neurocomputing, vol. 493, pp. 422–434, 2022. DOI:10.1016/j.neucom.2022.03.051
33. [33] D. Issa, M. F. Demirci, and A. Yazici, “Speech emotion recognition with deep convolutional neural networks,” Biomedical Signal Processing and Control, vol. 59, p. 101894, 2020. DOI:10.1016/j.bspc.2020.101894
34. [34] A. S. Yuksel and F. G. Tan, “Knowledge discovery in social networks with text mining techniques,” J. Eng. Sci. Design, vol. 6, no. 2, pp. 324–333, 2018. DOI:10.21923/jesd.384791
35. [35] M. Turan and E. Arığ, “Video sentiment analysis,” Eur. J. Sci. Technol. (EJOSAT) Supplementary Special Issue (HORA), pp. 34–41, 2021.
36. [36] C. D. Eyipınar, F. Büyükkalkan and K. Semiz, “Sentiment analysis of YouTube video comments on athlete nutrition,” Int J Phys Educ. Sports Technol., vol. 2, no. 2, pp. 27–39, 2021.
37. [37] A. Tepecik and E. Demir, “Analysis of Turkish voice recording data labeled with three emotions using machine learning algorithms,” Gazi Univ. Fac. Eng. Arch. J., 39(2):709–716.
38. [38] A. Sel, “Analysis of public opinion during the pandemic period using sentiment analysis method: The case of Türkiye,” Beykoz Academic Journal, vol. 10, no. 2, pp. 134–154, 2022.
39. [39] M. Uysal, “Comparison and applicability of modifying deep learning models for emotion recognition,” M.S. thesis, Burdur Mehmet Akif Ersoy Univ., 2024.
40. [40] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. C. Chen, “MobileNetV2: Inverted residuals and linear bottlenecks,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4510–4520, 2018.
41. [41] K. He, X. Zhang, S. Ren and J. Sun, “Deep residual learning for image recognition,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778, 2016.
42. [42] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
43. [43] J. Redmon, S. Divvala, R. Girshick and A. Farhadi, “You only look once: Unified, real-time object detection,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779–788, 2016.