Automated Detection of Fundamental Emotions from Literary Texts of Middle Grade Students

Year 2026, Volume: 17 Issue: 1 , 1 - 23 , 01.04.2026

Esma Yildirim , Stephanie Koester , Özlem Şener , Meral Yildirim , Erhan Sahin

https://izlik.org/JA23DF75JZ

Abstract

The middle school period covers the beginning of the adolescent years of individuals. In this period, children display very complex and chaotic negative emotions such as anxiety, shyness, loneliness, guilt, depression, and anger intensely. Detecting these emotions accurately can help facilitate coping with the crises of this period and regulate emotions. It also can prevent potential suffering in later ages, reduce societal costs and prevent secondary diagnosis from developing. Emotions are by their nature at the center of literature and are better understood through the experience of writing or reading literary texts. Studies also show that the best type of written text to analyze for detection of emotions is informal texts. We believe the literary value and spontaneity of texts written by children without any given topic allow them to better channel their fundamental emotions into these texts without any reservations.
In this study, we designed and developed a website for middle grade students to upload their literary works such as poems, fairytales, stories, etc. and collected and analyzed their texts to identify five fundamental emotions (anger, fear, disgust, sadness, and joy) using sentiment analysis methods and machine learning models. We use text representation techniques such as Bag-Of-Words, TF-IDF, and Word2Vector and neural network models such as Multi-layer Perceptrons (MLPs), and Convolutional Neural Networks (CNNs). We obtained the best results with TF-IDF + MLP model in terms of average f1-score, precision and recall metrics (0.73, 0.71, 0.83) and the best detected emotion is Joy while the worst detected is Disgust. We believe the accuracy of our models can increase with the inclusion of more data and promoting and making the website more accessible can allow us to enrich our dataset. Moreover, after the model is tested again with a larger sample, this system can be integrated with guidance and psychological counseling services, paving the way for psychosocial support needs analysis for students. A systematic monitoring mechanism could allow for the tracking of students' emotional changes over time and help develop early intervention strategies.

Keywords

sentiment analysis , emotion detection , neural networks , natural language processing , adolescent psychology

Ethical Statement

This is an observational study. The dataset was collected from a public website and no identifiable information was used. İnanç Türkeş Middle School administration has confirmed that no ethical approval is required.

Supporting Institution

Inanc Turkes Middle School

References

• Acheampong, F. A., Nunoo-Mensah, H., & Chen, W. (2021). Transformer models for text-based emotion detection: A review. IEEE Access, 9, 182760–182777. https://doi.org/10.1109/ACCESS.2021.3112728
• Akın, A. A., & Akın, M. D. (2007). Zemberek: An open source NLP framework for Turkic languages. Structure, 10, 1–5. https://doi.org/10.1.1.556.69
• Bauer, D. (1976). An exploratory study of developmental changes in children's fears. Journal of child psychology and psychiatry, and allied disciplines, 17 1, 69-74. https://doi.org/10.1111/j.1469-7610.1976.tb00375.x.
• Bhargava, C., Poornima, S., Mahur, S., & Pushpalatha, M. (2021). Depression detection using sentiment analysis of tweets. Turkish Journal of Computer and Mathematics Education, 12(11), 5411–5418. https://turcomat.org/index.php/turkbilmat/article/view/6770
• Boyacıoğlu, N. E., & Küçük, L. (2011). How do irrational beliefs affect test anxiety during adolescence? Journal of Psychiatric Nursing, 2(1), 40–45. https://phdergi.org/jvi.aspx?un=PHD-03164&volume=2&issue=1
• Callaghan, B., Meyer, H., Opendak, M., Van Tieghem, M., Harmon, C., Li, A., Lee, F., Sullivan, R., & Tottenham, N. (2019). Using a Developmental Ecology Framework to Align Fear Neurobiology Across Species. Annual review of clinical psychology, 15, 345-369. https://doi.org/10.1146/annurev-clinpsy-050718-095727.
• Camacho-Morles, J., Slemp, G., Pekrun, R., Loderer, K., Hou, H., & Oades, L. (2021). Activity Achievement Emotions and Academic Performance: A Meta-analysis. Educational Psychology Review, 33, 1051 - 1095. https://doi.org/10.1007/s10648-020-09585-3.
• Charte, F., Rivera, A. J., del Jesús, M. J., & Herrera, F. (2015). MLSMOTE: Approaching imbalanced multilabel learning through synthetic instance generation. Knowledge-Based Systems, 89, 385–397. https://doi.org/10.1016/j.knosys.2015.07.019
• Cormier, S., & Hackney, H. L. (2013). Counseling strategies and interventions: Pearson New International Edition. Pearson Higher Education.
• Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019), 4171–4186. https://doi.org/10.18653/v1/N19-1423
• Dong, Y., Hsiao, Y., Dawson, N., Banerji, N., & Nation, K. (2024). The emotional content of children’s writing: A data-driven approach. Cognitive Science, 48(3), e13423. https://doi.org/10.1111/cogs.13423
• Egbochuku, E. O., Obodo, B., & Obadan, N. O. (2008). Efficacy of rational-emotive behaviour therapy on the reduction of test anxiety among adolescents in secondary schools. European Journal of Social Sciences, 6(4), 152–164. https://www.researchgate.net/publication/254064865_Efficacy_of_Rational-Emotive_Behaviour_Therapy_on_the_Reduction_of_Test_Anxiety_Among_Adolescents_in_Secondary_Schools
• Ekman, P. (1992). Are there basic emotions? Psychological Review, 99(3), 550–553. https://doi.org/10.1037/0033-295X.99.3.550
• Ekman, P. (2004). What we become emotional about. In A. S. R. Manstead, N. H. Frijda, & A. H. Fischer (Eds.), Feelings and emotions: The Amsterdam symposium (pp. 119–135). Cambridge University Press. https://doi.org/10.1017/CBO9780511806582.008
• Ekman, P. (2016). What scientists who study emotion agree about. Perspectives on Psychological Science, 11(1), 31–34. https://doi.org/10.1177/1745691615596992
• Goleman, D. (1998). Duygusal zeka. İstanbul: Varlık Yayınları, 34. https://www.varlik.com.tr/kitapDetay.aspx?kategoriID=16&kitapID=49
• Google. (2024). Google news dataset.
• Göçen, G. (2019). The effect of creative writing activities on elementary school students’ creative writing achievement, writing attitude and motivation. Journal of Language and Linguistic Studies, 15(3), 1032-1044.
• Greenberg, L. S., & Pascual-Leone, A. (2006). Emotion in psychotherapy: A practice-friendly research review. Journal of Clinical Psychology, 62 (5), 611–630. https://doi.org/10.1002/jclp.20252
• Haggren, H., & Amethier, P. (2018). Data-driven predictions of outcome for an internet-delivered treatment against anxiety disorders: A comparison of clinician and algorithm performance (bachelor’s thesis, KTH Royal Institute of Technology). Diva Portal. https://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-230705
• Havigerová, J. M., Haviger, J., Kučera, D., & Hoffmannová, P. (2019). Text-based detection of the risk of depression. Frontiers in Psychology, 10, 513. https://doi.org/10.3389/fpsyg.2019.00513
• Hipson, W. (2019). Using sentiment analysis to detect affect in children’s and adolescents’ poetry. International Journal of Behavioral Development, 43, 375 - 382. https://doi.org/10.1177/0165025419830248.
• Hofmann, J., Troiano, E., Sassenberg, K., & Klinger, R. (2020). Appraisal Theories for Emotion Classification in Text. Proceedings of the 28th International Conference on Computational Linguistics, 125–138.
• Jacobs, A. M., Herrmann, B., Lauer, G., Lüdtke, J., & Schroeder, S. (2020). Sentiment analysis of children and youth literature: Is there a pollyanna effect? Frontiers in Psychology, 11, 574746. https://doi.org/10.3389/fpsyg.2020.574746
• Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017). Bag of tricks for efficient text classification. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2017), 427–431. https://doi.org/10.18653/v1/E17-2068
• Kauschke, C., Bahn, D., Vesker, M., & Schwarzer, G. (2019). The Role of Emotional Valence for the Processing of Facial and Verbal Stimuli—Positivity or Negativity Bias? Frontiers in Psychology, 10. https://doi.org/10.3389/fpsyg.2019.01654.
• Kırmızı, F., & Adıgüzel, D. Ç. (2023). İlkokul öğrencilerinin yaratıcı yazma ürünlerinin bazı değişkenlere göre değerlendirilmesi. Pamukkale Üniversitesi Eğitim Fakültesi Dergisi, 58, 370-397.
• Kidd, D. C., & Castano, E. (2013). Reading literary fiction improves theory of mind. Science, 342(6156), 377–380. https://doi.org/10.1126/science.1239918
• Kim, S., Valitutti, A., & Calvo, R. (2010). Evaluation of Unsupervised Emotion Models to Textual Affect Recognition. , 62-70.
• Krommyda, M., Rigos, A., Bouklas, K., & Amditis, A. (2021). An Experimental Analysis of Data Annotation Methodologies for Emotion Detection in Short Text Posted on Social Media. Informatics, 8, 19. https://doi.org/10.3390/informatics8010019.
• Kumschick, I., Beck, L., Eid, M., Witte, G., Klann-Delius, G., Heuser, I., Steinlein, R., & Menninghaus, W. (2014). Reading and Feeling: The effects of a literature-based intervention designed to increase emotional competence in second and third graders. Frontiers in Psychology, 5, 1448. https://doi.org/10.3389/fpsyg.2014.01448.
• Makinist, S., Hallaç, İ. R., Karakuş, B. A., & Aydın, G. (2017). Preparation of improved Turkish dataset for sentiment analysis in social media. ITM Web of Conferences, 13, 01030. https://doi.org/10.1051/itmconf/20171301030
• Mohsin, M. A., & Beltiukov, A. (2019, May). Summarizing emotions from text using Plutchik’s wheel of emotions. In Proceedings of the 7th Scientific Conference on Information Technologies for Intelligent Decision-Making Support (ITIDS 2019) (pp. 291–294). Atlantis Press. https://doi.org/10.2991/itids-19.2019.52
• Nook, E., Stavish, C., Sasse, S., Lambert, H., Mair, P., McLaughlin, K., & Somerville, L. (2020). Charting the development of emotion comprehension and abstraction from childhood to adulthood using observer-rated and linguistic measures. Emotion. 20(5), 773-792. https://doi.org/10.1037/emo0000609.
• Oatley, K. (2002). Emotions and the story worlds of fiction. In M. C. Green, J. J. Strange, & T. C. Brock (Eds.), Narrative impact: Social and cognitive foundations (pp. 39–69). Lawrence Erlbaum Associates. https://www.routledge.com/Narrative-Impact-Social-and-Cognitive-Foundations/Green-Strange-Brock/p/book/9780805832990
• Orhan, Z., Mercan, M., & Sertbaş, A. (2014). Turkish text analysis system for automatic detection of psychiatric disorders. Türkiye Bilişim Vakfı Bilgisayar Bilimleri ve Mühendisliği Dergisi, 7(1), 24–30. https://dergipark.org.tr/en/pub/tbbmd/issue/22247/238819
• Özgüle, E. T. U., & Sümer, N. (2017). Ergenlikte duygu düzenleme ve psikolojik uyum: Duygu düzenleme ölçeğinin Türkçe uyarlaması. Turk Psikoloji Yazıları, 20 (40), 1–18. https://dergipark.org.tr/tr/download/article-file/2133036
• Panksepp, J. (2004). Affective neuroscience: The foundations of human and animal emotions. Oxford University Press. https://global.oup.com/academic/product/affective-neuroscience-9780195178050
• Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2002). BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (pp. 311–318). Association for Computational Linguistics. https://doi.org/10.3115/1073083.1073135
• Plutchik, R. (2001). Integration, differentiation, and derivatives of emotion. Evolution and Cognition, 7(2)), 114–125. https://doi.org/10.1007/978-3-319-28099-8_547-1
• Provoost, S., Ruwaard, J., Van Breda, W., Riper, H., & Bosse, T. (2019). Validating automated sentiment analysis of online cognitive behavioral therapy patient texts: An exploratory study. Frontiers in psychology, 10, 1065. https://doi.org/10.3389/fpsyg.2019.01065
• Puspitoningrum, E., Khan, R., & Swanjaya, D. (2024). Creative Writing as an Effort to Improve Children's Emotional Intelligence. Qubahan Academic Journal. https://doi.org/10.48161/qaj.v4n3a695.
• Rutter, M. J., Bishop, D., Pine, D. S., Scott, S., Stevenson, J. S., Taylor, E. A., & Thapar, A. (2011). Rutter’s child and adolescent psychiatry (5th edition). John Wiley & Sons.
• Shirazi, Y., Punamäki, R. L., Peltonen, K., Malekzadeh, M., & Esmaeili, O. (2021). Narrative-based intervention and emotional intelligence in female children. East European Journal of Psycholinguistics, 8(1), 40–55. https://doi.org/10.29038/eejpl.2021.8.1.shi
• Stephen, J. J., & Prabu, P. (2019). Detecting the magnitude of depression in twitter users using sentiment analysis. International Journal of Electrical and Computer Engineering (IJECE), 9(4), 3247. https://doi.org/10.11591/ijece.v9i4.pp3247-3255
• Tanju, E. H. (2010). Çocuklarda kitap okuma alışkanlığı’na genel bir bakış. Sosyal Politika Çalışmaları Dergisi, 21(21), 30–39. https://doi.org/10.21560/spcd.43960
• Thompson, R. A. (2007). Socialization of emotion regulation in the family. In J. J. Gross (Ed.), Handbook of emotion regulation (pp. 249–268). The Guilford Press.
• Tokcaer, S. (2021). Türkçe metinlerde duygu analizi. Yaşar Üniversitesi E-Dergisi, 16 (63), 1514–1534. https://doi.org/10.19168/jyasar.928843
• Türkben, T. (2021). Ortaokul öğrencilerininn yazma becerileri, yazmaya yönelik tutumlari ve yazma motivasyonlari arasindaki ilişki. Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 21(3), 900–922. https://doi.org/10.17240/aibuefd.2021.21.64908-788146
• Uymaz, H., & Metin, S. (2023). Enriching Transformer-Based Embeddings for Emotion Identification in an Agglutinative Language: Turkish. IT Professional, 25, 67-73. https://doi.org/10.1109/mitp.2023.3278029.
• Villani, D., Carissoli, C., Triberti, S., Marchetti, A., Gilli, G., & Riva, G. (2018). Videogames for Emotion Regulation: A Systematic Review. Games for Health Journal, 7 2, 85-99. https://doi.org/10.1089/g4h.2017.0108.
• Walsh, C. G., Ribeiro, J. D., & Franklin, J. C. (2018). Predicting suicide attempts in adolescents with longitudinal clinical data and machine learning. Journal of child psychology and psychiatry, 59(12), 1261–1270. https://doi.org/10.1111/jcpp.12916
• Wang, S., & Manning, C. D. (2012). Baselines and bigrams: Simple, good sentiment and topic classification. Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (ACL 2012), 90–94.
• Wang, X., Zhang, C., Ji, Y., Sun, L., Wu, L., & Bao, Z. (2013). A depression detection model based on sentiment analysis in micro-blog social network. In L. Liu, J. Qu, & C. S. Jensen (Eds.), Trends and applications in knowledge discovery and data mining: PAKDD 2013 international workshops: DMApps, DANTH, QIMIE, BDM, CDA, CloudSD, Gold Coast, QLD, Australia, April 14-17, 2013, revised selected papers (Vol. 7867, pp. 201–213). Springer. https://doi.org/10.1007/978-3-642-38614-7_19
• Weismayer, C., Gunter, U., & Önder, I. (2021). Temporal variability of emotions in social media posts. Technological Forecasting and Social Change, 167, 120699. https://doi.org/10.1016/j.techfore.2021.120699
• Yiend, J., Mackintosh, B., & Mathews, A. (2005). Enduring consequences of experimentally induced biases in interpretation. Behaviour Research and Therapy, 43 (6), 779–797. https://doi.org/10.1016/j.brat.2004.05.003
• Zhou, D., Zhang, X., Zhou, Y., Zhao, Q., & Geng, X. (2016, November). Emotion distribution learning from texts. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (pp. 638-647). Association for Computational Linguistics. https://doi.org/10.18653/v1/D16-1061