Emotional Language Processing in Bilingualism: Subjective Affect and Prosodic Markers in Simultaneous Interpreting

Abstract

Emotion, as part of the overall sensorimotor, introspective, and affective system, is an essential part of language comprehension within the framework of embodied semantics. As emotional state influences semantic and syntactic processing, emotional language processing has been shown to modulate mood as well. The reciprocal relationship between language and emotion has also been informative in bilingualism. Here we take a relatively underresearched type of bilingual processing, simultaneous interpreting, as a case of extreme bilingualism and investigate the effect of emotional language rendering in the L1 on subjective affect and prosodic markers of L2 output. 18 trainee interpreters were asked to simultaneously interpret three speeches in Turkish that varied in emotionality, valence (negative, neutral, and positive), and difficulty in English. Responses to emotional language processing were analysed based on participants’ self-reported positive and negative affect using the Positive and Negative Affect Schedule (PANAS) and three prosodic parameters (intensity, pitch, and fluency). Results showed that interpreting emotionally negative speech increased negative affect, whereas interpreting emotionally positive speech did not modify positive affect. Intensity generally reflected cognitive load. Pitch and fluency, in particular, were more sensitive to changes in the valence of the source speech.

Keywords

• emotional language
• emotional load
• cognitive load
• valence
• bilingualism
• simultaneous interpreting
• prosodic parameters

İki Dillilikte Duygusal Dil İşlemleme: Andaş Çeviride Öznel Duygulanım ve Bürünsel Belirteçler

Abstract

Duygu; genel duyumotor, içgörü ve duygulanım sisteminin bir parçası olarak, bedenlenmiş anlambilim çerçevesinde dil anlamanın temel bir parçasıdır. Çalışmalar duygudurumun anlamsal ve sözdizimsel işlemlemeyi etkilediği gibi, duygusal dil işlemlemenin de duygudurumu etkilediğini göstermiştir. Dil ve duygu arasındaki karşılıklı ilişki, iki dillilik açısından da açıklayıcıdır. Bu çalışma, görece az çalışılan bir iki dillilik türü olan andaş çeviriyi bir tür “uç iki dillilik” olarak ele almakta ve ana dilde (L1) duygusal dil işlemlemenin öznel duygulanım ve ikinci dilde (L2) bürünsel belirteçler üzerindeki etkisini ortaya koymaktadır. Çalışmada 18 katılımcıdan duygusallık, duygusal değer (olumsuz, nötr ve olumlu) ve zorluk açısından farklılık gösteren üç Türkçe konuşmayı eş zamanlı olarak İngilizceye çevirmeleri istenmiştir. Duygusal dil işlemleme, katılımcıların Olumlu ve Olumsuz Duygulanım Ölçeği (PANAS) ile bildirdikleri olumlu ve olumsuz duygulanım puanları ile üç bürünsel parametre (yoğunluk, tizlik ve akıcılık) temelinde analiz edilmiştir. Sonuçlar, duygusal açıdan olumsuz konuşmanın çevirisinden sonra olumsuz duygulanımın arttığını ancak duygusal açıdan olumlu konuşmanın olumlu duygulanımı etkilemediğini göstermiştir. Seste yoğunluk genel olarak bilişsel yükü yansıtırken tizlik ve özellikle akıcılık kaynak konuşmanın duygusal değerindeki değişikliklere karşı daha hassastır.

Keywords

• duygusal dil
• duygusal yük
• bilişsel yük
• duygusal değer
• iki dillilik
• andaş çeviri
• bürünsel parametreler

References

1. Albuquerque, L., Valente, A. R. S., Teixeira, A., Figueiredo, D., Sa-Couto, P., & Oliveira, C. (2021). Association between acoustic speech features and non-severe levels of anxiety and depression symptoms across lifespan. PLOS ONE, 16(4), e0248842. https://doi.org/10.1371/journal.pone.0248842
2. Alku, P., & Vilkman, E. (1996). A comparison of glottal voice source quantification parameters in breathy, normal and pressed phonation of female and male speakers. Folia Phoniatrica et Logopaedica, 48(5), 240-254. https://doi.org/10.1159/000266415
3. Ateşman, E. (1997). Türkçe’de okunabilirliğin ölçülmesi. Dil Dergisi, 58, 71-74.
4. Bates, D. M., Mächler, M., Bolker, B., Walker, S., Maechler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1-48. https://doi.org/10.18637/jss.v067.i01
5. Boersma, P., & Weenink, D. (2022). Praat: Doing phonetics by computer (6.3.03) [Computer software]. http://www.praat.org/
6. Bohn-Gettler, C. M. (2019). Getting a grip: The PET framework for studying how reader emotions influence comprehension. Discourse Processes, 56(5-6), 386-401. https://doi.org/10.1080/0163853X.2019.1611174
7. Carstensen, L. L., & DeLiema, M. (2018). The positivity effect: A negativity bias in youth fades with age. Current Opinion in Behavioral Sciences, 19, 7-12. https://doi.org/10.1016/j.cobeha.2017.07.009
8. Conrad, M., Recio, G., & Jacobs, A. M. (2011). The time course of emotion effects in first and second language processing: A cross cultural ERP study with German-Spanish bilinguals. Frontiers in Psychology, 2. https://doi.org/10.3389/fpsyg.2011.00351

9. Dong, Y., & Li, P. (2020). Interpreting: A window into bilingual processing. Bilingualism: Language and Cognition, 23(4), 703-705. doi.org/10.1017/S1366728920000346
10. Filippa, M., Lima, D., Grandjean, A., Labbé, C., Coll, S. Y., Gentaz, E., & Grandjean, D. M. (2022). Emotional prosody recognition enhances and progressively complexifies from childhood to adolescence. Scientific Reports, 12(1), 17144. https://doi.org/10.1038/s41598-022-21554-0
11. Griffin, G. R., & Williams, C. E. (1987). The effects of different levels of task complexity on three vocal measures. Aviation, Space, and Environmental Medicine, 58(12), 1165-1170.
12. Gumul, E. (2021). Reporting stress in simultaneous interpreting. The analysis of trainee interpreters’ retrospective reports and outputs. Onomázein Revista de Lingüística Filología y Traducción, 8. https://doi.org/10.7764/onomazein.ne8.04
13. Habermas, T. (2018). Emotion and narrative: Perspectives in autobiographical storytelling (1st ed.). Cambridge University Press. https://doi.org/10.1017/9781139424615
14. Harris, C. L. (2004). Bilingual speakers in the lab: Psychophysiological measures of emotional reactivity. Journal of Multilingual and Multicultural Development, 25(2-3), 223-247. https://doi.org/10.1080/01434630408666530
15. Harris, C. L., Ayçiçegi, A., & Gleason, J. B. (2003). Taboo words and reprimands elicit greater autonomic reactivity in a first language than in a second language. Applied Psycholinguistics, 24(04), 561-579. https://doi.org/10.1017/S0142716403000286
16. Havas, D. A., Glenberg, A. M., & Rinck, M. (2007). Emotion simulation during language comprehension. Psychonomic Bulletin & Review, 14(3), 436-441. https://doi.org/10.3758/BF03194085
17. Hervais-Adelman, A., & Babcock, L. (2020). The neurobiology of simultaneous interpreting: Where extreme language control and cognitive control intersect. Bilingualism: Language and Cognition, 23(4), 740-751. https://doi.org/10.1017/S1366728919000324
18. Heyrani, R., Nejati, V., Abbasi, S., & Hartwigsen, G. (2022). Laterality in emotional language processing in first and second language. Frontiers in Psychology, 12, 736359. https://doi.org/10.3389/fpsyg.2021.736359
19. Holmes, E. A., & Mathews, A. (2010). Mental imagery in emotion and emotional disorders. Clinical Psychology Review, 30(3), 349-362. https://doi.org/10.1016/j.cpr.2010.01.001
20. Hothorn, T., Bretz, F., & Westfall, P. (2008). Simultaneous inference in general parametric models. Biometrical Journal, 50(3), 346-363. https://doi.org/10.1002/bimj.200810425
21. Huttunen, K., Keränen, H., Väyrynen, E., Pääkkönen, R., & Leino, T. (2011). Effect of cognitive load on speech prosody in aviation: Evidence from military simulator flights. Applied Ergonomics, 42(2), 348-357. https://doi.org/10.1016/j.apergo.2010.08.005
22. Johnson, E. J., & Tversky, A. (1983). Affect, generalization, and the perception of risk. Journal of Personality and Social Psychology, 45(1), 20-31. https://doi.org/10.1037/0022-3514.45.1.20
23. Juslin, P. N., & Laukka, P. (2003). Communication of emotions in vocal expression and music performance: Different channels, same code? Psychological Bulletin, 129(5), 770-814. https://doi.org/10.1037/0033-2909.129.5.770
24. Kensinger, E. A. (2009). Remembering the details: Effects of emotion. Emotion Review, 1(2), 99-113. https://doi.org/10.1177/1754073908100432
25. Klonowicz, T. (1990). A psychophysiological assessment of simultaneous interpreting: The interaction of individual differences and mental workload. Polish Psychological Bulletin, 21, 37-48.
26. Korpal, P. (2016). Interpreting as a stressful activity: Physiological measures of stress in simultaneous interpreting. Poznan Studies in Contemporary Linguistics, 52(2). https://doi.org/10.1515/psicl-2016-0011
27. Korpal, P. (2021). Stress and emotion in conference interpreting. In M. Albl-Mikasa, & E. Tiselius (Eds.), The Routledge handbook of conference interpreting (pp. 401-413). Routledge.
28. Korpal, P., & Jasielska, A. (2019). Investigating interpreters’ empathy: Are emotions in simultaneous interpreting contagious? Target, 31(1), 2-24. https://doi.org/10.1075/target.17123.kor
29. Kumcu, A. (2020). Konferans salonunda dağıtık biliş: Andaş çeviri sürecine yeni bir bakış. Hacettepe Üniversitesi Edebiyat Fakültesi Dergisi, 37(1), 170-185. https://doi.org/10.32600/huefd.621553
30. Kumcu, A., & Öztürk, A. (2023). Visual mental imagery and verbal working memory: Evidence from consecutive interpreting. Journal of Cognitive Psychology, 1-16. https://doi.org/10.1080/20445911.2023.2216917
31. Kurz, I. (2002). Physiological stress responses during media and conference interpreting. In G. Garzone & M. Viezzi (Eds.), Interpreting in the 21st century: Challenges and opportunities (pp. 195–202). John Benjamins Publishing Company.
32. Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017). lmerTest Package: Tests in linear mixed effects models. Journal of Statistical Software, 82(13), 1-26. https://doi.org/10.18637/jss.v082.i13
33. Lennes, M. (2002). Mark_pauses [Computer software]. http://phonetics.linguistics.ucla.edu/facilities/acoustic/mark_pauses.txt
34. Lindquist, K. A., MacCormack, J. K., & Shablack, H. (2015). The role of language in emotion: Predictions from psychological constructionism. Frontiers in Psychology, 6. https://doi.org/10.3389/fpsyg.2015.00444
35. Liu, M. (2011). Methodology in interpreting studies: A methodological review of evidence-based research. In B. Nicodemus & L. Swabey (Eds.), Advances in interpreting research: Inquiry in action (pp. 85-120). John Benjamins Publishing Company. https://doi.org/10.1075/btl.99.08liu
36. Mackintosh, J. (2003). The AIIC workload study. FORUM. Revue Internationale d’interprétation et de Traduction / International Journal of Interpretation and Translation, 1(2), 189-214. https://doi.org/10.1075/forum.1.2.09mac
37. Martínez-Sánchez, F., Muela-Martínez, J. A., Cortés-Soto, P., García Meilán, J. J., Vera Ferrándiz, J. A., Egea Caparrós, A., & Pujante Valverde, I. M. (2015). Can the acoustic analysis of expressive prosody discriminate schizophrenia? The Spanish Journal of Psychology, 18, E86. https://doi.org/10.1017/sjp.2015.85
38. Mellinger, C. D., & Hanson, T. A. (2019). Meta-analyses of simultaneous interpreting and working memory. Interpreting. International Journal of Research and Practice in Interpreting, 21(2), 165-195. https://doi.org/10.1075/intp.00026.mel
39. Moser-Mercer, B., Künzli, A., & Korac, M. (1998). Prolonged turns in interpreting: Effects on quality, physiological and psychological stress (Pilot study). Interpreting. International Journal of Research and Practice in Interpreting, 3(1), 47-64. https://doi.org/10.1075/intp.3.1.03mos
40. Mundt, J. C., Snyder, P. J., Cannizzaro, M. S., Chappie, K., & Geralts, D. S. (2007). Voice acoustic measures of depression severity and treatment response collected via interactive voice response (IVR) technology. Journal of Neurolinguistics, 20(1), 50-64. https://doi.org/10.1016/j.jneuroling.2006.04.001
41. Nour, S., Struys, E., & Stengers, H. (2020). Adaptive control in interpreters: Assessing the impact of training and experience on working memory. Bilingualism: Language and Cognition, 23(4), 772-779. https://doi.org/10.1017/S1366728920000127
42. Pavlenko, A. (2012). Affective processing in bilingual speakers: Disembodied cognition? International Journal of Psychology, 47(6), 405-428. https://doi.org/10.1080/00207594.2012.743665
43. Pisanski, K., & Sorokowski, P. (2021). Human stress detection: Cortisol levels in stressed speakers predict voice-based judgments of stress. Perception, 50(1), 80-87. https://doi.org/10.1177/0301006620978378
44. R Core Team. (2023). R: A language and environment for statistical computing [Computer software]. R Foundation for Statistical Computing. https://www.r-project.org/.
45. Rojo López, A. M., Foulquié-Rubio, A.-I., Espín López, L., & Martínez Sánchez, F. (2021). Analysis of speech rhythm and heart rate as indicators of stress on student interpreters. Perspectives, 29(4), 591–607. https://doi.org/10.1080/0907676X.2021.1900305
46. Rothkrantz, L. J. M., Wiggers, P., Wees, J.-W. A., & Vark, R. J. (2004). Voice stress analysis. In P. Sojka, I. Kopeček, & K. Pala (Eds.), Text, speech and dialogue (pp. 449-456). https://doi.org/10.1007/978-3-540-30120-2_57
47. Scherer, K. R. (1986a). Vocal affect expression: A review and a model for future research. Psychological Bulletin, 99(2), 143-165. https://doi.org/10.1037/0033-2909.99.2.143
48. Scherer, K. R. (1986b). Voice, stress, and emotion. In M. H. Appley & R. Trumbull (Eds.), Dynamics of stress (pp. 157-179). Springer US. https://doi.org/10.1007/978-1-4684-5122-1_9
49. Sondhi, S., Khan, M., Vijay, R., Salhan, A. K., & Chouhan, S. (2015). Acoustic analysis of speech under stress. International Journal of Bioinformatics Research and Applications, 11(5), 417. https://doi.org/10.1504/IJBRA.2015.071942
50. Splevins, K. A., Cohen, K., Joseph, S., Murray, C., & Bowley, J. (2010). Vicarious posttraumatic growth among interpreters. Qualitative Health Research, 20(12), 1705-1716. https://doi.org/10.1177/1049732310377457
51. Tamuri, K. (2014). Fundamental frequency in Estonian emotional read-out speech. Eesti Ja Soome-Ugri Keeleteaduse Ajakiri. Journal of Estonian and Finno-Ugric Linguistics, 6(1), 9-21. https://doi.org/10.12697/jeful.2015.6.1.01
52. Tisljár-Szabó, E., & Pléh, C. (2014). Ascribing emotions depending on pause length in native and foreign language speech. Speech Communication, 56, 35-48. https://doi.org/10.1016/j.specom.2013.07.009
53. Tolkmitt, F. J., & Scherer, K. R. (1986). Effect of experimentally induced stress on vocal parameters. Journal of Experimental Psychology: Human Perception and Performance, 12(3), 302-313. https://doi.org/10.1037/0096-1523.12.3.302
54. Vaish, A., Grossmann, T., & Woodward, A. (2008). Not all emotions are created equal: The negativity bias in social-emotional development. Psychological Bulletin, 134(3), 383-403. https://doi.org/10.1037/0033-2909.134.3.383
55. Van Bezooijen, R. (1995). Sociocultural aspects of pitch differences between Japanese and Dutch women. Language and Speech, 38(3), 253-265. https://doi.org/10.1177/002383099503800303
56. van den Broek, E. L. (2004). Emotional Prosody Measurement (EPM): A voice-based evaluation method for psychological therapy effectiveness. Studies in Health Technology and Informatics, 103, 118-125.
57. Van Der Linden, L., Van De Putte, E., Woumans, E., Duyck, W., & Szmalec, A. (2018). Does extreme language control training improve cognitive control? A comparison of professional interpreters, L2 teachers and monolinguals. Frontiers in Psychology, 9, 1998. https://doi.org/10.3389/fpsyg.2018.01998
58. Van Puyvelde, M., Neyt, X., McGlone, F., & Pattyn, N. (2018). Voice stress analysis: A new framework for voice and effort in human performance. Frontiers in Psychology, 9, 1994. https://doi.org/10.3389/fpsyg.2018.01994
59. Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54(6), 1063-1070. https://doi.org/10.1037/0022-3514.54.6.1063
60. Weeks, J. W., Lee, C.-Y., Reilly, A. R., Howell, A. N., France, C., Kowalsky, J. M., & Bush, A. (2012). “The Sound of Fear”: Assessing vocal fundamental frequency as a physiological indicator of social anxiety disorder. Journal of Anxiety Disorders, 26(8), 811-822. https://doi.org/10.1016/j.janxdis.2012.07.005
61. Winter, B. (2013). Linear models and linear mixed effects models in R with linguistic applications. http://arxiv.org/abs/1308.5499