Exploring Naturalness in MOOC Video Lecture MT Subtitles: A Case Study

Abstract

This case study investigates the naturalness of subtitle translation using massive open online course (MOOC) video lecture subtitles that were machine translation (MT)-generated by a computer-assisted translation (CAT) tool, Termsoup, after the pre- and post-editing phases. Both the pre- and post-editing phases were conducted by translator trainees while supervised by a professional translation trainer. With the accuracy secured, the naturalness of the MOOC translated subtitles from Chinese to English is further explored. In this study, distributions of lexical bundles (LBs) in terms of structural and functional properties are used to investigate the academic register in the MT output. The result shows that while in previous literature classroom teaching extensively uses a taxonomy of bundles in both written and spoken modes, the investigated corpus tends toward the ‘literate’ side rather than the ‘oral’ side. Nonetheless, the increase in the number of discourse organizers discovered in the corpus not only implies that the MOOC language may share comparable purposes with TED talks but also aids in the formation of spoken elements in the register. The study ends with suggestions for extending the corpus of MOOCs to gain a better grasp of specific formulaic languages in subtitle translation, along with proposals for pre-editing the source text to enhance the academic register before the MT process.

Keywords

• naturalness of subtitles
• massive open online course video lectures
• lexical bundles
• academic register
• machine translation output

References

1. Arenas, Ana Guerberof. 2019. “Pre-editing and Post-editing.” In The Bloomsbury Companion to Language Industry Studies, edited by Erik Angelone, Maureen Ehrensberger-Dow, and Gary Massey, 333–360. London: Bloomsbury.
2. Arnon, Inbal, and Neal Snider. 2010. “More Than Words: Frequency Effects for Multi-word Phrases.” Journal of Memory and Language 62 (1): 67–82. doi:10.1016/j.jml.2009.09.005.
3. Aston, Guy. 2018. “Acquiring the Language of Interpreters: A Corpus-based Approach.” In Making Way in Corpus-based Interpreting Studies, edited by Mariachiara Russo, Claudio Bendazzoli, and Bart Defrancq, 83–96. Singapore: Springer.
4. Atapattu, Thushari, and Katrina Falkner. 2018. “Impact of Lecturer’s Discourse for Students’ Video Engagement: Video Learning Analytics Case Study of MOOCs.” Journal of Learning Analytics 5 (3): 182–197. doi:10.18608/jla.2018.53.12.
5. Berūkštienė, Donta. 2017. “A Corpus-driven Analysis of Structural Types of Lexical Bundles in Court Judgments in English and Their Translation into Lithuanian.” Kalbotyra 70:7–31. doi:10.15388/Klbt.2017.11181.
6. Biber, Douglas. 1988. Variation Across Speech and Writing. Cambridge: Cambridge University Press.
7. Biber, Douglas. 2006a. “Stance in Spoken and Written University Registers.” Journal of English for Academic Purposes 5 (2): 97–116. doi:10.1016/j.jeap.2006.05.001.
8. Biber, Douglas.. 2006b. University Language: A Corpus-based Study of Spoken and Written Registers. Amsterdam: John Benjamins.

9. Biber, Douglas, and Federica Barbieri. 2007. “Lexical Bundles in University Spoken and Written Registers.” English for Specific Purposes 26 (3): 263–286. doi:10.1016/j.esp.2006.08.003.
10. Biber, Douglas, and Susan Conrad. 2019. Register, Genre, and Style. 2nd ed. Cambridge: Cambridge University Press.
11. Biber, Douglas, Susan Conrad, and Geoffrey Leech. 2002. Longman Student Grammar of Spoken and Written English. Harlow: Pearson.
12. Biber, Douglas, Susan Conrad, and Viviana Cortes. 2004. “If You Look at...: Lexical Bundles in University Teaching and Textbooks.” Applied Linguistics 25 (3): 371–405. doi:10.1093/applin/25.3.371.
13. Biel, Łucja. 2018. “Lexical Bundles in EU Law: The Impact of Translation Process on the Patterning of Legal Language.” In Phraseology in Legal and Institutional Settings: A Corpus-based Interdisciplinary Perspective, edited by Stanisiaw Goźdź-Roszkowski and Gianluca Pontrandolfo, 11–26. London: Routledge.
14. Bogucku, Łukasz. 2019. Areas and Methods of Audiovisual Translation Research. Bern: Peter Lang.
15. Chaume-Varela, Frederic. 2007. “Quality Standards in Dubbing: A Proposal.” TradTerm 13:71–89. doi:10.11606/issn.2317-9511.tradterm.2007.47466.
16. Chiaro, Delia. 2008. “Issues of Quality in Screen Translation: Problems and Solutions.” In Between Text and Image: Updating Research in Screen Translation, edited by Delia Chiaro, Christine Heiss, and Chiara Bucaria, 241–256. Amsterdam: John Benjamins.
17. Cortes, Viviana. 2004. “Lexical Bundles in Published and Student Disciplinary Writing: Examples from History and Biology.” English for Specific Purposes 23 (4): 397–423. doi:10.1016/j.esp.2003.12.001.
18. Coxhead, Averil, Dang, Thi Ngoc Yen, and Shota Mukai. 2017. “Single and Multi-word Unit Vocabulary in University Tutorials and Laboratories: Evidence from Corpora and Textbooks.” Journal of English for Academic Purposes 30:66–78. doi:10.1016/j.jeap.2017.11.001.
19. Crawford Camiciottoli, Belinda. 2007. The Language of Business Studies Lectures: A Corpus-assisted Analysis. Amsterdam: John Benjamins.
20. Csomay, Eniko. 2006. “Academic Talk in American University Classroom: Crossing the Boundaries of Oral-literate Discourse?” Journal of English for Academic Purposes 5 (2): 117–135. doi:10.1016/j.jeap.2006.02.001.
21. Csomay, Eniko. 2007. “A Corpus-based Look at Linguistic Variation in Classroom Interaction: Teacher Talk Versus Student Talk in American University Classes.” Journal of English for Academic Purposes 6 (4): 336–355. doi:10.1016/j.jeap.2007.09.004.
22. Danilaviciene, Greta, Jolita Horbacauskiene, and Ramune Kasperaviciene. 2017. “On Formulaic Language in Subtitling and Voice-over.” Translationes 9 (1): 71–82. doi:10.1515/tran-2017-0004.
23. Das, Ananda, and Partha Pratim Das. 2019. “Automatic Semantic Segmentation and Annotation of MOOC Lecture Videos.” In Digital Libraries at the Crossroads of Digital Information for the Future, edited by Adam Jatowt, Akira Maeda, and Sue Yeon Syn, 181–188. Berlin: Springer.
24. Días-Cintas, Jorge. 2019. “Audiovisual Translation.” In The Bloomsbury Companion to Language Industry Studies, edited by Erik Angelone, Maureen Ehrensberger-Dow, and Gary Massey, 209–230. London: Bloomsbury.
25. Doherty, Stephen. 2017. “Issues in Human and Automatic Translation Quality Assessment.” In Human Issues in Translation Technology, edited by Dorothy Kenny, 131–148. London: Routledge.
26. Doherty, Stephen, and Jan-Louis Kruger. 2018. “The Development Eye Tracking in Empirical Research on Subtitling and Captioning.” In Seeing into Screens. Eye Tracking and the Moving Image, edited by Tessa Dwyer, Claire Perkins, Sean Redmond, and Jodi Sita, 46–64. London: Bloomsbury.
27. Ellis, Nick C., and Rita Simpson-Vlach. 2009. “Formulaic Language in Native Speakers: Triangulating Psycholinguistics, Corpus Linguistics, and Education.” Corpus Linguistics and Linguistic Theory 5 (1): 61–78. doi:10.1515/CLLT.2009.003.
28. Ellis, Nick C., Rita Simpson-Vlach, and Carson Maynard. 2008. “Formulaic Language in Native and Second Language Speakers: Psycholinguistics, Corpus Linguistics, and TESOL.” TESOL Quarterly 42 (3): 375–396. doi:10.1002/j.1545-7249.2008.tb00137.x.
29. Georgakopoulou, Panayota (Yota). 2019. “Technologization of Audiovisual Translation.” In The Routledge Handbook of Audiovisual Translation, edited by Luis Pérez-González, 516–539. London: Routledge.
30. Grabowski, Łukasz. 2018. “On Identification of Bilingual Lexical Bundles for Translation Purposes: The Case of an English-Polish Comparable Corpus of Patient Information Leaflets.” In Multiword Units in Machine Translation and Translation Technology, edited by Ruslan Mitkov, Joanna Monti, Gloria Corpas Pastor, and Violeta Seretan, 181–200. Amsterdam: John Benjamins.
31. Hyland, Ken. 2008. “Academic Clusters: Text Patterning in Published and Postgraduate Writing.” International Journal of Applied Linguistics 18 (1): 41–62. doi:10.1111/j.1473-4192.2008.00178.x.
32. Kashiha, Hadi, and Swee Heng, Chan. 2014. “Discourse Functions of Formulaic Sequences in Academic Speech Across Two Disciplines.” GEMA: Online Journal of Language Studies 14 (2): 15–27. doi:10.17576/GEMA-2014-1402-02.
33. Kuo, Arista Szu-Yu. 2020. “The Tangled Strings of Parameters and Assessments in Subtitling Quality: An Overview.” In The Palgrave Handbook of Audiovisual Translation and Media Accessibility, edited by Łukasz Bogucki and Mikołaj Deckert, 437–458. Cham: Palgrave Macmillan.
34. Lee, Changsoo. 2013. “Using Lexical Bundle Analysis as Discovery Tool for Corpus-based Translation Research.” Perspectives: Studies in Translatology 21 (3): 378–395. doi:10.1080/0907676X.2012.657655.
35. Lee, Joseph J., and Nicholas C. Subtirelu. 2015. “Metadiscourse in the Classroom: A Comparable Analysis of EAP Lessons and University Lectures.” English for Specific Purposes 37:52–62. doi:10.1016/j.esp.2014.06.005.
36. Li, Jie, and Nobert Schmitt. 2009. “The Acquisition of Lexical Phrases in Academic Writing: A Longitudinal Chunks.” Modern Foreign Languages 39 (2): 246–256. doi:10.1016/j.jslw.2009.02.001.
37. Liu, Chen-Yu, and Hao-Jan Howard Chen. 2020a. “Analyzing the Function of Lexical Bundles in Undergraduate Academic Lectures for Pedagogical Use.” English for Specific Purposes 58:122–137. doi:10.1016/j.esp.2019.12.003.
38. Liu, Chen-Yu, and Hao-Jan Howard Chen. 2020b. “Functional Variation of Lexical Bundles in Academic Lectures and TED Talks.” Register Studies 2 (2): 176–208. doi:10.1075/rs.18003.liu.
39. Liu, Kanglong, Joyce Oiwun Cheung, and Riccardo Moratto. 2022. “Lexical Bundles in Fictional Dialogues of Two Hongloumeng Translations: A Corpus-assisted Approach.” In Advances in Corpus Applications in Literary and Translation Studies, edited by Riccado Moratto and Defeng Li, 229–253. London: Routledge.
40. Liu, Kanglong, and Muhammad Afzaal. 2021. “Translator’s Style Through Lexical Bundles: A Corpus-driven Analysis of Two English Translations of Hongloumeng.” Frontiers in Psychology 12. doi:10.3389/fpsyg.2021.633422.
41. Li, Yang, and Sandra L. Halverson. 2020. “A Corpus-based Exploration into Lexical Bundles in Interpreting.” Across Languages and Cultures 21 (1): 1–22. doi:10.1556/084.2020.00001.
42. Li, Yang, and Sandra L. Halverson. 2022. “Lexical Bundles in Formulaic Interpreting: A Corpus-based Descriptive Exploration.” Translation and Interpreting Studies 19 (2): 33–56. doi:10.1075/tis.19037.li.
43. Miyata, Rei. 2021. Controlled Document Authoring in a Machine Translation Age. London: Routledge.
44. Neely, Elizabeth, and Viviana Cortes. 2009. “A Little Bit About: Analyzing and Teaching Lexical Bundles in Academic Lectures.” Language Value 1 (1): 17–38. http://www.e-revistes.uji.es/languagevalue.
45. Nesi, Hilary, and Helen Basturkmen. 2006. “Lexical Bundles and Discourse Signalling in Academic Lectures.” International Journal of Corpus Linguistics 11 (3): 283–304. doi:10.1075/ijcl.11.3.04nes.
46. Nikolic, Kristijan. 2015. “The Pros and Cons of Using Templates in Subtitling.” In Audiovisual Translation in a Global Context. Mapping and Ever-changing Landscape, edited by Rocío Baños Piñero and Jorge Díaz Cintas, 192–202. Basingstoke: Palgrave Macmillan.
47. O’Brien, Sharon. 2017. “Machine Translation and Cognition.” In The Handbook of Translation and Cognition, edited by John W. Schwieter and Aline Ferreira, 313–331. New Jersey: Wiley Blackwell.
48. Pavesi, Maria. 2008. “Spoken Language in Film Dubbing: Target Language Norms, Interference and Translational Routines.” In Between Text and Image. Updating Research in Screen Translation, edited by Delia Chiaro, Christine Heiss, and Chiara Bucaria, 79–99. Amsterdam: John Benjamins.
49. Pavesi, Maria. 2009. “Dubbing English into Italian: A Closer Look at the Translation of Spoken Language.” In New Trends in Audiovisual Translation, edited by Jorge Díaz-Cintas, 197–209. Toronto: Multilingual Matters.
50. Pavesi, Maria. 2013. “This and That in the Language of Film Dubbing: A Corpus-based Analysis.” Meta 58 (1): 103–133. doi:10.7202/1023812ar.
51. Pavesi, Maria, Maicol Formentelli, and Elisa Ghia. 2014. The Languages of Dubbing. Mainstream Audiovisual Translation in Italy. Bern: Peter Lang.
52. Pedersen, Jan. 2017. “The FAR Model: Assessing Quality in Interlingual Subtitling.” The Journal of Specialised Translation, no. 28, 210–219. https://jostrans.soap2.ch/issue28/art_pedersen.php.
53. Perego, Elisa. 2016. “History, Development, Challenges and Opportunities of Empirical Research in Audiovisual Translation.” Across Languages and Cultures 17 (2): 155–162. doi:10.1556/084.2016.17.2.1.
54. Pérez-González, Luis. 2007. “Appraising Dubbed Conversation. Systemic Functional Insights into the Construal of Naturalness in Translated Film Dialogue.” The Translator 13 (1): 1–38. doi:10.1080/13556509.2007.10799227.
55. Pettit, Zoë. 2005. “Translating Register, Style and Tone in Dubbing and Subtitling.” The Journal of Specialised Translation, no. 4: 49–65. https://jostrans.soap2.ch/issue04/art_pettit.php.
56. Plevoets, Koen, and Bart Defrancq. 2018. “The Cognitive Load of Interpreters in the European Parliament: A Corpus-based Study of Predictors for the Disfluency uh(m).” Interpreting 20 (1): 1–32. doi:10.1075/intp.00001.ple.
57. Romero-Fresco, Pablo. 2009. “Naturalness in the Spanish Dubbing Language: A Case of Not-so-close Friends.” Meta 54 (1): 49–72. doi:10.7202/029793ar.
58. Romero-Fresco, Pablo. 2011. Subtitling through Speech Recognition: Respeaking. Manchester: St. Jerome.
59. Salazar, Danica. 2011. “Lexical Bundles in Scientific English: A Corpus-based Study of Native and Non-native Writing.” PhD diss., University of Barcelona.
60. Sharma, Kshitij, Sarah D’Angelo, Darren Gergle, and Pierre Dillenbourg. 2016. “Visual Augmentation of Deictic Gestures in MOOC Videos.” Paper presented at 12th International Conference of the Learning Sciences, Singapore, June 20–24.
61. Shi, Jianwei, Christian Otto, Anett Hoppe, Peter Holtz, and Ralph Ewerth. 2019. “Investigating Correlations of Automatically Extracted Multimodal Features and Lecture Video Quality.” In Proceedings of the 1st International Workshop on Search as Learning with Multimedia Information, 11–19. https://dl.acm.org/doi/10.1145/3347451.3356731.
62. Simpson, Rita. 2004. “Stylistic Features of Academic Speech: The Role of Formulaic Expressions.” In Discourse in the Professions: Perspectives from Corpus Linguistics, edited by Ulla Connor and Thomas A. Upton, 37–64. Amsterdam: John Benjamins.
63. Sinclair, John, ed. 1995. Collins Cobuild English Dictionary. Glasgow: HarperCollins.
64. Tremblay, Antoine, Bruce Derwing, Gary Libben, and Chris Westbury. 2011. “Processing Advantages of Lexical Bundles: Evidence from Self-paced Reading and Sentence Recall Tasks.” Language Learning 61 (2): 569–613. doi:10.1111/j.1467-9922.2010.00622.x.
65. Uchidiuno, Judith Odili, Amy Ogan, Evelyn Yarzebinski, and Jessica Hammer. 2018. “Going Global: Understanding English Language Learner’s Student Motivation in English-language MOOCs.” International Journal of Artificial Intelligence in Education 28 (4): 528–552. doi:10.1007/s40593-017-0159-7.
66. Valdéon, Roberto A. 2008. “Inserts in Modern Script-writing and Their Translation into Spanish.” In Between Text and Image. Updating Research in Screen Translation, edited by Delia Chiaro, Christine Heiss, and Chiara Bucaria, 117–132. Amsterdam: John Benjamins.
67. Vorobyeva, Alexandra. 2018. “Language Acquisition Through Massive Open Online Courses (MOOCs), Opportunities and Restrictions in Educational University Environment.” XLinguae 11 (2): 136–146. doi:10.18355/XL.2018.11.02.11.
68. Wang, Pei-Yu, Miao-Chin Chiu, and Yu-Tzu Lee. 2020. “Effects of Video Lecture Presentation Style and Questioning Strategy on Learning Flow Experience.” Innovations in Education and Teaching International 58 (4): 473–483. doi:10.1080/14703297.2020.1754272.
69. Wang, Wei, Lihuan Guio, Ling He, and Yenchun Jim Wu. 2019. “Effects of Social-Interactive Engagement on the Dropout Ratio in Online Learning: Insights from MOOC.” Behaviour & Information Technology 38 (6): 621–636. doi:10.1080/0144929X.2018.1549595.
70. Wang, Ying. 2017. “Lexical Bundles in Spoken Academic ELF.” International Journal of Corpus Linguistics 22 (2): 187–211. doi:10.1075/ijcl.22.2.02wan.
71. Wood, David. 2010. “Lexical Clusters in an EAP Textbook Corpus.” In Perspectives on Formulaic Language: Acquisition and Communication, edited by David Wood, 88–106. London: Continuum.
72. Wray, Allison, and Mick Perkins. 2000. “The Functions of Formulaic Language: An Integrated Model.” Language and Communication 20 (1): 1–28. doi:10.1016/S0271-5309(99)00015-4.
73. Yu, Xiaoli. 2022. “A Multi-dimensional Analysis of English-medium Massive Open Online Courses (MOOCs) Video Lectures in China.” Journal of English for Academic Purposes 55:1–14. doi:10.1016/j.jeap.2021.101079.