A Study on the Quality Assessment of the Chinese-English Subtitle Translation among Four Machine Translation Tools - A Case Study of Wreaths at the Foot of the Mountain

Year 2024, Volume: 7 Issue: 2, 29 - 56, 31.12.2024

Xu Zhou , Jinghua Zhou

https://doi.org/10.29228/transLogos.68

Abstract

As artificial intelligence (AI) continues to evolve, machine translation (MT), particularly neural MT, is experiencing rapid development and innovation, significantly influencing leading MT applications in China. This evolution comes at a pivotal moment when Chinese films are going global at a faster pace, highlighting the importance of effective subtitle translation. However, despite these advancements, there remains a considerable gap in the quality of subtitle translations. The existing two major subtitle translation assessment models—Multidimensional Quality Metrics (MQM) and Functional Equivalence, Acceptability, and Readability (FAR)—face several challenges that hinder their effectiveness. In light of these issues, this study aims to integrate the strengths of both MQM and FAR to develop an enhanced assessment framework, referred to as the FAR 2.0 model. This new model will facilitate a more comprehensive comparison and analysis of four prominent MT applications discussed in this paper, focusing on three critical aspects: functional equivalence (F), acceptance (A), and readability (R). By applying this model, the study seeks to illuminate the respective strengths and weaknesses of each MT tool, offering insights that can guide quality improvements in subtitle translation both now and in the future. Ultimately, this research not only aims to enhance the understanding of current MT capabilities, but also strives to contribute to the broader goal of elevating the global competitiveness of Chinese films through improved subtitle quality.

Keywords

machine translation, subtitle translation, MQM, FAR 2.0 model, MT tool

References

• Artetxe, Mikel, Gorka Labaka, and Eneko Agirre. 2019. “An Effective Approach to Unsupervised Machine Translation.” In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 194–203. doi:10.18653/v1/P19-1019.
• Chyxx Company. 2023. “2023-2029 Nian Zhongguo Jiqi Fanyi Hangye Shichang Yunyin Taishi Ji Touzi Zhanlue Guihua Baogao.” [2023-2029 China machine translation industry market operation situation and investment strategy planning report.] July 24. Accessed May 5, 2024. https://www.sohu.com/a/708793117_120950077.
• Feng, Yang, and Chenze Shao. 2020. “Shenjing Jiqi Fanyi Qianyan Zongshu.” [Frontiers in neural machine translation: A literature review]. Journal of Chinese Information Processing 34 (7): 1–18. http://jcip.cipsc.org.cn/CN/Y2020/V34/I7/1.
• Garg, Sarthak, Stephan Peitz, Udhyakumar Nallasamy, and Matthias Paulik. 2019. “Jointly Learning to Align and Translate with Transformer Models.” In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 4453–4462. doi:10.48550/arXiv.1909.02074.
• Goyal, Naman, Cynthia Gao, Vishrav Chaudhary, Peng-Jen Chen, Guillaume Wenzek, Da Ju, Sanjana Krishnan, Marc’Aurelio Ranzato, Francisco Guzman, and Angela Fan. 2021. “The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation.” doi:10.48550/arXiv.2106.03193.
• Hendy, Amr, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, Hany Hassan Awadalla. 2023. “How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation.” doi:10.48550/arXiv.2302.09210.
• Huang, Youyi. 2022. Cong Fanyi Shijie Dao Fanyi Zhongguo [From translating the world to translating China]. Beijing: Foreign Languages Press.
• Li, Fengqi. 2021. “Jiyu Shenjing Wangluo De Zaixian Jiqi Fanyi Xitong Yinghan Huyi Zhiliang Duibi Yanjiu.” [A comparative study on the quality of English-Chinese translation of online machine translation systems based on neural networks.] Shanghai Journal of Translators, no. 4, 46–52. doi:10.3969/j.issn.1672-9358.2021.04.013.
• Linfeng, Song, Daniel Gildea, Yue Zhang, Zhiguo Wang, and Jinsong Su. 2019. “Semantic Neural Machine Translation Using AMR.” Transactions of the Association for Computational Linguistics, no. 7, 19–31. doi:10.1162/tacl_a_00252.
• Liu, Jichao, Shisheng Lü, and Chengpan Liu. 2024. “Shenjing Wangluo Jiqi Fanyi De Renzhi Yinyu Jiegou.” [Deconstructing cognitive metaphors in neural network machine translation.] Translation Research and Teaching, no 1, 82–88. https://d.wanfangdata.com.cn/periodical/fyyjyjx202401013.
• Lommel, Arle Richard, Aljoscha Burchardt, and Hans Uszkoreit. 2013. “Multidimensional Quality Metrics: A Flexible System for Assessing Translation Quality.” In Proceedings of Translating and the Computer 35. London: Aslib. https://aclanthology.org/2013.tc-1.6.pdf.
• Mathur, Nitika, Timothy Baldwin, and Trevor Cohn. 2020. “Tangled Up In BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics.” In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 4984–4997. doi:10.18653/v1/2020.acl-main.448.
• Pedersen, Jan. 2017. “The FAR Model: Assessing Quality in Interlingual Subtitling.” Journal of Specialised Translation, no. 28. https://jostrans.soap2.ch/issue28/art_pedersen.php.
• Popel, Martin, Marketa Tomkova, Jakub Tomek, Łukasz Kaiser, Jakob Uszkoreit, Ondřej Bojar, and Zdeněk Žabokrtský. 2020. “Transforming Machine Translation: A Deep Learning System Reaches News Translation Quality Comparable to Human Professionals.” Nature Communications, no. 11. doi:10.1038/s41467-020-18073-9.
• Stahlberg, Felix. 2020. “Neural Machine Translation: A Review.” Journal of Artificial Intelligence Research, no. 69, 343–418. doi:10.1613/jair.1.12007.
• Wang, Jinquan, and Bojia He. 2024. “Jiyu Shenjing Wangluo Moxing De Fanyi Yuyi Zhiliang Lianghua Pingjia.” [Quantified assessment of translation semantic quality based on neural network model.] Chinese Foreign Languages 21 (1): 92–101. https://benjamins.com/online/etsb/publications/58188.
• Wen, Xu, and Yaling Tian. 2024. “ChatGPT Yingyong Yu Zhongguo Tese Huayu Fanyi De Youxiaoxing Yanjiu.” [The effectiveness of ChatGPT in translating China-specific discourse text.] Shanghai Journal of Translators 39 (2): 27–34+94–95. doi:10.3969/j.issn.1672-9358.2024.02.005.
• Wreaths at the Foot of the Mountain. 1984. Directed by Xie Jin. Shanghai Film Studios. War picture.
• Xiao, Weiqing. 2017. Yinghan Yingshi Fanyi Shiyong Jiaocheng [A practical guide to English-Chinese audiovisual translation]. Shanghai: East China University of Science and Technology Press. Xu, Wei. 2024. “Jiyu Shendu Xuexi De Nongji Jiqi Yingyu Yuliaoku De Sheji.” [Design of an English corpus for agricultural machines based on deep learning.] Journal of Agricultural Mechanization Research 46 (10): 208–212. doi:10.3969/j.issn.1003-188X.2024.10.035.
• Zhang, Biao, Philip Williams, Ivan Titov, and Rico Sennrich. 2020. “Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation.” In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 1628–1639. doi:10.18653/v1/2020.acl-main.148.
• Zhang, Wenbo, Xinlu Zhang, and Yating Yang. 2021. “Mianxiang Diziyuan Shenjing Jiqi Fanyi De Huiyi Fangfa.” [Back translation for low resources neural machine translation.] Journal of Xiamen University (Natural Science) 60 (4): 675–679. doi:10.6043/j.issn.0438-0479.202011025.
• Zhou, Xinghua, and Chuanying Wang. 2020. “Rengongzhineng Jishu Zai Jisuanji Fuzhu Fanyi Ruanjian Zhong De Yingyong Yu Pingjia.” [Application and evaluation of artificial intelligence technology in computer-aided translation software.] Chinese Translators Journal 41 (5): 121–129. https://qikan.cqvip.com/Qikan/Article/Detail?id=7102938325. Filmography