Scaling Students’ Self-Efficacy on Machine Translation Post-Editing: Psychometric Properties of the Scale and Their Associations

Year 2023, Volume: 10 Issue: 6, 229 - 244, 01.11.2023

Qing Li Tai-yi Huang

https://doi.org/10.17275/per.23.98.10.6

Abstract

Machine Translation Post-Editing (MTPE) has emerged as a productivity-enhancing practice in the language service industry, where human editors correct the output of machine translation systems. To ensure that students of translation possess the necessary skills for MTPE, it is essential to understand their self-efficacy in this domain. This research paper aims to assess students' self-efficacy in translation learning, specifically in the context of MTPE, and explore the factor structure, psychometric properties, and internal associations of their self-efficacy. The study utilized a modified survey adapted from the Scale for Assessing Translators' Self-Efficacy and collected responses from 65 undergraduate students in a Chinese university. The survey data underwent reliability and validity analyses, including exploratory factor analysis, to assess the measurement tool's consistency, stability, and construct validity. The results indicated a high reliability of the scale (Cronbach's Alpha = 0.914) and revealed three primary dimensions of self-efficacy: Decision-making of MTPE, Communicative Competence of MTPE, and Strategic Competence of MTPE, and the strong inter-correlations suggests that they collectively measure the construct of translators' self-efficacy of MTPE, providing insights into the skills and abilities required for effective MTPE. The findings contribute to the development of psychometric tools for further research in translation and promote pedagogical reform to align with evolving market trends emphasizing human-machine collaborative translation.

Keywords

self-efficacy, machine translation post-editing, psychometric investigation

References

• Albin, J. (2010). Being a Translator: How Does It Feel. In Translation and Meaning, Part 10, Proceedings of the Łodź Session of the 5th International Maastricht-Łodź Duo Colloquium on “Translation and Meaning (pp. 16-19).
• Atkinson, D. (2012). Freelance translator success and psychological skill: a study of translator competence with perspectives from work psychology (Doctoral dissertation, ResearchSpace@ Auckland).
• Bandura, A. (1995). Exercise of personal and collective efficacy in changing societies. Self-efficacy in changing societies, 15, 334.
• Bandura, A. (2006). Guide for constructing self-efficacy scales. Self-efficacy beliefs of adolescents, 5(1), 307-337.
• Bandura, A., & Walters, R. H. (1977). Social learning theory (Vol. 1). Prentice Hall: Englewood cliffs.
• Bandura, A., & Wessels, S. (1994). Self-efficacy (Vol. 4, pp. 71-81). na.
• Bar-Hillel, Y. (1960). The present status of automatic translation of languages. Advances in computers, 1, 91-163.
• Bolaños-Medina, A. (2014). Self-efficacy in translation. Translation and Interpreting Studies. The Journal of the American Translation and Interpreting Studies Association, 9(2), 197-218.
• Bolaños-Medina, A., & Núñez, J. L. (2018). A preliminary scale for assessing translators’ self-efficacy. Across Languages and Cultures, 19(1), 53–78. doi:10.1556/084.2018.19.1.3
• Dunn-Rankin, P., Knezek, G. A., Wallace, S. R., & Zhang, S. (2014). Scaling Methods. Taylor & Francis.
• Dziuban, C. D., & Shirkey, E. C. (1974). When is a correlation matrix appropriate for factor analysis? Some decision rules. Psychological bulletin, 81(6), 358.
• EAMT. (n.d.). What is machine translation?. European Association for Machine Translation. https://eamt.org/what-is-machine-translation/
• Gaspari, F., Almaghout, H., & Doherty, S. (2015). A survey of machine translation competences: Insights for translation technology educators and practitioners. Perspectives, 23(3), 333-358.
• Gudmundsson, E. (2009). Guidelines for translating and adapting psychological instruments. Nordic Psychology, 61(2), 29-45.
• Guerberof, A. (2009). Productivity and quality in MT post-editing. In Beyond Translation Memories: New Tools for Translators Workshop.
• Guerberof, A., & Moorkens, J. (2019). Machine translation and post-editing training as part of a master’s programme. Jostrans: The Journal of Specialised Translation, (31), 217-238.
• Hair, J. F., Anderson, R. E., Tatham, R. L., & William, C. (1998). Multivariate data analysis. Upper Saddle River., New Jersey: Prentice Hall.
• Haro-Soler, M. (2018). Self-confidence and its role in translator training. John Benjamins Publishing Company, 131.
• Heale, R., & Twycross, A. (2015). Validity and reliability in quantitative studies. Evidence-based nursing, 18(3), 66-67.
• Hendy, A., Abdelrehim, M., Sharaf, A., Raunak, V., Gabr, M., Matsushita, H., ... & Awadalla, H. H. (2023). How good are GPT models at machine translation? a comprehensive evaluation. arXiv preprint arXiv:2302.09210.
• Holmes, J. S. (2000). THE NAME AND NATURE OF TRANSLATION STUDIES¹. The translation studies reader, 172.
• Hutchins, J. (2006). Machine translation: history of research and use. Encyclopedia of Languages and Linguistics. 2a. edição, editado por Keith Brown (Oxford: Elsevier 2006), 7, 375-383.
• Hutchins, J. & Somers, H. L. (2009). An introduction to machine translation.
• Jääskeläinen, R. (2012). Translation psychology. Handbook of translation studies, 3, 191-197.
• Krings, H. P. (2001). Repairing texts: Empirical investigations of machine translation post-editing processes (Vol. 5). Kent State University Press.
• Koponen, M. (2016). Is machine translation post-editing worth the effort? A survey of research into post-editing and effort. The Journal of Specialized Translation, 25(2).
• Lacruz, I. (2017). Cognitive effort in translation, editing, and post‐editing. The handbook of translation and cognition, 386-401.
• Lacruz, I., Denkowski, M., & Lavie, A. (2014). Cognitive demand and cognitive effort in post-editing. In Proceedings of the 11th Conference of the Association for Machine Translation in the Americas (pp. 73-84).
• Levý, J. (2000). Translation as a decision process. The translation studies reader, 148-159.
• Lukat, J., Margraf, J., Lutz, R., van der Veld, W. M., & Becker, E. S. (2016). Psychometric properties of the positive mental health scale (PMH-scale). BMC psychology, 4, 1-14.
• Mellinger, C. D. (2017). Translators and machine translation: knowledge and skills gaps in translator pedagogy. The Interpreter and Translator Trainer, 11(4), 280-293.
• Moorkens, J., Toral, A., Castilho, S., & Way, A. (2018). Translators’ perceptions of literary post-editing using statistical and neural machine translation. Translation Spaces, 7(2), 240-262.
• Nitzke, J. (2019). Problem solving activities in post-editing and translation from scratch: A multi-method study. Language Science Press.
• Núñez, J. L., & Bolaños-Medina, A. (2018). Predictors of problem-solving in translation: Implications for translator training. The Interpreter and Translator Trainer, 12(3), 282-298.
• Pugh, J. (2002). The story so far: an evaluation of machine translation in the world today. In Computers in Translation (pp. 34-52). Routledge.
• Quah, C. K. (2006). Translation and Technology. Springer.
• Reiss, K. (1981). Type, kind and individuality of text: Decision making in translation. Poetics Today, 2(4), 121-131.
• Sedgwick, P. (2012). Pearson’s correlation coefficient. Bmj, 345.
• Slocum, J. (1985). A survey of machine translation: Its history, current status and future prospects. Computational linguistics, 11(1), 1-17.
• Tavakol, M., & Wetzel, A. (2020). Factor Analysis: a means for theory and instrument development in support of construct validity. International journal of medical education, 11, 245.
• Trope, Y. (1982). Self-assessment and task performance. Journal of Experimental Social Psychology, 18(2), 201-215.
• Veale, T., & Way, A. (1997). Gaijin: A bootstrapping, template-driven approach to example-based MT. In Proc. of the NeMNLP97.
• Williams, B., Onsman, A., & Brown, T. (2010). Exploratory factor analysis: A five-step guide for novices. Australasian journal of paramedicine, 8, 1-13.
• Wilss, W. (1996). Translation as intelligent behaviour. Benjamins Translation Library, 18, 161-168.
• Winship, C., & Mare, R. D. (1984). Regression models with ordinal variables. American sociological review, 512-525.
• Wu, D., Wei, L., & Mo, A. (2019). Training translation teachers in an initial teacher education programme: a self-efficacy beliefs perspective. Perspectives, 27(1), 74-90.
• Yang, X., Guo, X., & Yu, S. (2016). Effects of cooperative translation on Chinese EFL student levels of interest and self-efficacy in specialized English translation. Computer Assisted Language Learning, 29(3), 477-493.