Evaluating Large Language Models in Translation: A Theoretical and Practical Analysis Based on Skopos Theory
Öz
The aim of this study is to analyse the translation abilities of large language models, such as GPT-4, through various theoretical lenses within translation studies. This study is unique in its comprehensive evaluation of these models' translation performance based on skopos theory. This research assesses how well large language models align with these theoretical frameworks and their effectiveness in producing contextually appropriate and culturally sensitive translations. The research explores the architecture and operational principles of large language models, explaining their application in translation. Methodologically, the study employs a comparative analysis of translations generated by large language models across language pairs amongst Turkish, English and Spanish. The analysis focuses on key theoretical aspects, such as the purpose and functionality of translations. Additionally, the study examines the cultural and contextual appropriateness of translations generated by large language models, evaluating their ability to maintain cultural nuances and meet the expectations set by the respective translation theories. The findings reveal the strengths and limitations of large language models in adhering to theoretical principles, providing insights into their potential to enhance or challenge traditional translation practices. This research advances the theoretical understanding of machine translation and offers practical recommendations for improving the translation capabilities of large language models. By integrating theoretical analysis with practical applications, the study aims to provide insight into future developments in translation technologies and their role in the future of translation studies.
Anahtar Kelimeler
Large language models, translation theories, neural machine translation, skopos theory, translation technologies
Kaynakça
- Aharoni, Roee, Johnson, Melvin, & Firat, Orhan. (2019). Massively multilingual neural machine translation.
- Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 3874–3884.
- Bender, Emily M., Gebru, Timnit, McMillan-Major, Angelina, & Shmitchell, Shmargaret. (2021). On the dangers of stochastic parrots: Can language models be too big? 🦜. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610–623.
- Fan, Angela, Bhosale, Shruti, Schwenk, Holger, Ma, Xiaoqing, El-Kishky, Ahmed, Goyal, Naman, ... & Edunov, Sergey. (2021). Beyond English-centric multilingual machine translation. Journal of Machine Learning Research, 22(107), 1–48.
- Jiménez-Crespo, Miguel Ángel (2017). Crowdsourcing and Online Collaborative Translations: Expanding the Limits of Translation Studies. John Benjamins Publishing.
- Kocmi, Tom, & Federmann, Christian (2023). Large language models are state-of-the-art evaluators of translation quality.
- Koehn, Philipp (2020). Neural Machine Translation. Cambridge University Press.
- Koehn, Philipp, & Knowles, Rebecca. (2017). Six challenges for neural machine translation. Proceedings of the First Workshop on Neural Machine Translation, 28–39.
