Makine Çevirisi Sonrası Düzenlemede Çaba: Uzmanlığın Rolü

Abstract

Makine çevirisi sonrası düzenleme, çeviri sürecinin giderek daha önemli bir parçası hâline gelmekte ve verimliliği artırmaktadır. Teknolojinin temel amacının insan çabasını azaltmak olması sebebiyle, “çaba” kavramı bu bağlamda pek çok araştırmanın konusu olmuştur. Bu çalışma, makine çevirisi sonrası düzenlemede uzmanlık ve çaba arasındaki ilişkiyi inceleyerek şu soruları cevaplamayı hedeflemektedir: (1) Uzmanlık, makine çevirisi sonrası düzenlemede bilişsel, zamansal ve teknik/dilsel çabanın geçerli bir göstergesi olarak ne ölçüde kullanılabilir? (2) Uzmanlık, nesnel ve öznel çaba göstergeleri ve düzenlemeden geçirilmiş metnin doğruluğu arasındaki ilişki nedir? Bu soruların yanıtlanması amacıyla deneyimli çevirmenler, deneyimsiz çevirmenler ve alan uzmanlarından oluşan üç gruptan toplam 21 katılımcı ile karma yöntemli bir deney gerçekleştirilmiştir. Katılımcılardan, makine çevirisi aracıyla İngilizceden Türkçeye çevrilmiş üç hukuk metnini son düzenlemeden geçirmeleri istenmiştir. Düzenleme süreci esnasında katılımcılara ait toplam görev süresi, ortalama duraklama süresi, toplam duraklama süresi, duraklama yüzdesi, kelime başına duraklama, metin üretimi ve metin silimi, dakika başına kullanıcı ve üretim eylemleri gibi ölçütler klavye tuşlama kaydı yazılımı (Translog-II) ile kaydedilmiştir. Ayrıca düzenlemeden geçirilmiş metinlerin doğruluğu, katılımcıların öznel bildirimleri ve geriye dönük görüşlerin alındığı röportajlar da toplanmıştır. Elde edilen bulgular, deneyimli çevirmenlerin deneyimsiz çevirmenlere kıyasla daha fazla bilişsel ve zamansal çaba harcadıklarını ve deneyimsiz çevirmenler ile alan uzmanlarından daha yüksek teknik ve dilsel çaba sergilediklerini ortaya koymuştur.

Keywords

• Makine çevirisi sonrası düzenleme
• bilişsel çaba
• zamansal çaba
• teknik çaba
• uzmanlık

Effort in Machine Translation Post-Editing: The Role of Expertise

Abstract

Machine translation post-editing is becoming an increasingly important part of the translation process, boosting efficiency. As the primary aim of technology is to reduce human effort, the concept of effort has been widely investigated in this context. This study examines the relationship between expertise and effort in MTPE, addressing the following two questions: (1) To what extent can expertise serve as a valid indicator of cognitive, temporal, and technical/linguistic effort in MTPE? (2) What is the relationship between expertise, objective and self-reported effort, and post-editing accuracy? A mixed-methods experiment was conducted with 21 participants divided into three groups: experienced translators, inexperienced translators, and field experts. They were asked to post-edit three legal documents that had been machine-translated from English to Turkish. Their post-editing processes were recorded using keylogging software (Translog-II), which measured total task duration, mean pause duration, total pause duration, pause percentage, pauses per word, text production and elimination, and user and production events per minute. Post-editing accuracy, self-reports and retrospective think-aloud records were also collected. The findings revealed that experienced translators exerted higher cognitive and temporal effort than inexperienced translators, as well as higher technical and linguistic effort than inexperienced translators and field experts combined.

Keywords

• machine translation post-editing (MTPE)
• cognitive effort
• temporal effort
• technical effort
• expertise

References

1. Ateşman, E. (1997). Türkçe’de okunabilirliğin ölçülmesi. Dil Dergisi, 58, 71–74.
2. Carl, M. (2012, May). Translog-II: A program for recording user activity data for empirical reading and writing research. In N. Calzolari, K. Choukri, T. Declerck, M. U. Doğan, B. Maegaard, J. Mariani, A. Moreno, J. Odijk, & S. Piperidis (Eds.), Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12) (pp. 4108-4112). European Language Resources Association.
3. Dong, D., & Chen, M.-L. (2025). Metacognitive strategies in translation: A comparative study of student and professional translators. Humanities and Social Sciences Communications, 12(1), 890. https://doi.org/10.1057/s41599-025-05289-7
4. Dragsted, B. (2004). Segmentation in translation and translation memory systems: An empirical investigation of cognitive segmentation and effects of integrating a TM system into the translation process (Doctoral dissertation, Copenhagen Business School). Samfundslitteratur.
5. Dragsted, B., Hansen, I. G., & Sørensen, H. S. (2009). Experts exposed. In F. Alves & S. Göpferich (Eds.), Methodology, Technology and Innovation in Translation Process Research (pp. 293–317). Samfundslitteratur.
6. Ehrensberger-Dow, M., Albl-Mikasa, M., Andermatt, K., Hunziker Heeb, A., & Lehr, C. (2020). Cognitive load in processing ELF: Translators, interpreters, and other multilinguals. Journal of English as a Lingua Franca, 9(2), 217–238. https://doi.org/10.1515/jelf-2020-2039
7. Eszenyi, R. (2025). Post-editing van machinevertaling: Is dat een vorm van vertaling of revisie? Roczniki Humanistyczne, 73(10, Zeszyt specjalny), 15–33. https://doi.org/10.18290/rh257310.1s
8. Gieshoff, A. C., & Heeb, A. H. (2023). Cognitive load and cognitive effort: Probing the psychological reality of a conceptual difference. Translation, Cognition & Behavior, 6(1), 3–28. https://doi.org/10.1075/tcb.00073.gie

9. Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. In Advances in psychology (Vol. 52, pp. 139–183). Elsevier. https://doi.org/10.1016/S0166-4115(08)62386-9
10. Herbig, N., Pal, S., Krüger, A., & van Genabith, J. (2021). Multi-modal estimation of cognitive load in post-editing of machine translation. In O. Czulo, S. Hansen-Schirra, R. Rapp, & M. Bisiada (Eds.), Translation, interpreting, cognition: The way out of the box (pp. 1–32). Language Science Press. https://doi.org/10.5281/ZENODO.4544686
11. Hutchins, W. J. (2001). Machine translation and human translation: in competition or in complementation? International Journal of Translation, 13(1–2), 5–20.
12. Ibrahim, S. N. (2025). On assessing the accuracy of Arabic-English translation by machine and human. In M. H. Al Aqad (Ed.), Advances in computational intelligence and robotics (pp. 119–146). IGI Global. https://doi.org/10.4018/979-8-3373-0060-3.ch005
13. Immonen, S. (2006). Translation as a writing process: Pauses in translation versus monolingual text production. Target. International Journal of Translation Studies, 18(2), 313–336. https://doi.org/10.1075/target.18.2.06imm
14. Jakobsen, A. L. (2003). Effects of think aloud on translation speed, revision, and segmentation. In F. Alves (Ed.), Triangulating translation: Perspectives in process-oriented research (pp. 69–95). John Benjamins Publishing Company.
15. Kincaid, J. P., Fishburne, R. P., Jr., Rogers, R. L., & Chissom, B. S. (1975). Derivation of new readability formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy enlisted personnel (Research Branch Report 8-75). Chief of Naval Technical Training, Naval Air Station Memphis.
16. Koponen, M. (2016). Is machine translation post-editing worth the effort? A survey of research into post-editing and effort. The Journal of Specialised Translation, 25, 131–148.
17. Koponen, M., Salmi, L., & Nikulin, M. (2019). A product and process analysis of post‑editor corrections on neural, statistical and rule‑based machine translation output. Machine Translation, 33(1), 61–90. https://doi.org/10.1007/s10590-019-09228-7
18. Koponen, M., Sulubacak, U., Vitikainen, K., & Tiedemann, J. (2020, November). MT for subtitling: User evaluation of post-editing productivity. In A. Martins, H. Moniz, S. Fumega, B. Martins, F. Batista, L. Coheur, C. Parra, I. Trancoso, M. Turchi, A. Bisazza, J. Moorkens, A. Guerberof, M. Nurminen, L. Marg, & M. L. Forcada (Eds.), Proceedings of the 22nd Annual Conference of the European Association for Machine Translation (EAMT 2020) (pp. 115–124). European Association for Machine Translation.
19. Krings, H. P. (1986). Was in den Köpfen von Übersetzern vorgeht. Gunter Narr.
20. Krings, H. P. (2001). Repairing texts: Empirical investigations of machine translation post-editing processes. Kent State University Press.
21. Kumpulainen, M. (2015). On the operationalisation of ‘pauses’ in translation process research. Translation & Interpreting, 7(1), 47–58.
22. Lacruz, I. (2017). Cognitive effort in translation, editing, and post‐editing. In J. W. Schwieter & A. Ferreira (Eds.), The handbook of translation and cognition (pp. 386–401). Wiley.
23. Lacruz, I., Shreve, G. M., & Angelone, E. (2012, October 28). Average pause ratio as an indicator of cognitive effort in post-editing: A case study. In S. O’Brien, M. Simard, & L. Specia (Eds.), Proceedings of the Workshop on Post-Editing Technology and Practice, AMTA 2012 (pp. 1-10). Association for Machine Translation in the Americas.
24. Lörscher, W. (1991). Thinking-aloud as a method for collecting data on translation processes. In S. Tirkkonen-Condit (Ed.), Empirical research in translation and intercultural studies (pp. 67–77). Gunter Narr.
25. Muñoz Martín, R. (2014). Situating translation expertise: a review with a sketch of a construct. In J. W. Schwieter & A. Ferreira (Eds.), The development of translation competence: Theories and methodologies from psycholinguistics and cognitive science (pp. 2–54). Cambridge Scholars Publishing.
26. Mutta, M. (2016). Pausal behavior in the writing processes of foreign and native language writers: The importance of defining the individual pause length. In S. Plane, C. Bazerman, F. Rondelli, C. Donahue, A. N. Applebee, C. Boré, P. Carlino, M. Marquilló Larruy, P. Rogers, & D. Russell (Eds.), Recherches en écriture: Regards pluriels (Recherches textuelles, 13, pp. 583–604). Centre de Recherche sur les Médiations (Crem), Université de Lorraine.
27. O’Brien, S. (2006). Pauses as indicators of cognitive effort in post-editing machine translation output. Across Languages and Cultures, 7(1), 1–21. https://doi.org/10.1556/Acr.7.2006.1.1
28. O’Brien, S. (2011). Towards predicting post-editing productivity. Machine Translation, 25(3), 197–215. https://doi.org/10.1007/s10590-011-9096-7
29. O’Brien, S. (2022). How to deal with errors in machine translation: post-editing. In D. Kenny (Ed.), Machine translation for everyone: Empowering users in the age of artificial intelligence (pp. 105–120). Language Science Press.
30. O’Brien, S., Ehrensberger-Dow, M., Connolly, M., & Hasler, M. (2017). Irritating CAT Tool Features that Matter to Translators. HERMES - Journal of Language and Communication in Business, 56, 145–162. https://doi.org/10.7146/hjlcb.v0i56.97229
31. Paradowska, U., Sycz-Opon, J., & Badziński, A. (2025). MT use by professional translators in Poland – survey results. Perspectives, 1–22. https://doi.org/10.1080/0907676X.2025.2502937
32. Persky, A. M., & Robinson, J. D. (2017). Moving from novice to expertise and its implications for instruction. American Journal of Pharmaceutical Education, 81(9), 6065. https://doi.org/10.5688/ajpe6065
33. Popović, M., Lommel, A., Burchardt, A., Avramidis, E., & Uszkoreit, H. (2014, June). Relations between different types of post-editing operations, cognitive effort and temporal effort. In M. Cettolo, M. Federico, L. Specia, & A. Way (Eds.), Proceedings of the 17th Annual Conference of the European Association for Machine Translation (pp. 191–198). European Association for Machine Translation.
34. R Core Team. (2025). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. https://www.R-project.org
35. Revelle, W. (2018). psych: Procedures for personality and psychological research (Version 1.8.12) [R package]. https://CRAN.R-project.org/package=psych
36. Rico Pérez, C. (2024). Re-thinking machine translation post-editing guidelines. The Journal of Specialised Translation, 41, 26–47. https://doi.org/10.26034/cm.jostrans.2024.4696
37. Risku, H. (2002). Situatedness in translation studies. Cognitive Systems Research, 3(3), 523–533. https://doi.org/10.1016/S1389-0417(02)00055-4
38. Robson, S. G., Tangen, J. M., & Searston, R. A. (2021). The effect of expertise, target usefulness and image structure on visual search. Cognitive Research: Principles and Implications, 6(1), 16. https://doi.org/10.1186/s41235-021-00282-5
39. Ruokonen, M., & Koskinen, K. (2017). Dancing with technology: Translators’ narratives on the dance of human and machinic agency in translation work. The Translator, 23(3), 310–323. https://doi.org/10.1080/13556509.2017.1301846
40. Sánchez-Torrón, M., Ipek, E., & Raído, V. E. (2025). Creating terminological correspondence recognition tests with GPT-4: a case study in English-to-Turkish translations in the engineering domain. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-025-00465-x
41. Šarčević, S. (1997). New approach to legal translation. Kluwer Law International.
42. Sharma, S., Diwakar, M., Singh, P., Singh, V., Kadry, S., & Kim, J. (2023). Machine translation systems based on classical-statistical-deep-learning approaches. Electronics, 12(7), 1716. https://doi.org/10.3390/electronics12071716
43. Weng, Y., & Zheng, B. (2024). Automaticity in translation: Effects of time pressure and work experience. Meta, 69(2), 355–379. https://doi.org/10.7202/1118382ar
44. Yanmei, L., & Mingmei, D. (2014). A multiple case study of Chinese-English translation strategies. Studies in Literature and Language, 9(3), 58–65. http://dx.doi.org/10.3968/6098