The Opportunities and Burdens of Online Test Processes

Abstract

English language testing has been challenged by the COVID-19 pandemic as it required the test developers to instantly develop and implement online language tests in their unique contexts. In this regard, this study aims to investigate EFL language test writers’ reflections on their planning, development, administration, reporting, and reflection processes of a test in the emergency remote teaching environment and display the potential opportunities and burdens these processes have brought about. The data were collected by utilizing semi-structured interviews with nine test writers who had testing experience in language preparatory classes at three state universities in Turkiye. The results indicated that the test writers were mainly challenged by ensuring the security of the exams, adapting question types to online platforms, and dealing with technical problems which resulted in additional workload. However, amidst these challenges, the study also identified notable opportunities, including the practical application of exam procedures, the digital transformation of exams with a variety of resources, item banking, ecological benefits, and fostering professional development. Overall, this compulsory experience during the pandemic indicated numerous conveniences to enhance language testing and contribute to the development of robust language assessment and testing in the future.

Keywords

• Language testing
• language assessment
• remote teaching
• online testing
• testing processes

Çevrimiçi Sınav Süreçlerinin Ortaya Çıkardığı Fırsat ve Zorluklar

Öz

İngilizce dil sınavı hazırlayanlar, COVID-19 salgını nedeniyle başlayan süreçte beklenmedik bir anda çevrimiçi dil sınavı hazırlamak ve uygulamak zorunda kaldıklarından çeşitli yeniliklerle karşılaştı. Bu yenilikler çeşitli fırsatların yanı sıra bir takım zorluklar da getirmiştir. Bu bağlamda, bu çalışma, yabancı dil olarak İngilizce dil sınavı hazırlayanların uzaktan öğretim ortamında bir sınavı planlama, geliştirme, uygulama, raporlama ve dönüt verme süreçlerine ilişkin tecrübelerini ve yaklaşımlarını araştırmayı hedeflemiş ve bu süreçlerin getirdiği potansiyel fırsat ve yükleri ortaya koymayı amaçlamaktadır. Veriler, Türkiye'deki üç devlet üniversitesindeki dil hazırlık sınıflarında sınav deneyimi olan dokuz sınav yazarıyla yarı yapılandırılmış görüşmeler kullanılarak toplanmıştır. Sonuçlar, test yazarlarının çoğunlukla sınav güvenliğinin sağlanması, soru türlerinin çevrimiçi platformlara uyarlanması ve ek iş yüküne neden olan teknik sorunlarla uğraşmak gibi konularda zorluk yaşadıklarını gösterdi. Ancak bu zorlukların yanında, çeşitli pratik sınav prosedürlerinin uygulaması, sınavların farklı kaynaklarla dijital dönüşümü, soru bankacılığı, ekolojik faydalar ve mesleki gelişimin teşvik edilmesi dahil olmak üzere dikkate değer fırsatları ortaya koymuştur. Genel olarak, pandemi sırasındaki bu zorunlu deneyim, dil testini geliştirmek ve gelecekte sağlıklı dil değerlendirmesi ve testlerinin geliştirilmesine katkıda bulunmak açısından bir çok sayıda yeniliğe işaret etmiştir.

Anahtar Kelimeler

• Yabancı dil sınavları
• ölçme ve değerlendirme
• uzaktan öğretim
• çevrimiçi sınav
• sınav süreçleri

Kaynakça

1. Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests (Vol. 1). Oxford University Press.
2. Bailey, K. M. (1996). Working for washback: A review of the washback concept in language testing. Language Testing, 13(3), 257-279. https://doi.org/10.1177/0265532296013003
3. Brown, H. D. (2004). Language assessment principles and classroom practice. Oxford: Oxford University Press.
4. Bui, T. H. (2022). A Review of Language Testing and Assessment in Online Teaching. International Journal of English Linguistics, 12(4), 54–65. https://doi.org/10.5539/ijel.v12n4p54
5. Cohen, L., Manion, L., & Morrison, K. (2007). Research methods in education (6th ed.). London and New York, NY: Routledge Falmer.
6. Downing, S. M. & Haladyna, T.M., (Eds.). (2011). Handbook of Test Development (1st ed.). Routledge.
7. Fitriyah, I., & Jannah, M. (2021). Online Assessment Effect in EFL Classroom: An Investigation on Students and Teachers' Perceptions. Indonesian Journal of English Language Teaching and Applied Linguistics, 5(2), 265–284. https://doi.org/10.21093/ijeltal.v5i2.709
8. Guangul, F. M., Suhail, A. H., Khalit, M. I., & Khidhir, B. A. (2020). Challenges of remote assessment in higher education in the context of COVID-19: a case study of Middle East College. Educational Assessment Evaluation and Accountability, 32(4), 519–535. https://doi.org/10.1007/s11092-020-09340-w

9. Güngör, M. N., & Güngör, M. A. (2021). Reconceptualizing Testing in Times of COVID-19 from an Activity Theory Lens: A Narrative Inquiry of a Test-designer’s Experiences. The Asia-Pacific Education Researcher, 30(6), 531–539. https://doi.org/10.1007/s40299-021-00627-0
10. Hamad, M. M., Dafaallah, A. S., & Alhaj, A. A. (2021). The challenges of online English language teaching from EFL instructors’ point of view in KKU, Tehama Campus during the COVID-19 period. Journal of Language Teaching and Research, 12(4), 543–556. http://dx.doi.org/10.17507/jltr.1204.04
11. Kucherova, O., & Ushakova, I. (2022). Effectiveness of online testing in general English university courses from teacher and student perspectives. Information Technologies and Learning Tools, 87(1), 185–198. https://doi.org/10.33407/itlt.v87i1.4812
12. Kurniati, D., Riyono, A., & Utomo, B. (2023). Challenges in online assessment: The case of Indonesian teachers. International Journal of Social Science, 2(5), 2063–2068. https://doi.org/10.53625/ijss.v2i5.4811 Leavy, P. (Ed.). (2014). Oxford library of psychology. The Oxford handbook of qualitative research. Oxford University Press.
13. Meiantoni, D., Wiyaka, W., & Prastikawati, E. F. (2021). Online Assessment in English Classroom: EFL Teachers’ Practices and Challenges. Journal of English Education and Linguistics, 2(2), 26–36. https://doi.org/10.56874/jeel.v2i2.532
14. Montenegro-Rueda, M., Luque-de la Rosa, A., Sarasola Sánchez-Serrano, J. L., & Fernández-Cerero, J. (2021). Assessment in higher education during the COVID-19 pandemic: A systematic review. Sustainability, 13(19), 10509. https://doi.org/10.3390/su131910509
15. Naqvi, S., & Zehra, I. (2020). Online EFL emergency remote teaching during COVID-19, Challenges and innovative practices: A case of Oman. In Arab World English Journal (AWEJ) Proceedings of 2nd MEC TESOL Conference.
16. Nunan D. (1992). Research methods in language learning. Cambridge University Press.
17. Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd ed.). Sage Publications, Inc.
18. Stemler, Steven. (2001). An overview of content analysis. Practical assessment, research, and evaluation. 7. [available online at] http://pareonline.net/getvn.asp?v=7&n=17
19. Stoynoff, S., & Chapelle, C. A., (2005). ESOL tests and testing: A resource for teachers and program administrators. Alexandria, VA: TESOL.
20. Şenel, S., & Şenel, H. C. (2021). Remote assessment in higher education during the COVID-19 pandemic. International Journal of Assessment Tools in Education, 181–199. https://doi.org/10.21449/ijate.820140
21. Weber, R. P. (1990). Sage University paper series on quantitative applications in social sciences, No. 07-049. Basic Content Analysis (2nd ed.). Sage Publications, Inc.
22. Wibowo, F. E., & Novita Sari, U. (2021). An Analysis of Online Assessment in Teaching English. PROJECT (Professional Journal of English Education), 4(3), 521–529. https://doi.org/10.22460/project.v4i3.p521-529
23. Yulianto, D., & Mujtahid, N. M. (2021). Online assessment during COVID-19 pandemic: EFL teachers’ perspectives and their practices. JET (Journal of English Teaching), 7(2), 229–242. https://doi.org/10.33541/jet.v7i2.2770
24. Zhang, C., Yan, X., & Wang, J. (2021). EFL teachers’ online assessment practices during the COVID-19 pandemic: Changes and mediating factors. The Asia-Pacific Education Researcher, 30, 499-507. https://doi.org/10.1007/s40299-021-00589-3