Paper-and-pencil vs Online Assessment in Distance Education: Examining EFL Students’ Preferences and Challenges in a Mozambican Tertiary Institution / Uzaktan Eğitimde Kâğıt-Kalem ve Çevrim İçi Değerlendirme: Mozambik'teki Bir Yükseköğretim Kurumunda İngilizceyi Yabancı Dil Olarak Öğrenen Öğrencilerin Tercihleri ve Karşılaştıkları Zorlukların İncelenmesi

Abstract

This study aimed to examine the phenomenon of Distance Learning Assessment (DLA), in an English Language Teaching (ELT) Undergraduate Degree, at a public university in Zambézia Province, Mozambique. The purposes of this study were (1) to find out students’ preferences related to the assessment systems in use at a public higher education institution, and (2) analyse the challenges which they face in an online and paper-and-pencil testing system. To achieve these purposes, an explanatory sequential mixed-methods design was used. Therefore, quantitative data were obtained from four tests measuring 130 students’ progress in the topics previously discussed, and using descriptive statistics and a T-test, their scores from the two testing systems were presented and compared. Furthermore, a qualitative open-ended web-based questionnaire was employed to explore participants’ preferences and challenges in distance learning assessment systems. The findings indicate statistically significant differences between paper-and-pencil test scores and online test scores, with participants performing significantly better in the online testing format. Regarding their preference, most of the English as a Foreign language (EFL) students majoring in ELT prefer the online testing system due to its numerous advantages. However, its main challenges included internet service/s, the time the tests took place, and the availability of Moodle learning management system. Regarding the paper-and-pencil testing system, the major challenge reported was the travelling and accommodation costs involved since these tests take place at the resource centres. In alignment with these results, distance education stakeholders should take further steps towards improving the testing systems in use at this institution.

Keywords

• Distance Learning
• Testing systems
• Paper-and-pencil assessment
• online assessment
• ELT/ EFL
• Mixed-methods
• Mozambique

Uzaktan Eğitimde Kâğıt-Kalem ve Çevrim İçi Değerlendirme: Mozambik'teki Bir Yükseköğretim Kurumunda İngilizceyi Yabancı Dil Olarak Öğrenen Öğrencilerin Tercihleri ve Karşılaştıkları Zorlukların İncelenmesi /Paper-and-pencil vs Online Assessment in Distance Education: Examining EFL Students’ Preferences and Challenges in a Mozambican Tertiary Institution

Öz

Bu çalışma, Mozambik'in Zambézia eyaletindeki bir devlet üniversitesinde, İngilizce Öğretmenliği lisans programı çerçevesinde Uzaktan Eğitimde Değerlendirme (UED) olgusunu incelemeyi amaçlamıştır. Bu çalışmanın amaçları (1) öğrencilerin bir devlet yükseköğretim kurumunda kullanılan değerlendirme sistemlerine ilişkin tercihlerini ortaya çıkarmak ve (2) çevrimiçi ve kâğıt-kalem temelli ölçme sistemlerinde karşılaştıkları zorlukları incelemektir. Bu amaçlara ulaşmak için açımlayıcı sıralı karma yöntem tasarımı kullanılmıştır. Bu bağlamda, 130 öğrencinin daha öğrenilen konulardaki performanslarını ölçen dört farklı sınavdan nicel veriler elde edilmiş ve betimleyici istatistikler ve t-testi kullanılarak iki ölçme sisteminden aldıkları puanlar karşılaştırılmıştır. Ayrıca, katılımcıların uzaktan eğitim değerlendirme sistemlerine yönelik tercihlerini ve bu sistemde karşılaştıkları zorlukları araştırmak için açık uçlu web tabanlı bir anket aracılığı ile nitel veriler toplanmıştır. Bulgular, kâğıt-kalem testi puanları ile çevrimiçi test puanları arasında istatistiksel olarak anlamlı farklılıklar olduğunu göstermektedir ve katılımcılar çevrimiçi test formatında belirgin şekilde daha iyi performans sergilemiştir. Tercihleri konusunda ise, Yabancı Dil Olarak İngilizce (YDİ) öğrencilerinin çoğu, sayısız avantajı nedeniyle çevrimiçi sınav sistemini tercih etmektedir. Bununla birlikte, ana zorlukları arasında internet hizmeti / hizmetlerinde karşılaşılan sıkıntılar, sınavların yapıldığı zaman dilimi ve Moodle öğrenme yönetim sisteminin kullanılabilirliğine ilişkin sıkıntılar yer almaktadır. Kâğıt-kalem sınav sistemiyle ilgili olarak bildirilen en büyük zorluk ise, sınavların belirli sınav merkezlerinde yapılmasından dolayı ortaya çıkan seyahat ve konaklama masraflarıdır. Bu sonuçlarla uyumlu olarak, konu ile ilgili uzaktan eğitim paydaşları, kullanılan sınav sistemlerini iyileştirmeye yönelik daha fazla adım atmalıdır.

Anahtar Kelimeler

• Uzaktan Eğitim
• Sınav sistemleri
• Kâğıt-kalem ile değerlendirme
• çevrimiçi değerlendirme
• Yabancı Dil olarak İngilizce (YDİ)
• karma yöntemler
• Mozambik

References

1. Afacan Adanır, G., İsmailova, R., Omuraliev, A., & Muhametjanova, G. (2020). Learners’ perceptions of online exams: A comparative study in Turkey and Kyrgyzstan. The International Review of Research in Open and Distributed Learning, 21(3), 1-17. https://doi.org/10.19173/irrodl.v21i3.4679
2. Alberto, A. C., & Tumbo, D. L. (2022). O programa de ensino secundário a distancia na escola secundaria Quisse Mavota, cidade de Maputo: Entre realidades e desafios na usabilidade pedagógica das tecnologias [The distance secondary education program at Quisse Mavota secondary school, Maputo city: Between realities and challenges in the pedagogical usability of technologies]. Revista Internacional em Políticas, Currículo, Práticas e Gestão da Educação, 1(2), 198-215. https://doi.org/10.29327/235555
3. Al-Ghazo, A. (2023). Authentic assessment strategies used to improve distance learning outcomes. Journal of Applied Linguistics and Language Research, 10(2), 16-28.
4. Amin, F. M., & Sundari, H. (2020). EFL students’ preferences on digital platforms during emergency remote teaching: Video conference, LMS, or messenger application? Studies in English Language and Education, 7(2), 362-378. https://doi.org/10.24815/siele.v7i2.16929
5. Aquami, A., Husni, M., Bujuri, D. A., Khodijah, N., Darmayanti, K. K. H., Amilda, A., Baiti, M., Anggraini, E., & Malacapay, M. C. (2024). Elementary school students’ learning difficulties on distance learning during COVID-19: The psychological approach. International Journal of Evaluation and Research in Education (IJERE), 13(3), 1797. https://doi.org/10.11591/ijere.v13i3.27476
6. Arends, R. I. (2012). Learning to teach (9th ed.). McGraw-Hill. Babitha, M., Suchama, C., Gudivada, V. K., Kazi, K. S. L., & Banduru, S. R. (2022). Trends of artificial intelligence for online exams in education. International Journal of Early Childhood Special Education, 14(01), 2457-2463. https://doi.org/10.9756/INT-JECSE/V14I1.290
7. Backes, B., & Cowan, J. (2018). Is the pen stronger than the keyboard? The effect of online testing on measured student achievement. American Institute for Research, CALDER. https://caldercenter.org/sites/default/files/WP%20190.pdf
8. Bell, J., & Waters, S. (2018). Doing your research project: A guide for first-time researchers (7th ed.). McGraw-Hill.

9. Benek, K. (2025). EFL learners’ and teachers’ perceptions of AI-powered language learning technologies: Benefits and challenges. International Journal of Instruction, 18(2), 103-120.
10. Braun, V., & Clarke, V. (2023). Toward good practice in thematic analysis: Avoiding common problems and be(com)ing knowing researcher. International Journal of Transgender Health, 24(1), 1-6. https://doi.org/10.1080/26895269.2022.2129597
11. Braun, V., & Clarke, V. (2024). Supporting best practice in reflexive thematic analysis reporting in Palliative Medicine: A review of published research and introduction to the Reflexive Thematic Analysis Reporting Guidelines (RTARG). Palliative Medicine, 38(6), 608-616. https://doi.org/10.1177/02692163241234800
12. Cardoso, P., Morgado, L., & Teixeira, A. (2019). Open practices in public higher education in Portugal: Faculty perspectives. Open Praxis, 11(1), 55–70. https://doi.org/10.5944/openpraxis.11.1.823
13. Comiche, A. C., Germano,V. M., & Gonçalves, B. F. (2024). O m-learning como modalidade de ensino à distância em Moçambique [M-learning as a distance learning modality in Mozambique]. Revista Electrónica de Investigação e Desenvolvimento, 15(1), 1-13.
14. Cross, S., Aristeidou, M., Rossade, K.-D., Wood, C., & Brasher, A. (2023). The impact of online exams on the quality of distance learners’ exam and exam revision experience: Perspectives from the Open University UK. Online Learning, 27(2), 27-45.
15. Dağgöl, G. D., & Akçayoğlu, D. İ. (2023). From emergency remote teaching to remote online education: Challenges, benefits and differences in EFL setting. Kastamonu Education Journal, 31(1), 48-59. https://doi.org/10.24106/kefdergi.1243546
16. Domaç, G. G., Neccar, D., & Tüfekci, A. (2022). Koronavirüs (Covid-19) salgını sürecinde uzaktan eğitimin fen bilimleri öğretmenlerinin mesleklerine yansımaları [Reflections of distance education on science teachers’ profession during the coronavirus (Covid-19) pandemic]. Nitel Sosyal Bilimler, 4(1), 59-77. https://doi.org/10.47105/nsb.1117062
17. Elfirdoussi, S., Lachgar, M., Kabaili, H., Rochdi, A., Goujdami, D., & Firdoussi, E. (2020). Assessing distance learning in higher education during the covid-19 pandemic. Education Research International, 2020, 1-13, Article 8890633. https://doi.org/10.1155/2020/8890633.
18. Ergin, D. Y., Asutay, H., & Doğan, C. (2022). Distance education problems in foreign language education during the pandemic period. Open Journal for Educational Research, 6(1), 43-56. https://doi.org/10.32591/coas.ojer.0601.04043e
19. Gall, M. D., Gall, J. P., & Borg, W. R. (2007). Educational Research: An Introduction (8th ed.). Pearson. Gyamfi, G., Sukseemuang, P., Tantiwich, K., & Kaewkong, P. (2019). Self-study with the educational technology tell me more: What EFL learners do. Open Praxis, 11(1), 103-116. https://doi.org/10.5944/openpraxis.11.1.881
20. Hara, N. & Kling, R. (1999). Students’ frustrations with web-based distance education course. First Monday, 4(12). https://doi.org/10.5210/fm.v4i12.710
21. Huang, Y., Lazarides, R., & Richter, D. (2023). Teachers’ adaptations to COVID-19: Perceived preparedness for distance education, frequency of teacher-student contact, and resources in ICT. European Journal of Teacher Education, 1-26. https://doi.org/10.1080/02619768.2023.2288554
22. Kara, M., Erdoğdu, F., Kokoç, M., & Cagiltay, K. (2019). Challenges faced by adult learners in online distance education: A literature review. Open Praxis, 11(1), 5-22. https://doi.org/10.5944/openpraxis.11.1.929
23. Luckesi, C. C. (2011). Avaliação da aprendizagem componente do ato pedagógico [Assessment of learning as a component of the pedagogical act]. Cortez.
24. Lumbela, N. A. S. (2017). Educação a distância no ensino superior em Moçambique: Uma realidade, um desafio [Distance education in higher education in Mozambique: A reality, a challenge]. [Master’s Thesis, Escola Superior de Educação, Instituto Politécnico de Santarém]. https://repositorio.ipsantarem.pt/bitstream/10400.15/2078/1/DISSERTACAO-NARCISO%20LUMBELA-VFINAL.pdf
25. McNamara, T. (2000). Language testing. Oxford University Press.
26. Mombassa, A. B., & Arruda, E. P. (2019). Educação a distância em Moçambique: perspectivas para a inclusão da população no ensino superior [Distance education in Mozambique: Perspectives for inclusion of the population in higher education]. Revista Diálogo Educacional, 19(61), 899-919. https://doi.org/10.7213/1981-416x.19.061.ao10
27. Mombassa, A. Z. B., & Arruda, E. P. (2018). História da educação a distância em Moçambique: perspectivas atuais e as contribuições do Brasil [History of distance education in Mozambique: Current perspectives and contributions from Brazil]. Práxis Educativa, 13(3), 643–660. https://doi.org/10.5212/PraxEduc.v.13i3.0001
28. Nyakuleha, O. H., & Simengwa, D. (2023). Blended open distance learning (ODL) challenges in 21st century: Primary findings from ODL students from Malawi. Revista Científica Cognitionis, 6(2), 415-433. https://doi.org/10.38087/2595.8801.204
29. Oliveira, I. B., & Pacheco, D. C. (2003). Avaliação e currículo no cotidiano escolar [Assessment and curriculum in everyday school life]. In M. T. Esteban (Ed.), Escola currículo e avaliação (pp. 119 -136). Cortez.
30. Perdede, P. (2019). Mixed methods research designs in EFL. In P. Perdede (Ed.), Proceeding: English education department collegiate forum (EED CF) 2015-2018 - “EFL theory and practice: Voice of EED UKI” (pp. 230-243). UKI Press.
31. Pingol, K. F. G. (2022). Learners’ perceptions and preferences in distance learning in the new normal era. International Journal of Social Learning (IJSL), 3(1), 1-19. https://doi.org/10.47134/ijsl.v3i1.38
32. Preti, O., & Barbieri, M. A. (2013). Expansion of higher education distance-learning in Mozambique: student profile, expectations and perceptions of an international cooperation program. Associação Brasileira de Educação a Distância, 12, 253-277.
33. Qafzezi, E., & Kadi, F. (2023). Challenges of effective assessment of online learners. Facta Universitatis, Series: Teaching, Learning and Teacher Education. https://doi.org/10.22190/futlte220913001q
34. Rabelo, E. H. (1998). Avaliação: Novos tempos, novas práticas [Assessment: New times, new practices] (7th ed.). Vozes.
35. Senel, S., & Senel, H. C. (2021). Remote assessment in higher education during covid-19 pandemic. International Journal of Assessment Tools in Education, 8(2), 181–199 https://doi.org/10.21449/ijate.820140
36. Sousa, M. J., & Baptista, C. S. (2011). Como fazer investigação, dissertações, teses, and relatórios – Segundo Bolonha [How to conduct research, dissertations, theses, and reports – according to Bologna] (5th ed.). Pactor.
37. Souto Romero, M., Arias-Oliva, M., Lima Rua. O., & Pérez-Portabella, A. (2024). Online assessment in higher education during the COVID-19. Journal of Management and Business Education, 7(2), 207-219. https://doi.org/10.35564/jmbe.2024.0012
38. Stahl, N. A., & King, J. R. (2020). Expanding approaches for research: Understanding and using trustworthiness in qualitative research. Journal of Developmental Education, 44(1), 26-28.
39. Suharsih, S., & Wijayanti, M. A. (2021). Online learning for EFL learners: Perceptions, challenges, and expectations. Journal of English Language Studies, 6(2), 244. https://doi.org/10.30870/jels.v6i2.12122
40. Wallace, M. P., & Qin, C. Y. (2021). Language classroom assessment fairness: Perceptions from students. LEARN Journal: Language Education and Acquisition Research Network, 14(1), 492-521.
41. Winke, P. (2024). Sharing, collaborating, and building trust: How open science advances language testing. Language Testing, 41(4), 845-859. https://doi.org/10.1177/02655322231211159
42. Zubala, E., Wichrowska, K., & Marchlik, P. (2023). Assessment of early primary school students during remote education in Poland. International Journal of Pedagogy, Innovation and New Technologies, 10(2), 115-125. https://doi.org/10.5604/01.3001.0054.2795
43. Züll, C. (2016). Open-ended questions (Version 2.0). (GESIS Survey Guidelines). Mannheim: GESIS - Leibniz-Institut für Sozialwissenschaften. https://doi.org/10.15465/gesis-sg_en_002