The impact of artificial intelligence on online assessment: A preliminary review

Abstract

The purpose of this study is to examine the impact of artificial intelligence (AI) on online assessment in the context of opportunities and threats based on the literature. To this end, 19 articles related to the AI tool ChatGPT and online assessment were analysed through rapid literature review. In the content analysis, the themes of “AI's assistance role”, “automatic grading and feedback”, “improving assessment” and “time benefit” were obtained in the opportunities category, while the themes of “academic integrity concern”, “reliability issues” and “adaptability issues” were obtained in the threats category. The impact of AI on online assessment was explained within the scope of these themes. The results revealed that the most emphasis was placed on "improving assessment" themes in the opportunities category, and "academic integrity concern" themes in the threats category. At the end of this preliminary review, it was revealed that more studies investigating the integration of AI to online assessment are needed and all educational institutions, especially distance education institutions, should take measures to ensure the ethical use of AI.

Keywords

• Online assessment
• ChatGPT
• Opportunities
• Threats
• Generative AI

References

1. Al-Darei, I. S. & Ahmed, A. M. (2022). The effect of feedback type in the e-learning environment on students’ achievement and motivation. Journal of Educational Technology & Online Learning, 5(3), 694-705.
2. Arslan, K. (2017). Eğitimde yapay zekâ ve uygulamaları. Batı Anadolu Eğitim Bilimleri Dergisi, 11(1), 71-88.
3. Baidoo-Anu, D. & Owusu Ansah, L. (2023). Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. http://dx.doi.org/10.2139/ssrn.4337484
4. Bozkurt, A. & Sharma, R. C. (2020). Emergency remote teaching in a time of global crisis due to
5. CoronaVirus pandemic. Asian Journal of Distance Education, 15(1), i-vi.
6. https://doi.org/10.5281/zenodo.3778083
7. Bozkurt, A. & Uçar, H. (2018). e-Learning and e-Exams: Examination of learners’ perspectives concerning the authentication methods in online assessment processes. Mersin University Journal of the Faculty of Education, 14(2), 745-755.
8. Büyüköztürk, Ş., Kılıç Çakmak, E., Akgün, Ö. E., Karadeniz, Ş. & Demirel, F. (2020). Eğitimde bilimsel

9. araştırma yöntemleri. [Scientific Research Methods in Education]. Ankara: Pegem Akademi.
10. Conrad, D., & Openo, J. (2018). Assessment strategies for online learning: engagement and authenticity. Athabasca University Press. https://doi.org/10.15215/aupress/9781771992329.01
11. Creswell, J.W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research. Boston: Pearson.
12. de Winter, J. C. F. (2023). Can ChatGPT pass high school exams on English language comprehension? Researchgate. Preprint.
13. Farhat, F.; Sohail, S. S. & Madsen, D. Q. (2023). How trustworthy is ChatGPT? The case of bibliometric analyses. Cogent Engineering, 10. https://doi.org/10.1080/23311916.2023.2222988.
14. Fergus, S., Botha, M. & Ostovar, M. (2023). Evaluating academic answers generated using ChatGPT. Journal of Chemical Education, 100 (4), 1672-1675. DOI: 10.1021/acs.jchemed.3c00087
15. Guangul, F. M., Suhail, A. H., Khalit, M. I., & Khidhir, B. A. (2020). Challenges of remote assessment in higher education in the context of COVID-19: A case study of Middle East College. Educational Assessment, Evaluation and Accountability, 32, 519-535. https://doi.org/10.1007/s11092-020-09340-w
16. Gungea, M. (2023). How Covid-19 taught teachers how to teach online– a case study at the Open University of Mauritius. Journal of Educational Technology & Online Learning, 6(2), 482-491.
17. Haque, M.U.; Dharmadasa, I.; Sworna, Z.T.; Rajapakse, R.N. & Ahmad, H. (2022). I think this is the most disruptive technology": Exploring sentiments of ChatGPT early adopters using Twitter Data. arXiv preprint arXiv:2212.05856.
18. Heil, J. & Ifenthaler, D. (2023). Online assessment in higher education: A systematic review. Online Learning, 27(1), 187-218. DOI: 10.24059/olj.v27i1.3398
19. Huber, E.; Harris, L.; Wright, S.; White, A.; Raduescu, C.; Zeivots, S.; Cram, A. & Brodzeli, A. (2023). Towards a framework for designing and evaluating online assessments in business education. Assessment & Evaluation in Higher Education. DOI: 10.1080/02602938.2023.2183487
20. Lai, V.D.; Ngo, N.T.; Veyseh, A.P.B.; Man, H.; Dernoncurt, F.; Bui, T.&Nguyen, T.H. (2023). ChatGPT beyond English: Towards a comprehensive evaluation of large language models in multilingual learning. arXiv preprint arXiv:2304.05613.
21. Lee, V. W. Y.; Lam, P. L. C.; Lo, J. T. S.; Lee, J. L. F., and Li, J. T. S. (2022). Rethinking online assessment from university students’ perspective in COVID-19 pandemic. Cogent Education, 9: 2082079 https://doi.org/10.1080/2331186X.2022.2082079
22. Lo, C.K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Educational Sciences, 13. https://doi.org/10.3390/educsci13040410
23. Mhlanga, D. (2023). Open AI in Education, the Responsible and Ethical Use of ChatGPT Towards Lifelong Learning. SSRN 2023, 4354422. Available online: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4354422 (Accessed on 15 May 2023).
24. Naidu, K., & Sevnarayan, K. (2023). ChatGPT: An ever-increasing encroachment of artificial intelligence in online assessment in distance education. Online Journal of Communication and Media Technologies, 13(1), e2023xx. https://doi.org/10.30935/ojcmt/13291
25. Qadir, J. (2022). Engineering education in the era of ChatGPT: Promise and pitfalls of generative AI for education. TechRxiv. Preprint. DOI: 10.36227/techrxiv.21789434.v1
26. Roumeliotis, K. I. & Tselikas, N.D. (2023). ChatGPT and Open-AI Models: A Preliminary Review. Future Internet, 15, 1-24. https://doi.org/10.3390/fi15060192
27. Sallam, M. (2023). ChatGPT utility in healthcare education, research, and practice: Systematic review on the promising perspectives and valid concerns. Healthcare 2023, 11, 887. https://doi.org/10.3390/healthcare11060887
28. Semlambo, A. A.; Almasi, K. & Liechuka, Y. (2022). Perceived usefulness and ease of use of online examination system: A case of Institute of Accountancy Arusha. International Journal of Scientific Research and Management (IJSRM), 10(4), 851-861. DOI: 10.18535/ijsrm/v10i4.ec08
29. Shi, Z. Z. & Zheng, N. N. (2006). Progress and challenge of artificial intelligence. Journal of Computer Science and Technology, 21(5), 810-822. https://doi.org/10.1007/s11390-006-0810- 5.
30. Tricco, A.C.; Antony, J.; Zarin, W.; Strifler, L.; Ghassemi, M.; Ivory, J.; Perrier, L.; Hutton, B.; Moher, D.; Straus, S.E. (2015). A scoping review of rapid review methods. BMC Med 13, 224. https://doi.org/10.1186/s12916-015-0465-6
31. UNESCO. (2023). ChatGPT and Artificial Intelligence in higher education: Quick start guide. UNESCO.
32. Wei, X. (2018). The application and development of artificial intelligence in smart clothing. IOP Conference Series: Materials Science and Engineering, 320, 012017. https://doi.org/10.1088/1757-899X/320/1/012017
33. Yıldırım, A., & Şimşek, H. (2013). Sosyal bilimlerde nitel araştırma yöntemleri. Ankara: Seçkin Yayıncılık.
34. Yu, H. (2023). Reflection on whether Chat GPT should be banned by academia from the perspective of education and teaching. Front. Psychol. 14. Doi: 10.3389/fpsyg.2023.1181712