Impact of Artificial Intelligence on Assessment and Evaluation Approaches in Education

Year 2024, Volume: 5 Issue: 2, 139 - 152, 19.11.2024

Hande Tanberkan , Mahmut Özer , Selahattin Gelbal

Abstract

Artificial Intelligence (AI) technologies are being applied commonly in all aspects of life. Education is one of the leading areas in this respect. AI applications offer significant opportunities for students, educators and education administrators. Students can benefit from these technologies for individualized education and addressing their deficiencies. A similar situation applies to educators. However, students are the most vulnerable group to the current and long-term risks posed by these technologies. While students fulfill a significant part of their responsibilities through the opportunities provided by AI technologies, they face two options: succeeding through ethical violations or addressing their deficiencies ethically. Students lacking AI ethical literacy often choose the first option, masking their failures and getting involved in ethical violations that will bring heavy burdens in the long run. This study discusses the benefits offered by AI technologies in education and the problems they cause in the context of measurement and evaluation. AI will have an important place in the measurement and evaluation as an auxiliary tool in producing texts, creating questions using the produced texts, scoring open-ended exams, solving problems and creating research reports in accordance with ethical rules. It is highlighted that developing AI technologies in education with a participatory approach involving all educational stakeholders and continuously monitoring potential risks during the implementation phase are crucial for establishing a responsible AI culture in education. Finally, considering the dramatic pace of developments in AI, the importance of dynamically updating the measures against ethical violations at the same pace is emphasized.

Keywords

Artificial intelligence, measurement, evaluation, ethics

Ethical Statement

In this study, ethical approval was not required as a review was conducted using previously published studies in the literature.

References

• Acemoglu, D. (2024). The simple macroeconomics of AI (No. w32487). National Bureau of Economic Research. https://doi.org/10.3386/w32487
• Acemoglu, D., & Restrepo, P. (2018). Artificial intelligence, automation and work (No. w24196). National Bureau of Economic Research. https://doi.org/10.3386/w24196
• Acemoglu, D., Autor, D., & Johnson, S. (2023). Can we have pro-worker- AI? Choosing a path of machines in service of minds. CEPR Policy Insight No.123.
• Akgun, S., & Greenhow, C. (2022). Artificial intelligence in education: Addressing ethical challenges in K-12 settings. AI and Ethics, 2(3), 431-440. https://doi.org/10.1007/s43681-021-00096-7
• Alam, M. F., Lentsch, A., Yu, N., Barmack, S., Kim, S., Acemoglu, D., Hart, J., Johnson, S., & Ahmed, F. (2024). From automation to augmentation: Redefining engineering design and manufacturing in the age of NextGen-AI. An MIT Exploration of Generative AI. https://doi.org/10.21428/e4baedd9.e39b392d
• Al-Worafi, Y. M., Hermansyah, A., Goh, K. W., & Ming, L. C. (2023). Artificial intelligence use in university: Should we ban ChatGPT? Preprints https://doi.org/10.20944/preprints202302.0400.v1
• Aquino, Y.S.J. (2023). Making decisions: bias in artificial intelligence and data-driven diagnostic tools. Australian Journal of General Practice. 52 (7), 439-442. https://doi.org/10.31128/AJGP-12-22-6630
• Athaluri, S. A., Manthena, S. V., Kesapragada, V. K. M., Yarlagadda, V., Dave, T., & Duddumpudi, R. T. S. (2023). Exploring the boundaries of reality: investigating the phenomenon of artificial intelligence hallucination in scientific writing through ChatGPT references. Cureus, 15(4). https://doi.org/10.7759/cureus.37432
• Bezirhan, U., & von Davier, M. (2023). Automated reading passage generation with OpenAI's large language model. Computers and Education: Artificial Intelligence, 5. https://doi.org/10.1016/j.caeai.2023.100161
• Blasimme, A., & Vayena, E. (2020). What’s next for COVID-19 apps? Governance and oversight. Science, 370(6518), 760–762. https://doi.org/10.1126/science.abd9006
• Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative Research Journal, 9(2), 27-40.
• Brynjolfsson, E., Li, D., & Raymond, L. R. (2023). Generative AI at work (No. w31161). National Bureau of Economic Research.
• Cardona, M. A., Rodríguez, R. J., & Ishmael, K. (2023). Artificial intelligence and the future of teaching and learning: Insights and recommendations. Office of Educational Technology, US Department of Education. https://www2.ed.gov/documents/ai-report/ai-report.pdf
• Cotton, D. R., Cotton, P. A., & Shipway, J. R. (2024). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in education and teaching international, 61(2), 228-239. https://doi.org/10.1080/14703297.2023.2190148
• Couldry, N. (2020). Recovering critique in an age of datafication. New Media & Society, 22(7), 1135-1151. https://doi.org/10.1177/1461444820912536
• Du, X., Shao, J., & Cardie, C. (2017). Learning to ask: Neural question generation for reading comprehension. arXiv preprint arXiv:1705.00106. https://doi.org/10.48550/arXiv.1705.00106
• European Commission. (2022). Ethical guidelines on the use of artificial intelligence (AI) and data in teaching and learning for educators, Publications Office of the European Union. https://data.europa.eu/doi/10.2766/153756
• Foltýnek, T., Meuschke, N., & Gipp, B. (2019). Academic plagiarism detection: a systematic literature review. ACM Computing Surveys (CSUR), 52(6), 1-42. https://doi.org/10.1145/3345317
• Frank, M. R., Autor, D., Bessen, J. E., Brynjolfsson, E., Cebrian, M., Deming, D. J., Feldman, M., Groh, M., Lobo, J., Moro, E., Wang, D., Youn, H., & Rahwan, I. (2019). Toward understanding the impact of artificial intelligence on labor. PNAS, 116(14), 6531-6539. https://doi.org/10.1073/pnas.190094911
• Gardner, J., O'Leary, M., & Yuan, L. (2021). Artificial intelligence in educational assessment:‘Breakthrough? Or buncombe and ballyhoo?’. Journal of Computer Assisted Learning, 37(5), 1207-1216. https://doi.org/10.1111/jcal.12577
• Halaweh, M. (2023). ChatGPT in education: Strategies for responsible implementation. Contemporary Educational Technology, 15(2), ep421. https://doi.org/10.30935/cedtech/13036
• Hosseini, M., Rasmussen, L. M., & Resnik, D. B. (2023). Using AI to write scholarly publications. Accountability in Research, 1-9. https://doi.org/10.1080/08989621.2023.2168535
• Huallpa, J. J. (2023). Exploring the ethical considerations of using ChatGPT in university education. Periodicals of Engineering and Natural Sciences, 11(4), 105-115. http://dx.doi.org/10.21533/pen.v11i4.3770
• Huang, L. (2023). Ethics of artificial intelligence in education: Student privacy and data protection. Science Insights Education Frontiers, 16(2), 2577-2587. https://doi.org/10.15354/sief.23.re202
• Huang, X., Zou, D., Cheng, G., Chen, X., & Xie, H. (2023). Trends, research issues and applications of artificial intelligence in language education. Educational Technology & Society, 26(1), 112-131. https://doi.org/10.30191/ETS.202301_26(1).0009
• İlikhan, SU., Ozer, M., Tanberkan, H., Bozkurt, V. (2024). How to mitigate the risks of deployment of artificial intelligence in medicine? Turkish Journal of Medical Sciences, 54(3), 483-492. https://doi.org/10.55730/1300-0144.5814
• Jang, E. E. (2014). Focus on Assessment-Oxford Key Concepts for the Language Classroom. Oxford University Press.
• Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Yj., Madotto, A., & Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), 1-38.
• Johnson, M. (2020, April 22). A scalable approach to reducing gender bias in Google Translate. Google Research Blog. https://research.google/blog/a-scalable-approach-to-reducing-gender-bias-in-google-translate/
• Kamalov, F., Santandreu Calonge, D., & Gurrib, I. (2023). New era of artificial intelligence in education: Towards a sustainable multifaceted revolution. Sustainability, 15(16), 12451. https://doi.org/10.3390/su151612451
• Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M.,
• Schmidt, A., Seidel, T., ... & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and individual differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274
• Khaled, F., & Al-Tamimi, M. S. H. (2021). Plagiarism detection methods and tools: An overview. Iraqi Journal of Science, 2771-2783. https://doi.org/10.24996/ijs.2021.62.8.30
• Khalil, M., & Er, E. (2023, June). Will ChatGPT get you caught? Rethinking of plagiarism detection. In International Conference on Human-Computer Interaction (pp. 475-487). Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-34411-4_32
• Khosravi, H., Shum, S. B., Chen, G., Conati, C., Tsai, Y. S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022). Explainable artificial intelligence in education. Computers and Education: Artificial Intelligence, 3, 100074. https://doi.org/10.1016/j.caeai.2022.100074
• Korinek, A. (2023). Generative AI for economic research: Use cases and implications for economists. Journal of Economic Literature, 61(4), 1281-1317. https://doi.org/10.1257/jel.20231736
• Lancaster, T. (2023). Artificial intelligence, text generation tools and ChatGPT – does digital watermarking offer a solution?. International Journal for Educational Integrity, 19(10). https://doi.org/10.1007/s40979-023-00131-6
• Lebovitz, S., Levina, N., & Lifshitz-Assaf, H. (2021). Is AIground truth really true? The dangers of training andevaluating ai tools based on experts’ know-what. MISQuarterly, 45(3), 1501–1526. https://doi.org/10.25300/misq/2021/16564
• Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(410). https://doi.org/10.3390/educsci13040410
• Mulla, N., & Gharpure, P. (2023). Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications. Progress in Artificial Intelligence, 12(1), 1-32.
• Noy, S., & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187-192. https://doi.org/10.1126/science.adh2586
• Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447-453. https://doi.org/10.1126/science.aax2342
• Owan, V. J., Abang, K. B., Idika, D. O., Etta, E. O., & Bassey, B. A. (2023). Exploring the potential of artificial intelligence tools in educational measurement and assessment. EURASIA Journal of Mathematics, Science and Technology Education, 19(8), em2307. https://doi.org/10.29333/ejmste/13428
• Ozer, M. (2024). Potential Benefits and Risks of Artificial Intelligence in Education. Bartın University Journal of Faculty of Education, 13(2), 232-244. https://doi.org/10.14686/buefad.1416087
• Ozer, M., & Perc, M. (2024). Human complementation must aid automation to mitigate unemployment effects due to AI Technologies in the labor market. Reflektif Journal of Social Sciences, 5(2), 503-514.
• Ozer, M., Perc, M., & Suna, H.E. (2024b). Artificial Intelligence Bias and the Amplification of Inequalities in the Labor Market. Journal of Economy Culture and Society, 69, 159-168. https://doi.org/10.26650/JECS2023-1415085
• Ozer, M., Perc,M., & Suna, HE. (2024a). Participatory management can help AI ethics adhere to the social consensus. İstanbul University Journal of Sociology, 44(1), 221-238. https://doi.org/10.26650/SJ.2024.44.1.0001
• Page, E. B. (1966). The imminence of... grading essays by computer. The Phi Delta Kappan, 47(5), 238-243.
• Peng, S., Kalliamvakou, E., Cihon, P., & Demirer, M. (2023). The impact of AI on developer productivity: Evidence from GitHub Copilot. arXiv preprint arXiv:2302.06590. https://doi.org/10.48550/arXiv.2302.06590
• Perc, M., Ozer, M., & Hojnik, J. (2019). Social and juristic challenges of artificial intelligence. Palgrave Communications, 5(1). https://doi.org/10.1057/s41599-019-0278-x
• Qi, P., Zhang, Y., & Manning, C. D. (2020). Stay hungry, stay focused: Generating informative and specific questions in information-seeking conversations. arXiv preprint arXiv:2004.14530. https://doi.org/10.48550/arXiv.2004.14530
• Rane, N. L., Choudhary, S. P., Tawde, A., & Rane, J. (2023). ChatGPT is not capable of serving as an author: ethical concerns and challenges of large language models in education. International Research Journal of Modernization in Engineering Technology and Science, 5(10), 851-874. https://www.doi.org/10.56726/IRJMETS45212
• Salvagno, M., Taccone, F. S., & Gerli, A. G. (2023). Can artificial intelligence help for scientific writing?. Critical care, 27(1), 75. https://doi.org/10.1186/s13054-023-04380-2
• Sayın, A. & Gierl, M. (2024), Using OpenAI GPT to Generate Reading Comprehension Items. Educational Measurement: Issues and Practice, 43, 5-18. https://doi.org/10.1111/emip.12590
• Scialom, T., Piwowarski, B., & Staiano, J. (2019, July). Self-attention architectures for answer-agnostic neural question generation. In Proceedings of the 57th annual meeting of the Association for Computational Linguistics (pp. 6027-6032), Florence, Italy. https://doi.org/10.18653/v1/P19-1604
• Septiandri, A. A., Constantinidies, M., & Quercia, D. (2023). The impact of AI innovations on US occupations. Nokia Bell Labs, UK.
• Sherman, A. T., Herman, G. L., Oliva, L., Peterson, P. A., Golaszewski, E., Poulsen, S., Scheponik, T., & Gorti, A. (2020). Experiences and lessons learned creating and validating concept inventories for cybersecurity. In
• National Cyber Summit Research Track 2020 (pp. 3-34). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-58703-1_1
• Silva-Jurado, R. J., & Silva-Jurado, M. D. (2024). Educational innovation in the 21st century: gamification, artificial intelligence and art as transformative tools. YUYAY: Educational, Methodologies & Didactics Strategies, 3(1), 35-52. https://doi.org/10.59343/yuyay.v3i1.60
• Sok, S., & Heng, K. (2023). ChatGPT for education and research: A review of benefits and risks. Cambodian Journal of Educational Research, 3(1), 110-121. https://doi.org/10.2139/ssrn.4378735
• Stahl, B. C. (2023). Embedding responsibility in intelligent systems: from AI ethics to responsible AI ecosystems. Scientific Reports, 13(1), 7586. https://doi.org/10.1038/s41598-023-34622-w
• Stokel-Walker, C. (2022). AI bot ChatGPT writes smart essays - Should professors worry? Nature. https://doi.org/10.1038/d41586-022-04397-7
• Stokel-Walker, C. (2023). ChatGPT listed as author on research papers: Many scientists disapprove. Nature, 613(7945), 620-621. https://doi.org/10.1038/d41586-023-00107-z
• Suleyman, M. (2023). The coming wave: Technology, power and the 21st century’s greatest dilemma. Crown.
• Sullivan, M., Kelly, A. & McLaughlan, P. (2023). ChatGPT in higher education: Considerations for academic integrity and student learning. Journal of Applied Learning & Teaching, 6(1). https://doi.org/10.37074/jalt.2023.6.1.17
• Surahman, E., & Wang, T.H. (2022). Academic dishonesty and trustworthy assessment in online learning: A systematic literature review. Journal of Computer Assisted Learning, 38(6), 1535–1553. https://doi.org/10.1111/ jcal.12708
• Swiecki, Z., Khosravi, H., Chen, G., Martinez-Maldonado, R., Lodge, J. M., Milligan, S.,Selwyn, N., & Gašević, D. (2022). Assessment in the age of artificial intelligence. Computers and Education: Artificial Intelligence, 3, 100075. https://doi.org/10.1016/j.caeai.2022.100075
• Theodorou, A., & Dignum, V. (2020). Towards ethical and socio-legal governance in AI. Nature Machine Intelligence, 2(1), 10–12. https://doi.org/10.1038/s42256-019-0136-y
• Thorp, HH. (2023). ChatGPT is fun, but not an author. Science, 379(6630), 313. https://doi.org/10.1126/science.adg7879
• Varma, A., Dawkins, C., & Chaudhuri, K. (2023). Artificial intelligence and people management: A critical assessment through the ethical lens. Human Resource Management Review, 33(1), 100923. https://doi.org/10.1016/j.hrmr.2022.100923
• Verhoeven, F., Wendling, D., & Prati, C. (2023). ChatGPT: when artificial intelligence replace the rheumatologist in medical writing. Annals of the Rheumatic Diseases, 82(8), 1015-1017. https://doi.org/10.1136/ard-2023-223936
• Wang, X., Gong, Z., Wang, G., Jia, J., Xu, Y., Zhao, J., Fan, Q., Wu, S.,Hu, W., & Li, X. (2023). ChatGPT performs on the Chinese national medical licensing examination. Journal of Medical Systems, 47(1), 86. https://doi.org/10.1007/s10916-023-01961-0
• Warschauer, M., & Grimes, D. (2008). Automated writing assessment in the classroom. Pedagogies: An International Journal, 3(1), 22-36. https://doi.org/10.1080/15544800701771580
• Yanisky-Ravid, S., & Hallisey, S. (2018). ‘Equality and privacy by design’: Ensuring artificial intelligence (AI) is properly trained & fed: A new model of AI data transparency & certification as safe harbor procedures. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3278490
• Yu, H. (2023). Reflection on whether ChatGPT should be banned by academia from the perspective of education and teaching. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1181712
• Zhang, M., Press, O., Merrill, W., Liu, A., & Smith, N. A. (2023). How language model hallucinations can snowball. arXiv preprint arXiv:2305.13534. https://doi.org/10.48550/arXiv.2305.13534