Evolving landscape of artificial intelligence (AI) and assessment in education: A bibliometric analysis

Abstract

The rapid evolution of digital technologies and computer sciences is ushering society into a technologically driven future where machines continually advance to meet human needs and enhance their own intelligence. Among these groundbreaking innovations, Artificial Intelligence (AI) is a cornerstone technology with far-reaching implications. This study undertakes a bibliometric review to investigate contemporary AI and assessment topics in education, aiming to delineate its evolving scope. The Web of Science Databases provided the articles for analysis, spanning from 1994 to September 2023. The study seeks to address research questions about prominent publication years, authors, countries, universities, journals, citation topics, and highly cited articles. The study’s findings illuminate the dynamic nature of AI in educational assessment research, with AI firmly establishing itself as a vital component of education. The study underscores global collaboration, anticipates emerging technologies, and highlights pedagogical implications. Prominent trends emphasize machine learning, Chat GPT, and their application in higher education and medical education, affirming AI's transformative potential. Nevertheless, it is essential to acknowledge the limitations of this study, including data currency and the evolving nature of AI in education. Nonetheless, AI applications are poised to remain a prominent concern in educational technology for the foreseeable future, promising innovative solutions and insights.

Keywords

• Artificial intelligence (ai)
• Assessment
• Bibliometric analysis
• Education

References

1. Agarwal, A., Durairajanayagam, D., Tatagari, S., Esteves, S., Harlev, A., Henkel, R., Roychoudhury, S., Homa, S., Puchalt, N., Ramasamy, R., Majzoub, A., Ly, K., Tvrda, E., Assidi, M., Kesari, K., Sharma, R., Banihani, S., Ko, E., Abu-Elmagd, M., … Bashiri, A. (2016). Bibliometrics: tracking research impact by selecting the appropriate metrics. Asian Journal of Andrology, 18(2), 296. https://doi.org/10.4103/1008-682x.171582
2. Alam, A., Hasan & Raza, M. (2022). Impact of artificial intelligence (AI) on education: changing paradigms and approaches. Towards Excellence, 281 289. https://doi.org/10.37867/te140127
3. Bærøe, K., Miyata-Sturm, A., & Henden, E. (2020). How to achieve trustworthy artificial intelligence for health. Bulletin of the World Health Organization, 98(4), 257-262. https://doi.org/10.2471/blt.19.237289
4. Baker, R.S., & Inventado, P.S. (2014). Educational data mining and learning analytics. Learning Analytics, 61-75. https://doi.org/10.1007/978-1-4614-3305-7_4
5. Baker, R.S., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3-17.
6. Bozkurt, A. (2023). Generative artificial intelligence (AI) powered conversational educational agents: The inevitable paradigm shift. Asian Journal of Distance Education, 18(1). Retrieved from https://www.asianjde.com/ojs/index.php/AsianJDE/article/view/718
7. Bozkurt, A., Karadeniz, A., Baneres, D., Guerrero-Roldán, A.E., & Rodríguez, M.E. (2021). Artificial intelligence and reflections from educational landscape: a review of AI studies in half a century. Sustainability, 13(2), 800. https://doi.org/10.3390/su13020800
8. Bozkurt, A., Xiao, J., Lambert, S., Pazurek, A., Crompton, H., Koseoglu, S., Farrow, R., Bond, M., Nerantzi, C., Honeychurch, S., Bali, M., Dron, J., Mir, K., Stewart, B., Costello, E., Mason, J., Stracke, C.M., Romero-Hall, E., Koutropoulos, A., … Jandrić, P. (2023). Speculative futures on ChatGPT and generative artificial intelligence (AI): A collective reflection from the educational landscape. Asian Journal of Distance Education, 18(1), 53-130. https://doi.org/10.5281/zenodo.7636568

9. Braiki, B.A., Harous, S., Zaki, N., & Alnajjar, F. (2020). Artificial intelligence in education and assessment methods. Bulletin of Electrical Engineering and Informatics, 9(5), 1998-2007. https://doi.org/10.11591/eei.v9i5.1984
10. Chai, C.S., Wang, X., & Xu, C. (2020). An extended theory of planned behavior for the modelling of Chinese secondary school students' intention to learn artificial intelligence. Mathematics, 8(11), 2089. https://doi.org/10.3390/math8112089
11. Civaner, M.M., Uncu, Y., Bulut, F., Chalil, E.G., & Tatli, A. (2022). Artificial intelligence in medical education: a cross-sectional needs assessment. BMC Medical Education, 22(1), 772. https://doi.org/10.1186/s12909-022-03852-3
12. Cobo, M. J., López-Herrera, A. G., Herrera-Viedma, E., & Herrera, F. (2011). Science mapping software tools: Review, analysis, and cooperative study among tools. Journal of the American Society for Information Science and Technology, 62(7), 1382-1402. https://doi.org/10.1002/asi.21525
13. Donthu, N., Kumar, S., Pandey, N., Pandey, N., & Mishra, A. (2021). Mapping the electronic word-of-mouth (eWOM) research: A systematic review and bibliometric analysis. Journal of Business Research, 135, 758 773. https://doi.org/10.1016/j.jbusres.2021.07.015
14. Durán-Sánchez, A., Del Río-Rama, M. de la C., Álvarez-García, J., & García-Vélez, D.F. (2019). Mapping of scientific coverage on education for entrepreneurship in higher education. Journal of Enterprising Communities: People and Places in the Global Economy, 13(1/2), 84-104. https://doi.org/10.1108/jec-10-2018-0072
15. Erickson, J.A., Botelho, A.F., McAteer, S., Varatharaj, A., & Heffernan, N.T. (2020). The automated grading of student open responses in mathematics. In C. Rensing, & H. Drachsler (Eds.), Proceedings of the Tenth International Conference on Learning Analytics & Knowledge (pp. 615–624). Association for Computing Machinery. https://doi.org/10.1145/3375462.3375523
16. Fahimnia, B., Sarkis, J., & Davarzani, H. (2015). Green supply chain management: A review and bibliometric analysis. International Journal of Production Economics, 162, 101-114. https://doi.org/10.1016/j.ijpe.2015.01.003
17. Gardner, J., O'Leary, M., & Yuan, L. (2021). Artificial intelligence in educational assessment: 'Breakthrough? Or buncombe and ballyhoo?'. Journal of Computer Assisted Learning, 37(5), 1207-1216. https://doi.org/10.1111/jcal.12577
18. González-Calatayud, M.L., Fernández, C., & Meneses, J. (2019). Learning styles and educational assessment: A systematic review. Frontiers in Psychology, 10, 2381. https://doi.org/10.3389/fpsyg.2019.02381
19. González-Calatayud, V., Prendes-Espinosa, P., & Roig-Vila, R. (2021). Artificial intelligence for student assessment: a systematic review. Applied Sciences, 11(12), 5467. https://doi.org/10.3390/app11125467
20. Grüning, D.J. (2022). Synthesis of human and artificial intelligence: review of “how to stay smart in a smart world: why human intelligence still beats algorithms” by Gerd Gigerenzer. Futures & Foresight Science, 4(3-4). https://doi.org/10.1002/ffo2.137
21. Gülmez, D., Özteke, İ., & Gümüş, S. (2021). Overview of Educational Research from Turkey Published in International Journals: A Bibliometric Analysis. Education & Science/Eğitim ve Bilim, 46(206), 1-27. https://doi.org/10.15390/EB.2020.9317
22. Hassanien, A., Darwish, A., & El-Aska, H. (2020). Machine Learning and Data Mining in Aerospace Technology. Springer Nature Switzerland AG: Cham, Switzerland.
23. Janpla, S., & Piriyasurawong, P. (2018). The development of problem-based learning and concept mapping using a block-based programming model to enhance the programming competency of undergraduate students in computer science. TEM Journal, 7(4), 708.
24. Kaya, S. (2023). A bibliometric journey into research trends in curriculum field: Analysis of two journals. International Journal of Assessment Tools in Education, 10(3), 496-506. https://doi.org/10.21449/ijate.1278728
25. Kubsch, M., Czinczel, B., Lossjew, J., Wyrwich, T., Bednorz, D., Bernholt, S., Fiedler, D., Strauß, S., Cress, U., Drachsler, H., Neumann, K., & Rummel, N. (2022). Toward learning progression analytics — Developing learning environments for the automated analysis of learning using evidence centered design. Frontiers in Education, 7. https://doi.org/10.3389/feduc.2022.981910
26. Kuleto, V., Ilić, M., Dumangiu, M., Ranković, M., Martins, O.M.D., Păun, D., & Mihoreanu, L. (2021). Exploring opportunities and challenges of artificial intelligence and machine learning in higher education institutions. Sustainability, 13(18), 10424. https://doi.org/10.3390/su131810424
27. Lancaster, T. (2023). Artificial intelligence, text generation tools and ChatGPT–does digital watermarking offer a solution?. International Journal for Educational Integrity, 19(1), 10. https://doi.org/10.1007/s40979-023-00131-6
28. Latif, E., Mai, G., Nyaaba, M., Wu, X., Liu, N., Lu, G.,Li, S., Liu, T., & Zhai, X. (2023). Artificial general intelligence (AGI) for education. arXiv preprint arXiv:2304.12479. https://doi.org/10.48550/arXiv.2304.12479
29. Li, T., Reigh, E., He, P., & Adah Miller, E. (2023). Can we and should we use artificial intelligence for formative assessment in science? Journal of Research in Science Teaching, 60(6), 1385-1389. https://doi.org/10.1002/tea.21867
30. Luckin, R., Holmes, W., Griffiths, M., & Forcier, L.B. (2016). Intelligence unleashed: An argument for AI in education. Journal of Computer Assisted Learning, 32(3), 201-210. https://doi.org/10.1111/jcal.12140
31. Naismith, B., Mulcaire, P., & Burstein, J. (2023, July). Automated evaluation of written discourse coherence using GPT-4. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023) (pp. 394-403).
32. National Research Council. (2012). A Framework for K-12 Science Education. https://doi.org/10.17226/13165
33. National Research Council. (2013). Next Generation Science Standards: For states, by states. https://doi.org/10.17226/18290
34. Okagbue, E.F., Ezeachikulo, U.P., Nwigwe, E.O., & Juma, A.A. (n.d.). Machine learning and artificial intelligence in education research: a comprehensive overview of 22 years of research indexed in the scopus database. https://doi.org/10.21203/rs.3.rs-1845778/v1
35. Ouyang, F., Dinh, T.A., & Xu, W. (2023). A systematic review of AI-driven educational assessment in stem education. Journal for STEM Education Research, 6(3), 408-426. https://doi.org/10.1007/s41979-023-00112-x
36. Owan, V.J., Abang, K.B., Idika, D.O., Etta, E.O., & Bassey, B.A. (2023). Exploring the potential of artificial intelligence tools in educational measurement and assessment. Eurasia Journal of Mathematics, Science and Technology Education, 19(8), em2307. https://doi.org/10.29333/ejmste/13428
37. Qu, J., Zhao, Y., & Xie, Y. (2022). Artificial intelligence leads the reform of education models. Systems Research and Behavioral Science, 39(3), 581 588. https://doi.org/10.1002/sres.2864
38. Saito, T., & Watanobe, Y. (2020). Learning path recommendation system for programming education based on neural networks. International Journal of Distance Education Technologies (IJDET), 18(1), 36-64. https://doi.org/10.4018/IJDET.2020010103
39. Sapci, A.H., & Sapci, H.A. (2020). Artificial intelligence education and tools for medical and health informatics students: systematic review. JMIR Medical Education, 6(1), e19285. https://doi.org/10.2196/19285
40. Sharma, K., Papamitsiou, Z., & Giannakos, M. (2019). Building pipelines for educational data using AI and multimodal analytics: A “grey‐box” approach. British Journal of Educational Technology, 50(6), 3004-3031. https://doi.org/10.1111/bjet.12854
41. Sharma, P., & Harkishan, M. (2022). Designing an intelligent tutoring system for computer programing in the Pacific. Education and Information Technologies, 27(5), 6197-6209. https://doi.org/10.1007/s10639-021-10882-9
42. Siemens, G., & Baker, R.S.J.d. (2012). Learning analytics and educational data mining. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge. https://doi.org/10.1145/2330601.2330661
43. Tolsgaard, M.G., Pusic, M.V., Sebok-Syer, S.S., Gin, B., Svendsen, M.B., Syer, M.D., Brydges, R., Cuddy, M.M., & Boscardin, C.K. (2023). The fundamentals of Artificial Intelligence in medical education research: AMEE Guide No. 156. Medical Teacher, 45(6), 565-573. https://doi.org/10.1080/0142159x.2023.2180340
44. Toumi, Y., Bengherbia, B., Lachenani, S., & Ould Zmirli, M. (2022). FGPA implementation of a bearing fault classification system based on an envelope analysis and artificial neural network. Arabian Journal for Science and Engineering, 47(11), 13955-13977. https://doi.org/10.1007/s13369-022-06599-7
45. Wood, E.A., Ange, B.L., & Miller, D.D. (2021). Are we ready to integrate artificial intelligence literacy into medical school curriculum: students and faculty survey. Journal of Medical Education and Curricular Development, 8. https://doi.org/10.1177/23821205211024078
46. Yang, Y., Zheng, Z., Zhu, G., & Salas‐Pilco, S.Z. (2023). Analytics‐supported reflective assessment for 6th graders' knowledge building and data science practices: An exploratory study. British Journal of Educational Technology. https://doi.org/10.1111/bjet.13308
47. Zawacki-Richter, O., Marín, V.I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education - where are the educators? International Journal of Educational Technology in Higher Education, 16(1). https://doi.org/10.1186/s41239-019-0171-0
48. Zehner, F., & Hahnel, C. (2023). Artificial intelligence on the advance to enhance educational assessment: Scientific clickbait or genuine gamechanger?. Journal of Computer Assisted Learning, 39(3), 695-702. https://doi.org/10.1111/jcal.12810
49. Zhai, X. (2023, August 28- September 1). ChatGPT for next generation science learning [Paper presentation]. The 15th Conference of the European Science Education Research Association (ESERA), Cappadocia, Türkiye.
50. Zhai, X., & Nehm, R.H. (2023). AI and formative assessment: The train has left the station. Journal of Research in Science Teaching, 60(6), 1390 1398. https://doi.org/10.1002/tea.21885
51. Zhai, X., Shi, L., & Nehm, R.H. (2021). A meta-analysis of machine learning-based science assessments: factors impacting machine-human score agreements. Journal of Science Education and Technology, 30(3), 361-379. https://doi.org/10.1007/s10956-020-09875-z
52. Zhai, X., Haudek, K.C., Shi, L., Nehm, R.H., & Urban-Lurain, M. (2020). From substitution to redefinition: A framework of machine learning-based science assessment. Journal of Research in Science Teaching, 57(9), 1430-1459. https://doi.org/10.1002/tea.21658
53. Zupic, I., & Čater, T. (2015). Bibliometric methods in management and organization. Organizational Research Methods, 18(3), 429 472. https://doi.org/10.1177/1094428114562629