AI-based feedback tools in education: A comprehensive bibliometric analysis study

Year 2024, , 622 - 646, 15.11.2024

Mehmet Donmez

https://doi.org/10.21449/ijate.1467476

Abstract

This bibliometric analysis offers a comprehensive examination of AI-based feedback tools in education, utilizing data retrieved from the Web of Science (WoS) database. Encompassing a total of 239 articles from an expansive timeframe, spanning from inception to February 2024, this study provides a thorough overview of the evolution and current state of research in this domain. Through meticulous analysis, it tracks the growth trajectory of publications over time, revealing the increasing scholarly attention towards AI-driven feedback mechanisms in educational contexts. By describing critical thematic areas such as the role of feedback in enhancing learning outcomes, the integration of AI technologies into educational practices, and the efficacy of AI-based feedback tools in facilitating personalized learning experiences, the analysis offers valuable insights into the multifaceted nature of this field. By employing sophisticated bibliometric mapping techniques, including co-citation analysis and keyword co-occurrence analysis, the study uncovers the underlying intellectual structure of the research landscape, identifying prominent themes, influential articles, and emerging trends. Furthermore, it identifies productive authors, institutions, and countries contributing to the discourse, providing a detailed understanding of the collaborative networks and citation patterns within the community. This comprehensive synthesis of the literature serves as a valuable resource for researchers, practitioners, and policymakers alike, offering guidance on harnessing the potential of AI technologies to revolutionize teaching and learning practices in education.

Keywords

AI-driven feedback, Educational integration, Learning enhancement, Personalized learning, Bibliometric analysis

References

• Afzaal, M., Zia, A., Nouri, J., & Fors, U. (2024). Informative feedback and explainable ai-based recommendations to support students’ self-regulation. Technology, Knowledge and Learning, 29(1), 331–354. https://doi.org/10.1007/s10758-023-09650-0
• Aria, M., & Cuccurullo, C. (2017). bibliometrix: An R-tool for comprehensive science mapping analysis. Journal of Informetrics, 11(4), 959 975. https://doi.org/10.1016/j.joi.2017.08.007
• Bañeres, D., Rodríguez, M.E., Guerrero-Roldán, A.E., & Karadeniz, A. (2020). An early warning system to detect at-risk students in online higher education. Applied Sciences (Switzerland), 10(13). https://doi.org/10.3390/app10134427
• Barrett, A., & Pack, A. (2023). Not quite eye to A.I.: student and teacher perspectives on the use of generative artificial intelligence in the writing process. International Journal of Educational Technology in Higher Education, 20(1), 59. https://doi.org/10.1186/s41239-023-00427-0
• Bin-Hady, W.R.A., Al-Kadi, A., Hazaea, A., & Ali, J.K.M. (2023). Exploring the dimensions of ChatGPT in English language learning: a global perspective. Library Hi Tech. https://doi.org/10.1108/LHT-05-2023-0200
• Bui, N.M., & Barrot, J.S. (2024). ChatGPT as an automated essay scoring tool in the writing classrooms: how it compares with human scoring. Education and Information Technologies, 1–18. https://doi.org/10.1007/S10639-024-12891-W/TABLES/5
• Chang, D.H., Lin, M.P.-C., Hajian, S., & Wang, Q.Q. (2023). Educational design principles of using AI Chatbot that supports self-regulated learning in education: Goal setting, feedback, and personalization. Sustainability, 15(17), 12921. https://doi.org/10.3390/su151712921
• Chavez, H., Chavez-Arias, B., Contreras-Rosas, S., Alvarez-Rodríguez, J.M., & Raymundo, C. (2023). Artificial neural network model to predict student performance using nonpersonal information. Frontiers in Education, 8. https://doi.org/10.3389/FEDUC.2023.1106679
• Chen, X., Zou, D., Xie, H., Chen, G., Lin, J., & Cheng, G. (2023). Exploring contributors, collaborations, and research topics in educational technology: A joint analysis of mainstream conferences. Education and Information Technologies, 28(2), 1323–1358. https://doi.org/10.1007/s10639-022-11209-y
• Chen, Z. (2023). Artificial intelligence-virtual trainer: innovative didactics aimed at personalized training needs. Journal of the Knowledge Economy, 14(2), 2007–2025. https://doi.org/10.1007/s13132-022-00985-0
• Chin, D.B., Dohmen, I.M., Cheng, B.H., Oppezzo, M.A., Chase, C.C., & Schwartz, D.L. (2010). Preparing students for future learning with Teachable Agents. Educational Technology Research and Development, 58(6), 649–669. https://doi.org/10.1007/s11423-010-9154-5
• Chiu, M.-C., Hwang, G.-J., Hsia, L.-H., & Shyu, F.-M. (2022). Artificial intelligence-supported art education: a deep learning-based system for promoting university students’ artwork appreciation and painting outcomes. Interactive Learning Environments, 1–19. https://doi.org/10.1080/10494820.2022.2100426
• Conrad, E.J., & Hall, K.C. (2024). Leveraging generative AI to elevate curriculum design and pedagogy in public health and health promotion. Pedagogy in Health Promotion. https://doi.org/10.1177/23733799241232641
• Cowling, M., Crawford, J., Allen, K.-A., & Wehmeyer, M. (2023). Using leadership to leverage ChatGPT and artificial intelligence for undergraduate and postgraduate research supervision. Australasian Journal of Educational Technology, 39(4), 89 103. https://doi.org/10.14742/ajet.8598
• Cukurova, M., Kent, C., & Luckin, R. (2019). Artificial intelligence and multimodal data in the service of human decision‐making: A case study in debate tutoring. British Journal of Educational Technology, 50(6), 3032–3046. https://doi.org/10.1111/bjet.12829
• Ding, L., & Zou, D. (2024). Automated writing evaluation systems: A systematic review of Grammarly, Pigai, and Criterion with a perspective on future directions in the age of generative artificial intelligence. Education and Information Technologies. https://doi.org/10.1007/s10639-023-12402-3
• Ebenbeck, N., & Gebhardt, M. (2024). Differential Performance of Computerized Adaptive Testing in Students With and Without Disabilities A Simulation Study. Journal of Special Education Technology. https://doi.org/10.1177/01626434241232117
• Elmaoğlu, E., Coşkun, A.B., & Yüzer Alsaç, S. (2024). Digital Transformation: The Role, Potential, and Limitations of ChatGPT in Child Health Education. American Journal of Health Education, 55(1), 69–72. https://doi.org/10.1080/19325037.2023.2277937
• Farshad, S., Zorin, E., Amangeldiuly, N., & Fortin, C. (2023). Engagement assessment in project-based education: a machine learning approach in team chat analysis. Education and Information Technologies. https://doi.org/10.1007/s10639-023-12381-5
• Fu, S., Gu, H., & Yang, B. (2020). The affordances of AI‐enabled automatic scoring applications on learners’ continuous learning intention: An empirical study in China. British Journal of Educational Technology, 51(5), 1674–1692. https://doi.org/10.1111/bjet.12995
• Gao, R., Merzdorf, H.E., Anwar, S., Hipwell, M.C., & Srinivasa, A.R. (2024). Automatic assessment of text-based responses in post-secondary education: A systematic review. Computers and Education: Artificial Intelligence, 6, 100206. https://doi.org/10.1016/j.caeai.2024.100206
• Gligorea, I., Cioca, M., Oancea, R., Gorski, A.T., Gorski, H., & Tudorache, P. (2023). Adaptive Learning Using Artificial Intelligence in e-Learning: A Literature Review. Education Sciences, 13(12), 1216. https://doi.org/10.3390/EDUCSCI13121216
• Gu, Y. (2024). Research on Speech Communication Enhancement of English Web-based Learning Platform based on Human-computer Intelligent Interaction. Scalable Computing: Practice and Experience, 25(2), 709 720. https://doi.org/10.12694/scpe.v25i2.2544
• Heeg, D.M., & Avraamidou, L. (2023). The use of Artificial intelligence in school science: a systematic literature review. Educational Media International, 60(2), 125–150. https://doi.org/10.1080/09523987.2023.2264990
• Hopfenbeck, T.N., Zhang, Z., Sun, S.Z., Robertson, P., & McGrane, J.A. (2023). Challenges and opportunities for classroom-based formative assessment and AI: a perspective article. Frontiers in Education, 8. https://doi.org/10.3389/feduc.2023.1270700
• Hopgood, A.A., & Hirst, A.J. (2007). Keeping a Distance-Education Course Current Through eLearning and Contextual Assessment. IEEE Transactions on Education, 50(1), 85–96. https://doi.org/10.1109/TE.2006.888905
• Jaleniauskienė, E., Lisaitė, D., & Daniusevičiūtė-Brazaitė, L. (2023). Artificial Intelligence in Language Education: A Bibliometric Analysis. Sustainable Multilingualism, 23(1), 159–194. https://doi.org/10.2478/sm-2023-0017
• Kaldaras, L., Yoshida, N.R., & Haudek, K.C. (2022). Rubric development for AI-enabled scoring of three-dimensional constructed-response assessment aligned to NGSS learning progression. Frontiers in Education, 7, 1-15. https://doi.org/10.3389/feduc.2022.983055
• Kartal, G., & Yeşilyurt, Y.E. (2024). A bibliometric analysis of artificial intelligence in L2 teaching and applied linguistics between 1995 and 2022. ReCALL, 1 17. https://doi.org/10.1017/S0958344024000077
• Khoo, E., & Kang, S. (2022). Proactive learner empowerment: towards a transformative academic integrity approach for English language learners. International Journal for Educational Integrity, 18(1), 24. https://doi.org/10.1007/s40979-022-00111-2
• Kim, M., & Adlof, L. (2024). Adapting to the Future: ChatGPT as a Means for Supporting Constructivist Learning Environments. TechTrends, 68(1), 37 46. https://doi.org/10.1007/s11528-023-00899-x
• Kubsch, M., Czinczel, B., Lossjew, J., Wyrwich, T., Bednorz, D., Bernholt, S., Fiedler, … Rummel, N. (2022). Toward learning progression analytics - Developing learning environments for the automated analysis of learning using evidence centered design. Frontiers in Education, 7. https://doi.org/10.3389/feduc.2022.981910
• Kumar, V., & Boulanger, D. (2020). Explainable automated essay scoring: deep learning really has pedagogical value. Frontiers in Education, 5. https://doi.org/10.3389/feduc.2020.572367
• Lee, A.V.Y. (2023). Supporting students’ generation of feedback in large-scale online course with artificial intelligence-enabled evaluation. Studies in Educational Evaluation, 77, 101250. https://doi.org/10.1016/j.stueduc.2023.101250
• Lee, A.V.Y., Luco, A.C., & Tan, S.C. (2023). A human-centric automated essay scoring and feedback system for the development of ethical reasoning. Educational Technology & Society, 26(1), 147–159. https://doi.org/10.30191/ETS.202301_26(1).0011
• Lee, H.-Y., Chen, P.-H., Wang, W.-S., Huang, Y.-M., & Wu, T.-T. (2024). Empowering ChatGPT with guidance mechanism in blended learning: effect of self-regulated learning, higher-order thinking skills, and knowledge construction. International Journal of Educational Technology in Higher Education, 21(1), 16. https://doi.org/10.1186/s41239-024-00447-4
• Li, L., & Kim, M. (2024). It is like a friend to me: Critical usage of automated feedback systems by self-regulating English learners in higher education. Australasian Journal of Educational Technology, 40(1), 1–18. https://doi.org/10.14742/AJET.8821
• Li, T., Ji, Y., & Zhan, Z. (2024). Expert or machine? Comparing the effect of pairing student teacher with in-service teacher and ChatGPT on their critical thinking, learning performance, and cognitive load in an integrated-STEM course. Asia Pacific Journal of Education, 44(1), 45–60. https://doi.org/10.1080/02188791.2024.2305163
• Li, W., & Mohamad, M. (2023). An efficient probabilistic deep learning model for the oral proficiency assessment of student speech recognition and classification. International Journal on Recent and Innovation Trends in Computing and Communication, 11(6), 411–424. https://doi.org/10.17762/ijritcc.v11i6.7734
• Liang, H., Hwang, G., Hsu, T., & Yeh, J. (2024). Effect of an AI‐based chatbot on students’ learning performance in alternate reality game‐based museum learning. British Journal of Educational Technology. https://doi.org/10.1111/bjet.13448
• McLaren, B.M., DeLeeuw, KE., & Mayer, R.E. (2011). Polite web-based intelligent tutors: Can they improve learning in classrooms? Computers & Education, 56(3), 574–584. https://doi.org/10.1016/j.compedu.2010.09.019
• Mirchi, N., Bissonnette, V., Yilmaz, R., Ledwos, N., Winkler-Schwartz, A., & Del Maestro, R.F. (2020). The virtual operative assistant: An explainable artificial intelligence tool for simulation-based training in surgery and medicine. PLOS ONE, 15(2), e0229596. https://doi.org/10.1371/journal.pone.0229596
• Nazari, N., Shabbir, M.S., & Setiawan, R. (2021). Application of artificial intelligence powered digital writing assistant in higher education: randomized controlled trial. Heliyon, 7(5), e07014. https://doi.org/10.1016/j.heliyon.2021.e07014
• Nimy, E., Mosia, M., & Chibaya, C. (2023). Identifying at-risk students for early intervention-a probabilistic machine learning approach. Applied Sciences, 13(6), 3869. https://doi.org/10.3390/APP13063869
• Ouyang, F., Wu, M., Zheng, L., Zhang, L., & Jiao, P. (2023). Integration of artificial intelligence performance prediction and learning analytics to improve student learning in online engineering course. International Journal of Educational Technology in Higher Education, 20(1). https://doi.org/10.1186/s41239-022-00372-4
• Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. International Journal of Surgery, 88, 1 11. https://doi.org/10.1016/j.ijsu.2021.105906
• Palocsay, S.W., & Stevens, S.P. (2008). A study of the effectiveness of web‐based homework in teaching undergraduate business statistics. Decision Sciences Journal of Innovative Education, 6(2), 213–232. https://doi.org/10.1111/j.1540-4609.2008.00167.x
• Qiao, H., & Zhao, A. (2023). Artificial intelligence-based language learning: illuminating the impact on speaking skills and self-regulation in Chinese EFL context. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1255594
• Rad, H.S., Alipour, R., & Jafarpour, A. (2023). Using artificial intelligence to foster students’ writing feedback literacy, engagement, and outcome: a case of Wordtune application. Interactive Learning Environments, 1 21. https://doi.org/10.1080/10494820.2023.2208170
• Rahman, M.M., & Watanobe, Y. (2023). ChatGPT for education and research: opportunities, threats, and strategies. Applied Sciences, 13(9), 5783. https://doi.org/10.3390/app13095783
• Roldán-Álvarez, D., & Mesa, F.J. (2024). Intelligent deep-learning tutoring system to assist instructors in programming courses. IEEE Transactions on Education, 67(1), 153–161. https://doi.org/10.1109/TE.2023.3331055
• Rosé, C.P., McLaughlin, E.A., Liu, R., & Koedinger, K.R. (2019). Explanatory learner models: Why machine learning (alone) is not the answer. British Journal of Educational Technology, 50(6), 2943–2958. https://doi.org/10.1111/bjet.12858
• Rubio-Manzano, C., Lermanda Senoceaín, T., Martinez-Araneda, C., Vidal-Castro, C., & Segura-Navarrete, A. (2019). Fuzzy linguistic descriptions for execution trace comprehension and their application in an introductory course in artificial intelligence. Journal of Intelligent & Fuzzy Systems, 37(6), 8397–8415. https://doi.org/10.3233/JIFS-190935
• Saǧin, F.G., Özkaya, A.B., Tengiz, F., Geyik, Ö.G., & Geyik, C. (2023). Current evaluation and recommendations for the use of artificial intelligence tools in education. Turkish Journal of Biochemistry, 48(6), 620–625. https://doi.org/10.1515/tjb-2023-0254
• Sallam, M., Salim, N., Barakat, M., & Al-Tammemi, A. (2023). ChatGPT applications in medical, dental, pharmacy, and public health education: A descriptive study highlighting the advantages and limitations. Narra J, 3(1). https://doi.org/10.52225/narra.v3i1.103
• Shahriar, S., Allana, S., Hazratifard, S.M., & Dara, R. (2023). A survey of privacy risks and mitigation strategies in the artificial intelligence life cycle. IEEE Access, 11, 61829–61854. https://doi.org/10.1109/ACCESS.2023.3287195
• Sharma, K., Papamitsiou, Z., & Giannakos, M. (2019). Building pipelines for educational data using AI and multimodal analytics: A “grey‐box” approach. British Journal of Educational Technology, 50(6), 3004–3031. https://doi.org/10.1111/bjet.12854
• Shi, H., & Aryadoust, V. (2024). A systematic review of AI-based automated written feedback research. ReCALL, 1–23. https://doi.org/10.1017/S0958344023000265
• Song, P., & Wang, X. (2020). A bibliometric analysis of worldwide educational artificial intelligence research development in recent twenty years. Asia Pacific Education Review, 21(3), 473–486. https://doi.org/10.1007/s12564-020-09640-2
• Soofi, A.A., & Ahmed, M.U. (2019). A systematic review of domains, techniques, delivery modes and validation methods for intelligent tutoring systems. International Journal of Advanced Computer Science and Applications, 10(3), 99–107.
• Stojanov, A. (2023). Learning with ChatGPT 3.5 as a more knowledgeable other: an autoethnographic study. International Journal of Educational Technology in Higher Education, 20(1), 35. https://doi.org/10.1186/s41239-023-00404-7
• Su, J., & Yang, W. (2023). Unlocking the power of ChatGPT: a framework for applying generative AI in education. ECNU review of education, 6(3), 355 366. https://doi.org/10.1177/20965311231168423
• Su, Y., Lin, Y., & Lai, C. (2023). Collaborating with ChatGPT in argumentative writing classrooms. Assessing Writing, 57. https://doi.org/10.1016/j.asw.2023.100752
• Téllez, N.R., Villela, P.R., & Bautista, R.B. (2024). Evaluating ChatGPT-generated linear algebra formative assessments. International Journal of Interactive Multimedia and Artificial Intelligence, 8(5), 75–82. https://doi.org/10.9781/ijimai.2024.02.004
• Wang, L., Chen, X., Wang, C., Xu, L., Shadiev, R., & Li, Y. (2024). ChatGPT’s capabilities in providing feedback on undergraduate students’ argumentation: A case study. Thinking Skills and Creativity, 51, 101440. https://doi.org/10.1016/j.tsc.2023.101440
• Warschauer, M., & Grimes, D. (2008). Automated writing assessment in the classroom. pedagogies: An International Journal, 3(1), 22 36. https://doi.org/10.1080/15544800701771580
• Williams, R.T. (2024). The ethical implications of using generative chatbots in higher education. Frontiers in Education, 8. https://doi.org/10.3389/feduc.2023.1331607
• Wong, R.S.Y., Ming, L.C., & Ali, R.A.R. (2023). The intersection of ChatGPT, clinical medicine, and medical education. JMIR Medical Education, 9(1), 1–8. https://doi.org/10.2196/47274
• Wu, J.-Y., & Tsai, C.-C. (2022). Harnessing the power of promising technologies to transform science education: prospects and challenges to promote adaptive epistemic beliefs in science learning. International Journal of Science Education, 44(2), 346–353. https://doi.org/10.1080/09500693.2022.2028927
• Yavuz, F., Çelik, Ö., & Yavaş Çelik, G. (2024). Utilizing large language models for EFL essay grading: An examination of reliability and validity in rubric-based assessments. British Journal of Educational Technology, 00, 1–17. https://doi.org/10.1111/BJET.13494
• Zhang, W., Cai, M., Lee, H. J., Evans, R., Zhu, C., & Ming, C. (2024). AI in medical education: Global situation, effects and challenges. Education and Information Technologies, 29(4), 4611–4633. https://doi.org/10.1007/s10639-023-12009-8
• Zhao, R., Zhuang, Y., Zou, D., Xie, Q., & Yu, P.L.H. (2023). AI-assisted automated scoring of picture-cued writing tasks for language assessment. Education and Information Technologies, 28(6), 7031–7063. https://doi.org/10.1007/s10639-022-11473-y
• Zheng, L., Zhong, L., Niu, J., Long, M., & Zhao, J. (2021). Effects of personalized intervention on collaborative knowledge building, group performance, socially shared metacognitive regulation, and cognitive load in computer-supported collaborative learning. Educational Technology & Society, 24(3), 174–193.