AI-based feedback tools in education: A comprehensive bibliometric analysis study

Öz

This bibliometric analysis offers a comprehensive examination of AI-based feedback tools in education, utilizing data retrieved from the Web of Science (WoS) database. Encompassing a total of 239 articles from an expansive timeframe, spanning from inception to February 2024, this study provides a thorough overview of the evolution and current state of research in this domain. Through meticulous analysis, it tracks the growth trajectory of publications over time, revealing the increasing scholarly attention towards AI-driven feedback mechanisms in educational contexts. By describing critical thematic areas such as the role of feedback in enhancing learning outcomes, the integration of AI technologies into educational practices, and the efficacy of AI-based feedback tools in facilitating personalized learning experiences, the analysis offers valuable insights into the multifaceted nature of this field. By employing sophisticated bibliometric mapping techniques, including co-citation analysis and keyword co-occurrence analysis, the study uncovers the underlying intellectual structure of the research landscape, identifying prominent themes, influential articles, and emerging trends. Furthermore, it identifies productive authors, institutions, and countries contributing to the discourse, providing a detailed understanding of the collaborative networks and citation patterns within the community. This comprehensive synthesis of the literature serves as a valuable resource for researchers, practitioners, and policymakers alike, offering guidance on harnessing the potential of AI technologies to revolutionize teaching and learning practices in education.

Anahtar Kelimeler

• AI-driven feedback
• Educational integration
• Learning enhancement
• Personalized learning
• Bibliometric analysis

References

1. Afzaal, M., Zia, A., Nouri, J., & Fors, U. (2024). Informative feedback and explainable ai-based recommendations to support students’ self-regulation. Technology, Knowledge and Learning, 29(1), 331–354. https://doi.org/10.1007/s10758-023-09650-0
2. Aria, M., & Cuccurullo, C. (2017). bibliometrix: An R-tool for comprehensive science mapping analysis. Journal of Informetrics, 11(4), 959 975. https://doi.org/10.1016/j.joi.2017.08.007
3. Bañeres, D., Rodríguez, M.E., Guerrero-Roldán, A.E., & Karadeniz, A. (2020). An early warning system to detect at-risk students in online higher education. Applied Sciences (Switzerland), 10(13). https://doi.org/10.3390/app10134427
4. Barrett, A., & Pack, A. (2023). Not quite eye to A.I.: student and teacher perspectives on the use of generative artificial intelligence in the writing process. International Journal of Educational Technology in Higher Education, 20(1), 59. https://doi.org/10.1186/s41239-023-00427-0
5. Bin-Hady, W.R.A., Al-Kadi, A., Hazaea, A., & Ali, J.K.M. (2023). Exploring the dimensions of ChatGPT in English language learning: a global perspective. Library Hi Tech. https://doi.org/10.1108/LHT-05-2023-0200
6. Bui, N.M., & Barrot, J.S. (2024). ChatGPT as an automated essay scoring tool in the writing classrooms: how it compares with human scoring. Education and Information Technologies, 1–18. https://doi.org/10.1007/S10639-024-12891-W/TABLES/5
7. Chang, D.H., Lin, M.P.-C., Hajian, S., & Wang, Q.Q. (2023). Educational design principles of using AI Chatbot that supports self-regulated learning in education: Goal setting, feedback, and personalization. Sustainability, 15(17), 12921. https://doi.org/10.3390/su151712921
8. Chavez, H., Chavez-Arias, B., Contreras-Rosas, S., Alvarez-Rodríguez, J.M., & Raymundo, C. (2023). Artificial neural network model to predict student performance using nonpersonal information. Frontiers in Education, 8. https://doi.org/10.3389/FEDUC.2023.1106679

9. Chen, X., Zou, D., Xie, H., Chen, G., Lin, J., & Cheng, G. (2023). Exploring contributors, collaborations, and research topics in educational technology: A joint analysis of mainstream conferences. Education and Information Technologies, 28(2), 1323–1358. https://doi.org/10.1007/s10639-022-11209-y
10. Chen, Z. (2023). Artificial intelligence-virtual trainer: innovative didactics aimed at personalized training needs. Journal of the Knowledge Economy, 14(2), 2007–2025. https://doi.org/10.1007/s13132-022-00985-0
11. Chin, D.B., Dohmen, I.M., Cheng, B.H., Oppezzo, M.A., Chase, C.C., & Schwartz, D.L. (2010). Preparing students for future learning with Teachable Agents. Educational Technology Research and Development, 58(6), 649–669. https://doi.org/10.1007/s11423-010-9154-5
12. Chiu, M.-C., Hwang, G.-J., Hsia, L.-H., & Shyu, F.-M. (2022). Artificial intelligence-supported art education: a deep learning-based system for promoting university students’ artwork appreciation and painting outcomes. Interactive Learning Environments, 1–19. https://doi.org/10.1080/10494820.2022.2100426
13. Conrad, E.J., & Hall, K.C. (2024). Leveraging generative AI to elevate curriculum design and pedagogy in public health and health promotion. Pedagogy in Health Promotion. https://doi.org/10.1177/23733799241232641
14. Cowling, M., Crawford, J., Allen, K.-A., & Wehmeyer, M. (2023). Using leadership to leverage ChatGPT and artificial intelligence for undergraduate and postgraduate research supervision. Australasian Journal of Educational Technology, 39(4), 89 103. https://doi.org/10.14742/ajet.8598
15. Cukurova, M., Kent, C., & Luckin, R. (2019). Artificial intelligence and multimodal data in the service of human decision‐making: A case study in debate tutoring. British Journal of Educational Technology, 50(6), 3032–3046. https://doi.org/10.1111/bjet.12829
16. Ding, L., & Zou, D. (2024). Automated writing evaluation systems: A systematic review of Grammarly, Pigai, and Criterion with a perspective on future directions in the age of generative artificial intelligence. Education and Information Technologies. https://doi.org/10.1007/s10639-023-12402-3
17. Ebenbeck, N., & Gebhardt, M. (2024). Differential Performance of Computerized Adaptive Testing in Students With and Without Disabilities A Simulation Study. Journal of Special Education Technology. https://doi.org/10.1177/01626434241232117
18. Elmaoğlu, E., Coşkun, A.B., & Yüzer Alsaç, S. (2024). Digital Transformation: The Role, Potential, and Limitations of ChatGPT in Child Health Education. American Journal of Health Education, 55(1), 69–72. https://doi.org/10.1080/19325037.2023.2277937
19. Farshad, S., Zorin, E., Amangeldiuly, N., & Fortin, C. (2023). Engagement assessment in project-based education: a machine learning approach in team chat analysis. Education and Information Technologies. https://doi.org/10.1007/s10639-023-12381-5
20. Fu, S., Gu, H., & Yang, B. (2020). The affordances of AI‐enabled automatic scoring applications on learners’ continuous learning intention: An empirical study in China. British Journal of Educational Technology, 51(5), 1674–1692. https://doi.org/10.1111/bjet.12995
21. Gao, R., Merzdorf, H.E., Anwar, S., Hipwell, M.C., & Srinivasa, A.R. (2024). Automatic assessment of text-based responses in post-secondary education: A systematic review. Computers and Education: Artificial Intelligence, 6, 100206. https://doi.org/10.1016/j.caeai.2024.100206
22. Gligorea, I., Cioca, M., Oancea, R., Gorski, A.T., Gorski, H., & Tudorache, P. (2023). Adaptive Learning Using Artificial Intelligence in e-Learning: A Literature Review. Education Sciences, 13(12), 1216. https://doi.org/10.3390/EDUCSCI13121216
23. Gu, Y. (2024). Research on Speech Communication Enhancement of English Web-based Learning Platform based on Human-computer Intelligent Interaction. Scalable Computing: Practice and Experience, 25(2), 709 720. https://doi.org/10.12694/scpe.v25i2.2544
24. Heeg, D.M., & Avraamidou, L. (2023). The use of Artificial intelligence in school science: a systematic literature review. Educational Media International, 60(2), 125–150. https://doi.org/10.1080/09523987.2023.2264990
25. Hopfenbeck, T.N., Zhang, Z., Sun, S.Z., Robertson, P., & McGrane, J.A. (2023). Challenges and opportunities for classroom-based formative assessment and AI: a perspective article. Frontiers in Education, 8. https://doi.org/10.3389/feduc.2023.1270700
26. Hopgood, A.A., & Hirst, A.J. (2007). Keeping a Distance-Education Course Current Through eLearning and Contextual Assessment. IEEE Transactions on Education, 50(1), 85–96. https://doi.org/10.1109/TE.2006.888905
27. Jaleniauskienė, E., Lisaitė, D., & Daniusevičiūtė-Brazaitė, L. (2023). Artificial Intelligence in Language Education: A Bibliometric Analysis. Sustainable Multilingualism, 23(1), 159–194. https://doi.org/10.2478/sm-2023-0017
28. Kaldaras, L., Yoshida, N.R., & Haudek, K.C. (2022). Rubric development for AI-enabled scoring of three-dimensional constructed-response assessment aligned to NGSS learning progression. Frontiers in Education, 7, 1-15. https://doi.org/10.3389/feduc.2022.983055
29. Kartal, G., & Yeşilyurt, Y.E. (2024). A bibliometric analysis of artificial intelligence in L2 teaching and applied linguistics between 1995 and 2022. ReCALL, 1 17. https://doi.org/10.1017/S0958344024000077
30. Khoo, E., & Kang, S. (2022). Proactive learner empowerment: towards a transformative academic integrity approach for English language learners. International Journal for Educational Integrity, 18(1), 24. https://doi.org/10.1007/s40979-022-00111-2
31. Kim, M., & Adlof, L. (2024). Adapting to the Future: ChatGPT as a Means for Supporting Constructivist Learning Environments. TechTrends, 68(1), 37 46. https://doi.org/10.1007/s11528-023-00899-x
32. Kubsch, M., Czinczel, B., Lossjew, J., Wyrwich, T., Bednorz, D., Bernholt, S., Fiedler, … Rummel, N. (2022). Toward learning progression analytics - Developing learning environments for the automated analysis of learning using evidence centered design. Frontiers in Education, 7. https://doi.org/10.3389/feduc.2022.981910
33. Kumar, V., & Boulanger, D. (2020). Explainable automated essay scoring: deep learning really has pedagogical value. Frontiers in Education, 5. https://doi.org/10.3389/feduc.2020.572367
34. Lee, A.V.Y. (2023). Supporting students’ generation of feedback in large-scale online course with artificial intelligence-enabled evaluation. Studies in Educational Evaluation, 77, 101250. https://doi.org/10.1016/j.stueduc.2023.101250
35. Lee, A.V.Y., Luco, A.C., & Tan, S.C. (2023). A human-centric automated essay scoring and feedback system for the development of ethical reasoning. Educational Technology & Society, 26(1), 147–159. https://doi.org/10.30191/ETS.202301_26(1).0011
36. Lee, H.-Y., Chen, P.-H., Wang, W.-S., Huang, Y.-M., & Wu, T.-T. (2024). Empowering ChatGPT with guidance mechanism in blended learning: effect of self-regulated learning, higher-order thinking skills, and knowledge construction. International Journal of Educational Technology in Higher Education, 21(1), 16. https://doi.org/10.1186/s41239-024-00447-4
37. Li, L., & Kim, M. (2024). It is like a friend to me: Critical usage of automated feedback systems by self-regulating English learners in higher education. Australasian Journal of Educational Technology, 40(1), 1–18. https://doi.org/10.14742/AJET.8821
38. Li, T., Ji, Y., & Zhan, Z. (2024). Expert or machine? Comparing the effect of pairing student teacher with in-service teacher and ChatGPT on their critical thinking, learning performance, and cognitive load in an integrated-STEM course. Asia Pacific Journal of Education, 44(1), 45–60. https://doi.org/10.1080/02188791.2024.2305163
39. Li, W., & Mohamad, M. (2023). An efficient probabilistic deep learning model for the oral proficiency assessment of student speech recognition and classification. International Journal on Recent and Innovation Trends in Computing and Communication, 11(6), 411–424. https://doi.org/10.17762/ijritcc.v11i6.7734
40. Liang, H., Hwang, G., Hsu, T., & Yeh, J. (2024). Effect of an AI‐based chatbot on students’ learning performance in alternate reality game‐based museum learning. British Journal of Educational Technology. https://doi.org/10.1111/bjet.13448
41. McLaren, B.M., DeLeeuw, KE., & Mayer, R.E. (2011). Polite web-based intelligent tutors: Can they improve learning in classrooms? Computers & Education, 56(3), 574–584. https://doi.org/10.1016/j.compedu.2010.09.019
42. Mirchi, N., Bissonnette, V., Yilmaz, R., Ledwos, N., Winkler-Schwartz, A., & Del Maestro, R.F. (2020). The virtual operative assistant: An explainable artificial intelligence tool for simulation-based training in surgery and medicine. PLOS ONE, 15(2), e0229596. https://doi.org/10.1371/journal.pone.0229596
43. Nazari, N., Shabbir, M.S., & Setiawan, R. (2021). Application of artificial intelligence powered digital writing assistant in higher education: randomized controlled trial. Heliyon, 7(5), e07014. https://doi.org/10.1016/j.heliyon.2021.e07014
44. Nimy, E., Mosia, M., & Chibaya, C. (2023). Identifying at-risk students for early intervention-a probabilistic machine learning approach. Applied Sciences, 13(6), 3869. https://doi.org/10.3390/APP13063869
45. Ouyang, F., Wu, M., Zheng, L., Zhang, L., & Jiao, P. (2023). Integration of artificial intelligence performance prediction and learning analytics to improve student learning in online engineering course. International Journal of Educational Technology in Higher Education, 20(1). https://doi.org/10.1186/s41239-022-00372-4
46. Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. International Journal of Surgery, 88, 1 11. https://doi.org/10.1016/j.ijsu.2021.105906
47. Palocsay, S.W., & Stevens, S.P. (2008). A study of the effectiveness of web‐based homework in teaching undergraduate business statistics. Decision Sciences Journal of Innovative Education, 6(2), 213–232. https://doi.org/10.1111/j.1540-4609.2008.00167.x
48. Qiao, H., & Zhao, A. (2023). Artificial intelligence-based language learning: illuminating the impact on speaking skills and self-regulation in Chinese EFL context. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1255594
49. Rad, H.S., Alipour, R., & Jafarpour, A. (2023). Using artificial intelligence to foster students’ writing feedback literacy, engagement, and outcome: a case of Wordtune application. Interactive Learning Environments, 1 21. https://doi.org/10.1080/10494820.2023.2208170
50. Rahman, M.M., & Watanobe, Y. (2023). ChatGPT for education and research: opportunities, threats, and strategies. Applied Sciences, 13(9), 5783. https://doi.org/10.3390/app13095783
51. Roldán-Álvarez, D., & Mesa, F.J. (2024). Intelligent deep-learning tutoring system to assist instructors in programming courses. IEEE Transactions on Education, 67(1), 153–161. https://doi.org/10.1109/TE.2023.3331055
52. Rosé, C.P., McLaughlin, E.A., Liu, R., & Koedinger, K.R. (2019). Explanatory learner models: Why machine learning (alone) is not the answer. British Journal of Educational Technology, 50(6), 2943–2958. https://doi.org/10.1111/bjet.12858
53. Rubio-Manzano, C., Lermanda Senoceaín, T., Martinez-Araneda, C., Vidal-Castro, C., & Segura-Navarrete, A. (2019). Fuzzy linguistic descriptions for execution trace comprehension and their application in an introductory course in artificial intelligence. Journal of Intelligent & Fuzzy Systems, 37(6), 8397–8415. https://doi.org/10.3233/JIFS-190935
54. Saǧin, F.G., Özkaya, A.B., Tengiz, F., Geyik, Ö.G., & Geyik, C. (2023). Current evaluation and recommendations for the use of artificial intelligence tools in education. Turkish Journal of Biochemistry, 48(6), 620–625. https://doi.org/10.1515/tjb-2023-0254
55. Sallam, M., Salim, N., Barakat, M., & Al-Tammemi, A. (2023). ChatGPT applications in medical, dental, pharmacy, and public health education: A descriptive study highlighting the advantages and limitations. Narra J, 3(1). https://doi.org/10.52225/narra.v3i1.103
56. Shahriar, S., Allana, S., Hazratifard, S.M., & Dara, R. (2023). A survey of privacy risks and mitigation strategies in the artificial intelligence life cycle. IEEE Access, 11, 61829–61854. https://doi.org/10.1109/ACCESS.2023.3287195
57. Sharma, K., Papamitsiou, Z., & Giannakos, M. (2019). Building pipelines for educational data using AI and multimodal analytics: A “grey‐box” approach. British Journal of Educational Technology, 50(6), 3004–3031. https://doi.org/10.1111/bjet.12854
58. Shi, H., & Aryadoust, V. (2024). A systematic review of AI-based automated written feedback research. ReCALL, 1–23. https://doi.org/10.1017/S0958344023000265
59. Song, P., & Wang, X. (2020). A bibliometric analysis of worldwide educational artificial intelligence research development in recent twenty years. Asia Pacific Education Review, 21(3), 473–486. https://doi.org/10.1007/s12564-020-09640-2
60. Soofi, A.A., & Ahmed, M.U. (2019). A systematic review of domains, techniques, delivery modes and validation methods for intelligent tutoring systems. International Journal of Advanced Computer Science and Applications, 10(3), 99–107.
61. Stojanov, A. (2023). Learning with ChatGPT 3.5 as a more knowledgeable other: an autoethnographic study. International Journal of Educational Technology in Higher Education, 20(1), 35. https://doi.org/10.1186/s41239-023-00404-7
62. Su, J., & Yang, W. (2023). Unlocking the power of ChatGPT: a framework for applying generative AI in education. ECNU review of education, 6(3), 355 366. https://doi.org/10.1177/20965311231168423
63. Su, Y., Lin, Y., & Lai, C. (2023). Collaborating with ChatGPT in argumentative writing classrooms. Assessing Writing, 57. https://doi.org/10.1016/j.asw.2023.100752
64. Téllez, N.R., Villela, P.R., & Bautista, R.B. (2024). Evaluating ChatGPT-generated linear algebra formative assessments. International Journal of Interactive Multimedia and Artificial Intelligence, 8(5), 75–82. https://doi.org/10.9781/ijimai.2024.02.004
65. Wang, L., Chen, X., Wang, C., Xu, L., Shadiev, R., & Li, Y. (2024). ChatGPT’s capabilities in providing feedback on undergraduate students’ argumentation: A case study. Thinking Skills and Creativity, 51, 101440. https://doi.org/10.1016/j.tsc.2023.101440
66. Warschauer, M., & Grimes, D. (2008). Automated writing assessment in the classroom. pedagogies: An International Journal, 3(1), 22 36. https://doi.org/10.1080/15544800701771580
67. Williams, R.T. (2024). The ethical implications of using generative chatbots in higher education. Frontiers in Education, 8. https://doi.org/10.3389/feduc.2023.1331607
68. Wong, R.S.Y., Ming, L.C., & Ali, R.A.R. (2023). The intersection of ChatGPT, clinical medicine, and medical education. JMIR Medical Education, 9(1), 1–8. https://doi.org/10.2196/47274
69. Wu, J.-Y., & Tsai, C.-C. (2022). Harnessing the power of promising technologies to transform science education: prospects and challenges to promote adaptive epistemic beliefs in science learning. International Journal of Science Education, 44(2), 346–353. https://doi.org/10.1080/09500693.2022.2028927
70. Yavuz, F., Çelik, Ö., & Yavaş Çelik, G. (2024). Utilizing large language models for EFL essay grading: An examination of reliability and validity in rubric-based assessments. British Journal of Educational Technology, 00, 1–17. https://doi.org/10.1111/BJET.13494
71. Zhang, W., Cai, M., Lee, H. J., Evans, R., Zhu, C., & Ming, C. (2024). AI in medical education: Global situation, effects and challenges. Education and Information Technologies, 29(4), 4611–4633. https://doi.org/10.1007/s10639-023-12009-8
72. Zhao, R., Zhuang, Y., Zou, D., Xie, Q., & Yu, P.L.H. (2023). AI-assisted automated scoring of picture-cued writing tasks for language assessment. Education and Information Technologies, 28(6), 7031–7063. https://doi.org/10.1007/s10639-022-11473-y
73. Zheng, L., Zhong, L., Niu, J., Long, M., & Zhao, J. (2021). Effects of personalized intervention on collaborative knowledge building, group performance, socially shared metacognitive regulation, and cognitive load in computer-supported collaborative learning. Educational Technology & Society, 24(3), 174–193.