Research trends and the impact of ChatGPT on educational environments

Year 2025, Volume: 11 Issue: 1, 45 - 62, 22.03.2025

Thoriqi Firdaus , Rizqoh Mufidah , Rika Nur Hamida , R'maya Inkya Febrianti , Alvira Eka Rahel Guivara

https://doi.org/10.24289/ijsser.1609741

https://izlik.org/JA96CC49AZ

Abstract

This study aims to explore research trends and patterns and analyze ChatGPT's impact on education. The methodology employs a mixed-method approach, incorporating bibliometric analysis and a systematic literature review. Research data were sourced from the Scopus database using the keywords "ChatGPT" AND "Education" OR "Learning." The findings indicate that the trend of document publications in the Scopus database related to ChatGPT has seen a notable increase since its introduction in 2022, continuing through 2024. The journal JMIR Medical Education has emerged as the foremost source of citations, making significant contributions. The United States leads the way in article contributions (22.6%), followed by China (9.6%). Countries such as the United Kingdom, Canada, and Italy display high levels of international collaboration, likely enhancing the diversification and quality of research.

Keywords

ChatGPT , Education , Learning , Trends , Impact

Thanks

I would like to thank Academy of Science and Cultural Literacy Research Community (KOPI ALINEA) for supporting this research.

References

• Alanzi, T. M. (2023). Impact of ChatGPT on Teleconsultants in Healthcare: Perceptions of Healthcare Experts in Saudi Arabia. Journal of Multidisciplinary Healthcare, 16, 2309–2321. https://doi.org/10.2147/JMDH.S419847
• Al Ghazali, S., Zaki, N., Ali, L., & Harous, S. (2024). Exploring the potential of ChatGPT as a Substitute teacher: A Case Study. International Journal of Information and Education Technology, 14(2), 271-278.
• Al Lily, A. E., Ismail, A. F., Abunaser, F. M., Al-Lami, F., & Abdullatif, A. K. A. (2023). ChatGPT and the rise of semi-humans. Humanities and Social Sciences Communications, 10(1), 1-12.
• Avello-Martínez, R., Gajderowicz, T., & Gómez-Rodríguez, V. G. (2024). Is ChatGPT helpful for graduate students in acquiring knowledge about digital storytelling and reducing their cognitive load? An experiment. Revista de Educación a Distancia, 24(78). https://doi.org/10.6018/red.604621
• Castillo, A. G. R., Rivera, H. V. H., Teves, R. M. V., Lopez, H. R. P., Reyes, G. Y., Rodriguez, M. A. M., ... & Arias-Gonzáles, J. L. (2023). Effect of Chat GPT on the digitized learning process of university students. Journal of Namibian Studies: History Politics Culture, 33, 1-15.
• Choi, W. (2023). Assessment of the capacity of ChatGPT as a self-learning tool in medical pharmacology: a study using MCQs. BMC Medical Education, 23(1). https://doi.org/10.1186/s12909-023-04832-x
• Cohen, A., Alter, R., Lessans, N., Meyer, R., Brezinov, Y., & Levin, G. (2023). Performance of ChatGPT in Israeli Hebrew OBGYN national residency examinations. Archives of Gynecology and Obstetrics, 308(6), 1797–1802. https://doi.org/10.1007/s00404-023-07185-4
• Collins, K. M., Jiang, A. Q., Frieder, S., Wong, L., Zilka, M., Bhatt, U., ... & Jamnik, M. (2024). Evaluating language models for mathematics through interactions. Proceedings of the National Academy of Sciences, 121(24), e2318124121.
• Currie, G., & Barry, K. (2023). ChatGPT in nuclear medicine education. Journal of Nuclear Medicine Technology, 51(3), 247-254.
• Dalalah, D., & Dalalah, O. M. (2023). The false positives and false negatives of generative AI detection tools in education and academic research: The case of ChatGPT. The International Journal of Management Education, 21(2), 1-12.
• Dalgıç, A., Yaşar, E., & Demir, M. (2024). ChatGPT and learning outcomes in tourism education: The role of digital literacy and individualized learning. Journal of Hospitality, Leisure, Sport & Tourism Education, 34, 1-13.
• Dindorf, C., Bartaguiz, E., Gassmann, F., & Fröhlich, M. (2022). Conceptual structure and current trends in artificial intelligence, machine learning, and deep learning research in sports: a bibliometric review. International Journal of Environmental Research and Public Health, 20(1), 173.
• Donthu, N., Kumar, S., Mukherjee, D., Pandey, N., & Lim, W. M. (2021). How to conduct a bibliometric analysis: An overview and guidelines. Journal of business research, 133, 285-296.
• Dowling, M., & Lucey, B. (2023). ChatGPT for (finance) research: The Bananarama conjecture. Finance Research Letters, 53, 103662.
• Duarte, A. V., Zhao, X., Oliveira, A. L., & Li, L. (2024). De-cop: Detecting copyrighted content in language models training data. arXiv preprint arXiv:2402.09910.
• Dubiel, M., Barghouti, Y., Kudryavtseva, K., & Leiva, L. A. (2024). On-device query intent prediction with lightweight LLMs to support ubiquitous conversations. Scientific Reports, 14(1). https://doi.org/10.1038/s41598-024-63380-6
• Einarsson, H., Lund, S. H., & Jónsdóttir, A. H. (2024). Application of ChatGPT for automated problem reframing across academic domains. Computers and Education: Artificial Intelligence, 6, 100194.
• ElSayary, A. (2024). An investigation of teachers' perceptions of using ChatGPT as a supporting tool for teaching and learning in the digital era. Journal of computer assisted learning, 40(3), 931-945.
• Farazouli, A., Cerratto-Pargman, T., Bolander-Laksov, K., & McGrath, C. (2024). Hello GPT! Goodbye home examination? An exploratory study of AI chatbots impact on university teachers’ assessment practices. Assessment and Evaluation in Higher Education, 49(3), 363–375. https://doi.org/10.1080/02602938.2023.2241676
• Firdaus, T. (2023). Representative platform cyber metaverse terkoneksi BYOD sebagai upaya preventive urgensi digital pada sistem pendidikan Indonesia. Jurnal Integrasi dan Harmoni Inovatif Ilmu-Ilmu Sosial, 3(2), 123-131. https://doi.org/10.17977/um063v3i2p123-131
• Firdaus, T., Sholeha, S. A., Jannah, M., & Setiawan, A. R. (2024). Comparison of ChatGPT and Gemini AI in Answering Higher-Order Thinking Skill Biology Questions: Accuracy and Evaluation. International Journal of Science Education and Teaching, 3(3), 126-138. https://doi.org/10.14456/ijset.2024.11
• Fütterer, T., Fischer, C., Alekseeva, A., Chen, X., Tate, T., Warschauer, M., & Gerjets, P. (2023). ChatGPT in education: global reactions to AI innovations. Scientific reports, 13(1), 15310.
• George, A. S., George, A. H., & Martin, A. G. (2023). The environmental impact of AI: a case study of water consumption by chat GPT. Partners Universal International Innovation Journal, 1(2), 97-104.
• Gill, S. S., Xu, M., Patros, P., Wu, H., Kaur, R., Kaur, K., ... & Buyya, R. (2024). Transformative effects of ChatGPT on modern education: Emerging Era of AI Chatbots. Internet of Things and Cyber-Physical Systems, 4, 19-23.
• Gobira, M., Nakayama, L. F., Moreira, R., Andrade, E., Regatieri, C. V. S., & Belfort, R. (2023). Performance of ChatGPT-4 in answering questions from the Brazilian National Examination for Medical Degree Revalidation. Revista Da Associacao Medica Brasileira, 69(10). https://doi.org/10.1590/1806-9282.20230848 Groza, T., Caufield, H., Gration, D., Baynam, G., Haendel, M. A., Robinson, P. N., ... & Reese, J. T. (2024). An evaluation of GPT models for phenotype concept recognition. BMC Medical Informatics and Decision Making, 24(1), 30.
• Hosseini, M., Gao, C. A., Liebovitz, D. M., Carvalho, A. M., Ahmad, F. S., Luo, Y., MacDonald, N., Holmes, K. L., & Kho, A. (2023). An exploratory survey about using ChatGPT in education, healthcare, and research. PLoS ONE, 18(10 October). https://doi.org/10.1371/journal.pone.0292216
• Javaid, M., Haleem, A., Singh, R. P., Khan, S., & Khan, I. H. (2023). Unlocking the opportunities through ChatGPT Tool towards ameliorating the education system. BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 3(2), 100115.
• Jo, M. H., Kim, M. J., Oh, H. K., Choi, M. J., Shin, H. R., Lee, T. G., ... & Kang, S. B. (2024). Communicative competence of generative artificial intelligence in responding to patient queries about colorectal cancer surgery. International Journal of Colorectal Disease, 39(1), 94.
• Jošt, G., Taneski, V., & Karakatič, S. (2024). The Impact of Large Language Models on Programming Education and Student Learning Outcomes. Applied Sciences, 14(10), 4115.
• Kabir, S., Udo-Imeh, D. N., Kou, B., & Zhang, T. (2024, May). Is stack overflow obsolete? an empirical study of the characteristics of ChatGPT answers to stack overflow questions. In Proceedings of the CHI Conference on Human Factors in Computing Systems (pp. 1-17).
• Kamalov, F., Santandreu Calonge, D., & Gurrib, I. (2023). New era of artificial intelligence in education: Towards a sustainable multifaceted revolution. Sustainability, 15(16), 12451.
• Kämmer, J. E., Hautz, W. E., Krummrey, G., Sauter, T. C., Penders, D., Birrenbach, T., & Bienefeld, N. (2024). Effects of interacting with a large language model compared with a human coach on the clinical diagnostic process and outcomes among fourth-year medical students: study protocol for a prospective, randomised experiment using patient vignettes. BMJ Open, 14(7), e087469. https://doi.org/10.1136/bmjopen-2024-087469
• Karataş, F., Abedi, F. Y., Ozek Gunyel, F., Karadeniz, D., & Kuzgun, Y. (2024). Incorporating AI in foreign language education: An investigation into ChatGPT’s effect on foreign language learners. Education and Information Technologies, 1-24.
• Kosar, T., Ostojić, D., Liu, Y. D., & Mernik, M. (2024). Computer Science Education in ChatGPT Era: Experiences from an Experiment in a Programming Course for Novice Programmers. Mathematics, 12(5). https://doi.org/10.3390/math12050629
• Kufel, J., Paszkiewicz, I., Bielówka, M., Bartnikowska, W., Janik, M., Stencel, M., Czogalik, Ł., Gruszczyńska, K., & Mielcarska, S. (2023). Will ChatGPT pass the Polish specialty exam in radiology and diagnostic imaging? Insights into strengths and limitations. Polish Journal of Radiology, 88(1), e430–e434. https://doi.org/10.5114/pjr.2023.131215
• Li, H. (2023). Effects of a ChatGPT-based flipped learning guiding approach on learners’ courseware project performances and perceptions. Australasian Journal of Educational Technology, 39(5), 40-58.
• Limna, P., Kraiwanit, T., Jangjarat, K., Klayklung, P., & Chocksathaporn, P. (2023). The use of ChatGPT in the digital era: Perspectives on chatbot implementation. Journal of Applied Learning and Teaching, 6(1), 64-74.
• Lyu, Q., Tan, J., Zapadka, M. E., Ponnatapura, J., Niu, C., Myers, K. J., Wang, G., & Whitlow, C. T. (2023). Translating radiology reports into plain language using ChatGPT and GPT-4 with prompt learning: results, limitations, and potential. Visual Computing for Industry, Biomedicine, and Art, 6(1). https://doi.org/10.1186/s42492-023-00136-5
• Ng, R., & Chow, T. Y. J. (2024). Powerful tool or too powerful? Early public discourse about ChatGPT across 4 million tweets. Plos one, 19(3), e0296882.
• Nightingale, A. (2009). A guide to systematic literature reviews. Surgery (Oxford), 27(9), 381-384.
• Noy, S., & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187-192.
• OpenAi. (2022). Introducing ChatGPT (November 30, 2022). https://openai.com/index/ChatGPT/
• OpenAi. (2024). Introducing GPT-4o and more tools to ChatGPT free users (May 13, 2024). https://openai.com/index/gpt-4o-and-more-tools-to-ChatGPT-free/
• Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., ... & Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. bmj, 372.
• Papastratis, I., Stergioulas, A., Konstantinidis, D., Daras, P., & Dimitropoulos, K. (2024). Can ChatGPT provide appropriate meal plans for NCD patients?. Nutrition, 121, 112291.
• Piccolo, S. R., Denny, P., Luxton-Reilly, A., Payne, S. H., & Ridge, P. G. (2023). Evaluating a large language model’s ability to solve programming exercises from an introductory bioinformatics course. PLoS Computational Biology, 19(9 September). https://doi.org/10.1371/journal.pcbi.1011511
• Rahman, M. M., & Watanobe, Y. (2023). ChatGPT for Education and Research: Opportunities, Threats, and Strategies. Applied Sciences (Switzerland), 13(9). https://doi.org/10.3390/app13095783
• Sabir, A., Ali, H. A., & Aljabery, M. A. (2024). ChatGPT Tweets Sentiment Analysis Using Machine Learning and Data Classification. Informatica (Slovenia), 48(7), 103–112. https://doi.org/10.31449/inf.v48i7.5535
• Sadiku, M. N., Musa, S. M., & Chukwu, U. C. (2022). Artificial intelligence in education. iUniverse.
• Shoufan, A. (2023). Exploring students’ perceptions of ChatGPT: Thematic analysis and follow-up survey. IEEE Access, 11, 38805-38818.
• Similiarweb. (2024). Chatopenai. https://www.similarweb.com/website/chat.openai.com/#traffic
• Sullivan, M., Kelly, A., & McLaughlan, P. (2023). ChatGPT in higher education: Considerations for academic integrity and student learning.
• Srinivasan, N., Samaan, J. S., Rajeev, N. D., Kanu, M. U., Yeo, Y. H., & Samakar, K. (2024). Large language models and bariatric surgery patient education: a comparative readability analysis of GPT-3.5, GPT-4, Bard, and online institutional resources. Surgical Endoscopy, 38(5), 2522-2532.
• Sun, H. (2024). Multi-scenario application of ChatGPT-based language modeling for empowering English language teaching and learning. Applied Mathematics and Nonlinear Sciences, 9(1). https://doi.org/10.2478/amns-2024-0790
• Taloni, A., Borselli, M., Scarsi, V., Rossi, C., Coco, G., Scorcia, V., & Giannaccare, G. (2023). Comparative performance of humans versus GPT-4.0 and GPT-3.5 in the self-assessment program of American Academy of Ophthalmology. Scientific Reports, 13(1). https://doi.org/10.1038/s41598-023-45837-2
• Tang, L., Sun, Z., Idnay, B., Nestor, J. G., Soroush, A., Elias, P. A., Xu, Z., Ding, Y., Durrett, G., Rousseau, J. F., Weng, C., & Peng, Y. (2023). Evaluating large language models on medical evidence summarization. Npj Digital Medicine, 6(1). https://doi.org/10.1038/s41746-023-00896-7
• Tirado-Olivares, S., Navío-Inglés, M., O’Connor-Jiménez, P., & Cózar-Gutiérrez, R. (2023). From human to machine: investigating the effectiveness of the conversational AI ChatGPT in historical thinking. Education Sciences, 13(8), 803.
• Tong, W., Guan, Y., Chen, J., Huang, X., Zhong, Y., Zhang, C., & Zhang, H. (2023). Artificial intelligence in global health equity: an evaluation and discussion on the application of ChatGPT, in the Chinese National Medical Licensing Examination. Frontiers in Medicine, 10. https://doi.org/10.3389/fmed.2023.1237432
• Tsai, C. Y., Hsieh, S. J., Huang, H. H., Deng, J. H., Huang, Y. Y., & Cheng, P. Y. (2024). Performance of ChatGPT on the Taiwan urology board examination: insights into current strengths and shortcomings. World Journal of Urology, 42(1). https://doi.org/10.1007/s00345-024-04957-8
• Tyni, J., Turunen, A., Bednarik, R., Kahila, J., & Tedre, M. (2024). International Journal of Serious Games Can ChatGPT Match Experts? Comparing input for Serious Game Development. International Journal of Serious Games I, 11(2). https://doi.org/10.17083/ijsg
• Tyson, J. (2023). Shortcomings of ChatGPT. Journal of Chemical Education, 100(8), 3098-3101.
• Wang, X., Gong, Z., Wang, G., Jia, J., Xu, Y., Zhao, J., Fan, Q., Wu, S., Hu, W., & Li, X. (2023). ChatGPT Performs on the Chinese National Medical Licensing Examination. Journal of Medical Systems, 47(1). https://doi.org/10.1007/s10916-023-01961-0
• Xu, X. S., Wang, X. B., Zhang, Y. F., & Zheng, R. (2024). Applying ChatGPT to tackle the side effects of personal learning environments from learner and learning perspective: An interview of experts in higher education. PLoS ONE, 19(1 January). https://doi.org/10.1371/journal.pone.0295646
• Vargas-Murillo, A. R., de la Asuncion, I. N. M., & de Jesús Guevara-Soto, F. (2023). Challenges and opportunities of AI-assisted learning: A systematic literature review on the impact of ChatGPT usage in higher education. International Journal of Learning, Teaching and Educational Research, 22(7), 122-135