Harnessing AI-based learning media in education: A meta-analysis of its effects on student achievement

Year 2025, Volume: 12 Issue: 1, 222 - 242

Risky Setiawan , Umi Farisiyah , Muhammad Zainal , Widiyawanti Widiyawanti

https://doi.org/10.17275/per.25.12.12.1

Abstract

From the most straightforward kind of technology—audiovisual learning—to the application of artificial intelligence in education, technology has been used in education for over 20 years. Despite the growing popularity of AI-based learning media technology, there is still a dearth of reliable empirical data about its effects on student achievements. This meta-analysis aims to investigate the impact of intervention time and combine findings from several studies to paint a more comprehensive picture of the usefulness of AI media in education. In this study, a meta-analysis design is employed in quantitative research. The Publish or Perish tool gathered secondary data from published papers using the Scopus database and Google Scholar—data analysis for group contrast meta-analysis data using the R software. The study's findings demonstrate how using AI-based learning resources greatly impacts students' academic performance. P value total effect size and three moderator variables (continent, gained achievement, and intervention duration < 0.05) show that the aggregate value of the summary effect in AI-based learning media, which integrates technology products with software, web programs, augmented reality, and gamification in increasing student achievement from elementary school to tertiary level from 2019 to 2024, is still providing significant influence. Thus, artificial intelligence (AI) should be used more extensively in preparing learning media to maximize students' academic and non-academic successes.

Keywords

AI-based learning media, Artificial Intelligence, Achievement, Meta-analysis, learning media

References

• Abass, F., & Abas, N. (2019). The role of advance technologies in motivated learning : Case study of Saudi learners in universities. International Conference on Research in Education.
• Adcock, P. K. (2008). Evolution of teaching and learning through technology Evolution of Teaching and. The Delta Kappa Gamma Bulletin, 74(4), 37. Retrieved from https://digitalcommons.unomaha.edu/cgi/viewcontent.cgi?article=1057&context=tedfacpub
• Alomari, M. A. (2020). The effect of the use of an educational software based on the strategy of artificial intelligence on students’ achievement and their attitudes towards it. Management Science Letters, 10(13), 2951–2960. https://doi.org/10.5267/j.msl.2020.5.030
• Angwaomaodoko, E. A. (2023). An appraisal on the role of technology in modern education, opportunities and challenges. Path of Science, 9(12), 3019–3028. https://doi.org/10.22178/pos.99-4
• Bhargavi, S., & Guruprasad, N. (2019). Impact of artificial intelligence in the field of Education. Proceedings of the Second International Conference on Emerging Trends in Science & Technologies For Engineering Systems (ICETSE-2019), May, 35–39.
• Bhatt, C., Singh, S., Chauhan, R., Singh, T., & Uniyal, A. (2023). Artificial intelligence in current education: Roles, applications & challenges. The 3rd International Conference on Pervasive Computing and Social Networking (ICPCSN), 241–244.
• Borenstein, M. (2019). Common mistakes in meta-analysis and how to avoid them. Englewood, NJ: Biostat, Inc.
• Borenstein, M., Hedges, L. V, Higgins, J. P. T., & Rothstein, H. R. (2021). Introduction to meta-analysis. Oxford, UK: John Wiley & Sons.
• Candra, & Retnawati, H. (2020). A meta-analysis of constructivism learning implementation towards the learning outcomes on civic education lesson. International Journal of Instruction, 13(2), 835–846. https://doi.org/10.29333/iji.2020.13256a
• Chiu, T. K. F., Xia, Q., Zhou, X., Chai, C. S., & Cheng, M. (2023). Systematic literature review on opportunities, challenges, and future research recommendations of artificial intelligence in education. Computers and Education: Artificial Intelligence, 4, 100118.
• Cooper, H., Hedges, L. V., Valentine, J. C. (2009). The handbook of research synthesis and meta-analysis. New York: Russel Sage Foundation.
• Dai, Y., Lin, Z., Liu, A., & Wang, W. (2024). An embodied, analogical and disruptive approach of AI pedagogy in upper elementary education: An experimental study. British Journal of Educational Technology, 55(1), 417-434. https://doi.org/10.1111/bjet.13371
• Das, A., Malaviya, S., & Singh, M. (2023). The Impact of AI-Driven Personalization on Learners’ Performance. International Journal of Computer Sciences and Engineering, 11(8), 15-22. Retrieved from https://www.researchgate.net/profile/Amit-Das-18/publication/373424876_The_Impact_of_AI-Driven_Personalization_on_Learners'_Performance/links/64eaeb130453074fbdb66c1f/The-Impact-of-AI-Driven-Personalization-on-Learners-Performance.pdf
• del Campo, J. M., Negro, V., & Núñez, M. (2012). The history of technology in education. A comparative study and forecast. Procedia-Social and Behavioral Sciences, 69, 1086–1092. https://doi.org/10.1016/j.sbspro.2012.12.036
• Dhaya, R., Kanthavel, R., & Venusamy, K. (2022). AI-based learning model management framework for private cloud computing. Journal of Internet Technology, 23(7), 1633–1642. https://doi.org/10.53106/160792642022122307017
• Duval, S., & Tweedie, R. (2000). Trim and Fill: A Simple Funnel-Plot-Based Method. Biometrics, 56(2), 455–463. https://doi.org/10.1111/j.0006-341X.2000.00455.x
• Etemadfar, P., Soozandehfar, S. M. A., & Namaziandost, E. (2020). An account of EFL learners’ listening comprehension and critical thinking in the flipped classroom model. Cogent Education, 7(1). https://doi.org/10.1080/2331186X.2020.1835150
• García-Martínez, I. (2023). Analysing the Impact of Artificial Intelligence and Computational Sciences on Student Performance: Systematic Review and Meta-analysis. Journal of New Approaches in Educational Research, 12(1), 171–197. https://doi.org/10.7821/naer.2023.1.1240
• Hansen, C., Steinmetz, H., & Block, J. (2022). How to conduct a meta-analysis in eight steps: a practical guide. Management Review Quarterly, 72(1), 1–19. https://doi.org/10.1007/s11301-021-00247-4
• Harzing, A. W. (2007). Publish or Perish.
• Hassan, G. (2023). Technology and the transformation of educational practices: A future perspective. International Journal of Economic, Business, Accounting, Agriculture Management and Sharia Administration, 3(1), 1596–1603. | https://radjapublika.com/index.php/IJEBAS
• Higgins, J. P. T., & Green, S. (2011). Cochrane handbook for systematic reviews of interventions. London, UK: The Cochrane Collaboration.
• Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. Education and Debate, 327(7414), 557–560. https://doi.org/10.1136/bmj.327.7414.557
• Huang, A. Y. Q., Lu, O. H. T., & Yang, S. J. H. (2023). Effects of artificial Intelligence–Enabled personalized recommendations on learners’ learning engagement, motivation, and outcomes in a flipped classroom. Computers &Education, 194, 104684. https://doi.org/10.1016/j.compedu.2022.104684
• Huddar, R., & Kharade, K. (2023). Designing of AI-based teaching-learning model for revitalizing education. International Conference on the Future Global Business and Technology, Kolhapur, India. Retrieved from https://www.researchgate.net/profile/Kabir-Kharade/publication/370561052_Designing_of_AI-Based_Teaching-Learning_Model_for_Revitalizing_Education/links/6455fade809a53502150a740/Designing-of-AI-Based-Teaching-Learning-Model-for-Revitalizing-Education.pdf
• Hooda, M., Rana, C., Dahiya, O., Rizwan, A., & Hossain, M. S. (2022). Artificial intelligence for assessment and feedback to enhance student success in higher education. Mathematical Problems in Engineering, 2022(1), 5215722.
• Hwang, S. (2022). Examining the Effects of Artificial Intelligence on Elementary Students’ Mathematics Achievement: A Meta-Analysis. Sustainability (Switzerland), 14(20). https://doi.org/10.3390/su142013185
• Junaidi, J. (2020). Artificial intelligence in EFL context: Rising students’ speaking performance with Lyra virtual assistance. International Journal of Advanced Science and Technology Rehabilitation, 29(5), 6735-6741.
• Kaledio, P., Robert, A., & Frank, L. (2024). The Impact of Artificial Intelligence on Students’ Learning Experience. Available at SSRN 4716747.
• Kalyani, L. K. (2024). The role of technology in education: Enhancing learning outcomes and 21st century skills. International Journal of Scientific Research in Modern Science and Technology, 3(4), 5–10. Retrieved from https://ijsrmst.com/index.php/ijsrmst/article/view/199
• Kanvaria, V. K., & Suraj, M. T. (2024). The role of AI in Mathemathics education: Assessing the effects of an auto draw webtool on middle level achievement. The Online Journal of Distance Education and e-Learning, 12(1), 49. Retrieved from https://tojqih.net/cgi-sys
• Kiong, J. F. (2022). The Impact of Technology on Education: A Case Study of Schools. Journal of Education Review Provision, 2(2), 43–47. https://doi.org/10.55885/jerp.v2i2.153
• Li, K. (2023). Determinants of college students’ actual use of AI-based systems: An extension of the technology acceptance model. Sustainability, 15(6), 5221. https://doi.org/10.3390/su15065221
• Liu, C. C., Liao, M. G., Chang, C. H., & Lin, H. M. (2022). An analysis of children’interaction with an AI chatbot and its impact on their interest in reading. Computers &Education, 189, 104576. https://doi.org/10.1016/j.compedu.2022.104576
• Lokare, V. T., & Jadhav, P. M. (2024). An AI-based learning style prediction model for personalized and effective learning. Thinking Skills and Creativity, 51. 101421. https://doi.org/10.1016/j.tsc.2023.101421
• Mathur, M. B., & Vanderweele, T. J. (2020). Sensitivity analysis for publication bias in meta-analyses. Applied Statistics, 69(5), 1091–1119. https://doi.org/10.1111/rssc.12440
• Mengist, W., & Soromessa, T. (2020). Method for conducting systematic literature review and meta-analysis for environmental science research. MethodsX, 7, 100777. https://doi.org/10.1016/j.mex.2019.100777
• Nethra R MBA, N. (2019). Impact of technology on education. Journal of Emerging Technologies and Innovative Research, 6(7), 166–169.
• Raja, R., & Nagasubramani, P. C. (2018). Impact of modern technology in education. Journal of Applied and Advanced Research, 3(1), 33–35. https://dx.doi.org/10.21839/jaar.2018.v3S1.165
• Retnawati, H., Apino, E., Kartianom, & Djidu, H.; Anazifa, R. D. (2018). Introduction to Meta Analysis (Pengantar Analisis Meta). Yogyakarta, Indonesia: Parama Publishing.
• Rosenthal, R. (1979). The " File Drawer Problem " and Tolerance for Null Results. Psychological Bulletin, 86(3), 638–641. https://doi.org/10.1037/0033-2909.86.3.638
• RStudio_Team. (2020). RStudio: Integrated Development for R. RStudio, PBC, Boston, MA.
• Samra, E. M. (2021). The effect of introducing infographic pattern on developing cognitive understanding by using AI technology for university students during the COVID-19 pandemic. Journal of Healthcare Engineering 2021(1). https://doi.org/10.1155/2021/7197224
• Sharaf, A., & Musawi, A. (2011). Redefining Technology Role in Education. Creative Education 2(2), 130–135. https://doi.org/10.4236/ce.2011.22018
• Simanungkalit, I., Wardi, W., Jaya, C. A., & Loretha, A. F. (2024). The effectiveness of AI-based video to increase digital literacy in junior high school. Journal of Curriculum Indonesia, 7(1), 9-16. Retrieved from https://www.hipkinjateng.org/jurnal/index.php/jci/article/view/103
• Song, D. (2024). Artificial intelligence for human learning : A review of machine learning techniques used in education research and a suggestion of a learning design model. American Journal of Education and Learning, 9(1), 1–21. https://doi.org/10.55284/ajel.v9i1.1024
• Sterne, J. A. C., & Egger, M. (2001). Funnel plots for detecting bias in meta-analysis : Guidelines on choice of axis. Journal of Clinical Epidemiology, 54(10), 1046–1055. https://doi.org/10.1016/S0895-4356(01)00377-8
• Stogiannis, D., Siannis, F., & Androulakis, E. (2024). Heterogeneity in meta-analysis: a comprehensive overview. The International Journal of Biostatistics, 20(1), 169–199. https://doi.org/10.1515/ijb-2022-0070
• Tahir, M., Hassan, F. D., & Shagoo, M. R. (2024). Role of artificial intelligence in education : A conceptual review. World Journal of Advanced Research and Reviews, 22(01), 1469–1475. https://doi.org/10.30574/wjarr.2024.22.1.1217
• Timmers, K. (2018). Evolution of technology in the classroom. In Teaching in the fourth industrial revolution (pp. 106–123). Routledge.
• Topal, A. D., Eren, C. D., & Gecer, A. K. (2021). Chatbot application in a 5th grade science course. Education and Information Technologies, 26(5). 6241-6265 https://doi.org/10.1007/s10639-021-10627-8
• Velayutham, G., Raja, A., & Chalke, Daniel F. J. (2022). Impact of new technologies in education. Journal of Pharmaceutical Negative Results, 13(9), 1393–1396. https://doi.org/10.47750/pnr.2022.13.S09.167
• Vicente-Sáez, R., & Martínez-Fuentes, C. (2018). Open Science now: A systematic literature review for an integrated definition. Journal of Business Research, 88. 428-436. https://doi.org/10.1016/j.jbusres.2017.12.043
• Wei, L. (2023). Artificial intelligence in language instruction: Impact on English learning achievement, L2 motivation, and self-regulated learning. Frontiers in Psychology, 14. 1261955. https://doi.org/10.3389/fpsyg.2023.1261955
• Wu, S., & Wang, F. (2021). Artificial intelligence‐based simulation research on the flipped classroom mode of listening and speaking teaching for English majors. Mobile Information Systems, 2021(1). https://doi.org/10.1155/2021/4344244
• Younes, S. S. (2021). Examining the effectiveness of using adaptive AI-enabled e-learning during the Pandemic of COVID-19. Journal of Healthcare Engineering. 2021(1). https://doi.org/10.1155/2021/3928326
• Zheng, L., Niu, J., Zhong, L., & Gyasi, J. F. (2023). The effectiveness of artificial intelligence on learning achievement and learning perception: A meta-analysis. Interactive Learning Environments, 31(9), 5650–5664.