Harnessing AI-based learning media in education: A meta-analysis of its effects on student achievement

Risky Setiawan; Umi Farisiyah; Muhammad Zainal; Widiyawanti Widiyawanti

doi:10.17275/per.25.12.12.1

EN

Harnessing AI-based learning media in education: A meta-analysis of its effects on student achievement

Abstract

From the most straightforward kind of technology—audiovisual learning—to the application of artificial intelligence in education, technology has been used in education for over 20 years. Despite the growing popularity of AI-based learning media technology, there is still a dearth of reliable empirical data about its effects on student achievements. This meta-analysis aims to investigate the impact of intervention time and combine findings from several studies to paint a more comprehensive picture of the usefulness of AI media in education. In this study, a meta-analysis design is employed in quantitative research. The Publish or Perish tool gathered secondary data from published papers using the Scopus database and Google Scholar—data analysis for group contrast meta-analysis data using the R software. The study's findings demonstrate how using AI-based learning resources greatly impacts students' academic performance. P value total effect size and three moderator variables (continent, gained achievement, and intervention duration < 0.05) show that the aggregate value of the summary effect in AI-based learning media, which integrates technology products with software, web programs, augmented reality, and gamification in increasing student achievement from elementary school to tertiary level from 2019 to 2024, is still providing significant influence. Thus, artificial intelligence (AI) should be used more extensively in preparing learning media to maximize students' academic and non-academic successes.

Keywords

AI-based learning media, Artificial Intelligence, Achievement, Meta-analysis, learning media

References

Abass, F., & Abas, N. (2019). The role of advance technologies in motivated learning : Case study of Saudi learners in universities. International Conference on Research in Education.
Adcock, P. K. (2008). Evolution of teaching and learning through technology Evolution of Teaching and. The Delta Kappa Gamma Bulletin, 74(4), 37. Retrieved from https://digitalcommons.unomaha.edu/cgi/viewcontent.cgi?article=1057&context=tedfacpub
Alomari, M. A. (2020). The effect of the use of an educational software based on the strategy of artificial intelligence on students’ achievement and their attitudes towards it. Management Science Letters, 10(13), 2951–2960. https://doi.org/10.5267/j.msl.2020.5.030
Angwaomaodoko, E. A. (2023). An appraisal on the role of technology in modern education, opportunities and challenges. Path of Science, 9(12), 3019–3028. https://doi.org/10.22178/pos.99-4
Bhargavi, S., & Guruprasad, N. (2019). Impact of artificial intelligence in the field of Education. Proceedings of the Second International Conference on Emerging Trends in Science & Technologies For Engineering Systems (ICETSE-2019), May, 35–39.
Bhatt, C., Singh, S., Chauhan, R., Singh, T., & Uniyal, A. (2023). Artificial intelligence in current education: Roles, applications & challenges. The 3rd International Conference on Pervasive Computing and Social Networking (ICPCSN), 241–244.
Borenstein, M. (2019). Common mistakes in meta-analysis and how to avoid them. Englewood, NJ: Biostat, Inc.
Borenstein, M., Hedges, L. V, Higgins, J. P. T., & Rothstein, H. R. (2021). Introduction to meta-analysis. Oxford, UK: John Wiley & Sons.
Candra, & Retnawati, H. (2020). A meta-analysis of constructivism learning implementation towards the learning outcomes on civic education lesson. International Journal of Instruction, 13(2), 835–846. https://doi.org/10.29333/iji.2020.13256a
Chiu, T. K. F., Xia, Q., Zhou, X., Chai, C. S., & Cheng, M. (2023). Systematic literature review on opportunities, challenges, and future research recommendations of artificial intelligence in education. Computers and Education: Artificial Intelligence, 4, 100118.

Cooper, H., Hedges, L. V., Valentine, J. C. (2009). The handbook of research synthesis and meta-analysis. New York: Russel Sage Foundation.
Dai, Y., Lin, Z., Liu, A., & Wang, W. (2024). An embodied, analogical and disruptive approach of AI pedagogy in upper elementary education: An experimental study. British Journal of Educational Technology, 55(1), 417-434. https://doi.org/10.1111/bjet.13371
Das, A., Malaviya, S., & Singh, M. (2023). The Impact of AI-Driven Personalization on Learners’ Performance. International Journal of Computer Sciences and Engineering, 11(8), 15-22. Retrieved from https://www.researchgate.net/profile/Amit-Das-18/publication/373424876_The_Impact_of_AI-Driven_Personalization_on_Learners'_Performance/links/64eaeb130453074fbdb66c1f/The-Impact-of-AI-Driven-Personalization-on-Learners-Performance.pdf
del Campo, J. M., Negro, V., & Núñez, M. (2012). The history of technology in education. A comparative study and forecast. Procedia-Social and Behavioral Sciences, 69, 1086–1092. https://doi.org/10.1016/j.sbspro.2012.12.036
Dhaya, R., Kanthavel, R., & Venusamy, K. (2022). AI-based learning model management framework for private cloud computing. Journal of Internet Technology, 23(7), 1633–1642. https://doi.org/10.53106/160792642022122307017
Duval, S., & Tweedie, R. (2000). Trim and Fill: A Simple Funnel-Plot-Based Method. Biometrics, 56(2), 455–463. https://doi.org/10.1111/j.0006-341X.2000.00455.x
Etemadfar, P., Soozandehfar, S. M. A., & Namaziandost, E. (2020). An account of EFL learners’ listening comprehension and critical thinking in the flipped classroom model. Cogent Education, 7(1). https://doi.org/10.1080/2331186X.2020.1835150
García-Martínez, I. (2023). Analysing the Impact of Artificial Intelligence and Computational Sciences on Student Performance: Systematic Review and Meta-analysis. Journal of New Approaches in Educational Research, 12(1), 171–197. https://doi.org/10.7821/naer.2023.1.1240
Hansen, C., Steinmetz, H., & Block, J. (2022). How to conduct a meta-analysis in eight steps: a practical guide. Management Review Quarterly, 72(1), 1–19. https://doi.org/10.1007/s11301-021-00247-4
Harzing, A. W. (2007). Publish or Perish.
Hassan, G. (2023). Technology and the transformation of educational practices: A future perspective. International Journal of Economic, Business, Accounting, Agriculture Management and Sharia Administration, 3(1), 1596–1603. | https://radjapublika.com/index.php/IJEBAS
Higgins, J. P. T., & Green, S. (2011). Cochrane handbook for systematic reviews of interventions. London, UK: The Cochrane Collaboration.
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. Education and Debate, 327(7414), 557–560. https://doi.org/10.1136/bmj.327.7414.557
Huang, A. Y. Q., Lu, O. H. T., & Yang, S. J. H. (2023). Effects of artificial Intelligence–Enabled personalized recommendations on learners’ learning engagement, motivation, and outcomes in a flipped classroom. Computers &Education, 194, 104684. https://doi.org/10.1016/j.compedu.2022.104684
Huddar, R., & Kharade, K. (2023). Designing of AI-based teaching-learning model for revitalizing education. International Conference on the Future Global Business and Technology, Kolhapur, India. Retrieved from https://www.researchgate.net/profile/Kabir-Kharade/publication/370561052_Designing_of_AI-Based_Teaching-Learning_Model_for_Revitalizing_Education/links/6455fade809a53502150a740/Designing-of-AI-Based-Teaching-Learning-Model-for-Revitalizing-Education.pdf
Hooda, M., Rana, C., Dahiya, O., Rizwan, A., & Hossain, M. S. (2022). Artificial intelligence for assessment and feedback to enhance student success in higher education. Mathematical Problems in Engineering, 2022(1), 5215722.
Hwang, S. (2022). Examining the Effects of Artificial Intelligence on Elementary Students’ Mathematics Achievement: A Meta-Analysis. Sustainability (Switzerland), 14(20). https://doi.org/10.3390/su142013185
Junaidi, J. (2020). Artificial intelligence in EFL context: Rising students’ speaking performance with Lyra virtual assistance. International Journal of Advanced Science and Technology Rehabilitation, 29(5), 6735-6741.
Kaledio, P., Robert, A., & Frank, L. (2024). The Impact of Artificial Intelligence on Students’ Learning Experience. Available at SSRN 4716747.
Kalyani, L. K. (2024). The role of technology in education: Enhancing learning outcomes and 21st century skills. International Journal of Scientific Research in Modern Science and Technology, 3(4), 5–10. Retrieved from https://ijsrmst.com/index.php/ijsrmst/article/view/199
Kanvaria, V. K., & Suraj, M. T. (2024). The role of AI in Mathemathics education: Assessing the effects of an auto draw webtool on middle level achievement. The Online Journal of Distance Education and e-Learning, 12(1), 49. Retrieved from https://tojqih.net/cgi-sys
Kiong, J. F. (2022). The Impact of Technology on Education: A Case Study of Schools. Journal of Education Review Provision, 2(2), 43–47. https://doi.org/10.55885/jerp.v2i2.153
Li, K. (2023). Determinants of college students’ actual use of AI-based systems: An extension of the technology acceptance model. Sustainability, 15(6), 5221. https://doi.org/10.3390/su15065221
Liu, C. C., Liao, M. G., Chang, C. H., & Lin, H. M. (2022). An analysis of children’interaction with an AI chatbot and its impact on their interest in reading. Computers &Education, 189, 104576. https://doi.org/10.1016/j.compedu.2022.104576
Lokare, V. T., & Jadhav, P. M. (2024). An AI-based learning style prediction model for personalized and effective learning. Thinking Skills and Creativity, 51. 101421. https://doi.org/10.1016/j.tsc.2023.101421
Mathur, M. B., & Vanderweele, T. J. (2020). Sensitivity analysis for publication bias in meta-analyses. Applied Statistics, 69(5), 1091–1119. https://doi.org/10.1111/rssc.12440
Mengist, W., & Soromessa, T. (2020). Method for conducting systematic literature review and meta-analysis for environmental science research. MethodsX, 7, 100777. https://doi.org/10.1016/j.mex.2019.100777
Nethra R MBA, N. (2019). Impact of technology on education. Journal of Emerging Technologies and Innovative Research, 6(7), 166–169.
Raja, R., & Nagasubramani, P. C. (2018). Impact of modern technology in education. Journal of Applied and Advanced Research, 3(1), 33–35. https://dx.doi.org/10.21839/jaar.2018.v3S1.165
Retnawati, H., Apino, E., Kartianom, & Djidu, H.; Anazifa, R. D. (2018). Introduction to Meta Analysis (Pengantar Analisis Meta). Yogyakarta, Indonesia: Parama Publishing.
Rosenthal, R. (1979). The " File Drawer Problem " and Tolerance for Null Results. Psychological Bulletin, 86(3), 638–641. https://doi.org/10.1037/0033-2909.86.3.638
RStudio_Team. (2020). RStudio: Integrated Development for R. RStudio, PBC, Boston, MA.
Samra, E. M. (2021). The effect of introducing infographic pattern on developing cognitive understanding by using AI technology for university students during the COVID-19 pandemic. Journal of Healthcare Engineering 2021(1). https://doi.org/10.1155/2021/7197224
Sharaf, A., & Musawi, A. (2011). Redefining Technology Role in Education. Creative Education 2(2), 130–135. https://doi.org/10.4236/ce.2011.22018
Simanungkalit, I., Wardi, W., Jaya, C. A., & Loretha, A. F. (2024). The effectiveness of AI-based video to increase digital literacy in junior high school. Journal of Curriculum Indonesia, 7(1), 9-16. Retrieved from https://www.hipkinjateng.org/jurnal/index.php/jci/article/view/103
Song, D. (2024). Artificial intelligence for human learning : A review of machine learning techniques used in education research and a suggestion of a learning design model. American Journal of Education and Learning, 9(1), 1–21. https://doi.org/10.55284/ajel.v9i1.1024
Sterne, J. A. C., & Egger, M. (2001). Funnel plots for detecting bias in meta-analysis : Guidelines on choice of axis. Journal of Clinical Epidemiology, 54(10), 1046–1055. https://doi.org/10.1016/S0895-4356(01)00377-8
Stogiannis, D., Siannis, F., & Androulakis, E. (2024). Heterogeneity in meta-analysis: a comprehensive overview. The International Journal of Biostatistics, 20(1), 169–199. https://doi.org/10.1515/ijb-2022-0070
Tahir, M., Hassan, F. D., & Shagoo, M. R. (2024). Role of artificial intelligence in education : A conceptual review. World Journal of Advanced Research and Reviews, 22(01), 1469–1475. https://doi.org/10.30574/wjarr.2024.22.1.1217
Timmers, K. (2018). Evolution of technology in the classroom. In Teaching in the fourth industrial revolution (pp. 106–123). Routledge.
Topal, A. D., Eren, C. D., & Gecer, A. K. (2021). Chatbot application in a 5th grade science course. Education and Information Technologies, 26(5). 6241-6265 https://doi.org/10.1007/s10639-021-10627-8
Velayutham, G., Raja, A., & Chalke, Daniel F. J. (2022). Impact of new technologies in education. Journal of Pharmaceutical Negative Results, 13(9), 1393–1396. https://doi.org/10.47750/pnr.2022.13.S09.167
Vicente-Sáez, R., & Martínez-Fuentes, C. (2018). Open Science now: A systematic literature review for an integrated definition. Journal of Business Research, 88. 428-436. https://doi.org/10.1016/j.jbusres.2017.12.043
Wei, L. (2023). Artificial intelligence in language instruction: Impact on English learning achievement, L2 motivation, and self-regulated learning. Frontiers in Psychology, 14. 1261955. https://doi.org/10.3389/fpsyg.2023.1261955
Wu, S., & Wang, F. (2021). Artificial intelligence‐based simulation research on the flipped classroom mode of listening and speaking teaching for English majors. Mobile Information Systems, 2021(1). https://doi.org/10.1155/2021/4344244
Younes, S. S. (2021). Examining the effectiveness of using adaptive AI-enabled e-learning during the Pandemic of COVID-19. Journal of Healthcare Engineering. 2021(1). https://doi.org/10.1155/2021/3928326
Zheng, L., Niu, J., Zhong, L., & Gyasi, J. F. (2023). The effectiveness of artificial intelligence on learning achievement and learning perception: A meta-analysis. Interactive Learning Environments, 31(9), 5650–5664.

Details

Primary Language

English

Subjects

Specialist Studies in Education (Other)

Journal Section

Systematic Reviews and Meta Analysis

Authors

Risky Setiawan ^*
0000-0002-4269-996X
Indonesia

Umi Farisiyah
0000-0003-1076-4816
Indonesia

Muhammad Zainal
0009-0007-7921-9724
Indonesia

Widiyawanti Widiyawanti
0009-0003-7499-3682
Indonesia

Early Pub Date

December 27, 2024

Publication Date

January 1, 2025

Submission Date

August 25, 2024

Acceptance Date

November 26, 2024

Published in Issue

Year 2025 Volume: 12 Number: 1

DOI

https://doi.org/10.17275/per.25.12.12.1

IZ

https://izlik.org/JA92MB86TW

APA

Setiawan, R., Farisiyah, U., Zainal, M., & Widiyawanti, W. (2025). Harnessing AI-based learning media in education: A meta-analysis of its effects on student achievement. Participatory Educational Research, 12(1), 222-242. https://doi.org/10.17275/per.25.12.12.1

AMA

1.Setiawan R, Farisiyah U, Zainal M, Widiyawanti W. Harnessing AI-based learning media in education: A meta-analysis of its effects on student achievement. PER. 2025;12(1):222-242. doi:10.17275/per.25.12.12.1

Chicago

Setiawan, Risky, Umi Farisiyah, Muhammad Zainal, and Widiyawanti Widiyawanti. 2025. “Harnessing AI-Based Learning Media in Education: A Meta-Analysis of Its Effects on Student Achievement”. Participatory Educational Research 12 (1): 222-42. https://doi.org/10.17275/per.25.12.12.1.

EndNote

Setiawan R, Farisiyah U, Zainal M, Widiyawanti W (January 1, 2025) Harnessing AI-based learning media in education: A meta-analysis of its effects on student achievement. Participatory Educational Research 12 1 222–242.

IEEE

[1]R. Setiawan, U. Farisiyah, M. Zainal, and W. Widiyawanti, “Harnessing AI-based learning media in education: A meta-analysis of its effects on student achievement”, PER, vol. 12, no. 1, pp. 222–242, Jan. 2025, doi: 10.17275/per.25.12.12.1.

ISNAD

Setiawan, Risky - Farisiyah, Umi - Zainal, Muhammad - Widiyawanti, Widiyawanti. “Harnessing AI-Based Learning Media in Education: A Meta-Analysis of Its Effects on Student Achievement”. Participatory Educational Research 12/1 (January 1, 2025): 222-242. https://doi.org/10.17275/per.25.12.12.1.

JAMA

1.Setiawan R, Farisiyah U, Zainal M, Widiyawanti W. Harnessing AI-based learning media in education: A meta-analysis of its effects on student achievement. PER. 2025;12:222–242.

MLA

Setiawan, Risky, et al. “Harnessing AI-Based Learning Media in Education: A Meta-Analysis of Its Effects on Student Achievement”. Participatory Educational Research, vol. 12, no. 1, Jan. 2025, pp. 222-4, doi:10.17275/per.25.12.12.1.

Vancouver

1.Risky Setiawan, Umi Farisiyah, Muhammad Zainal, Widiyawanti Widiyawanti. Harnessing AI-based learning media in education: A meta-analysis of its effects on student achievement. PER. 2025 Jan. 1;12(1):222-4. doi:10.17275/per.25.12.12.1