The Bibliometric Analysis of Kirkpatrick Four-Level Training Evaluation Model: 2020-2025 Period

Abstract

This bibliometric study highlights the significance of the Kirkpatrick Four-Level Training Model for evaluating training programs and its systematic approach to evaluation, which has attracted the attention of many researchers. The bibliometric analysis highlights trends in the literature, influential publications, and collaborative networks, contributing to a broader understanding of its impact and evolution in training evaluation. The analysis of the Kirkpatrick Four-Level Training Evaluation Model reveals a significant increase in research output, particularly in medical education, underscoring the growing importance of evaluating training effectiveness. By analyzing citation patterns and co-authorship networks, the study highlights the model’s influence and development within the training evaluation framework. This finding also highlights the model’s widespread application across institutions and countries, providing insight into robust interest in enhancing educational effectiveness, quality, and outcomes across multiple fields. Furthermore, these findings suggest that the Kirkpatrick Four-Level Training Evaluation Model is integrated across various academic contexts, demonstrating its versatility and applicability in training programs. In conclusion, this bibliometric analysis can guide future research directions and practical applications, thereby contributing to more effective implications of training programs that align with various organizational goals and objectives and enhance overall performance.

Keywords

• Bibliometric analysis
• Evaluation of training
• Bibliometric study
• Kirkpatrick model
• Four-level training evaluation model

Ethical Statement

CONFLICT OF INTEREST STATEMENT The authors declared no conflicts of interest in this study. RESEARCH AND PUBLICATION ETHICS STATEMENT The authors declared that they adhered to research and publication ethics in this study. AUTHOR LIABILITY STATEMENT The authors declare that the “Conceptual Framework, Analysis, and Visualization” parts of this work were done by Shafiyu JIBRIL, and the “Conceptual Framework and Method” parts of this work were done by Feyza CANDAN ÇİFTÇİ. GENERATIVE ARTIFICIAL INTELLIGENCE USAGE STATEMENT The author declares that GenAI tools (Napkin.ai) were used in this study for “Data visualization, Image generation” purposes.

References

1. Aguinis, H., & Kraiger, K. (2009). Benefits of training and development for individuals, teams, organizations, and society. Annual Review of Psychology, 60(1), 451–474. https://doi.org/10.1146/annurev.psych.60.110707.163505
2. Alliger, G. M., & Janak, E. A. (1989). Kirkpatrick's levels of training criteria: Thirty years later. Personnel Psychology, 42(2), 331–342. https://doi.org/10.1111/j.1744-6570.1989.tb00661.x
3. Haq, I. U., & Alqahtani, H. M. (2023). Publication trends and collaborative patterns in periodontics research from Saudi Arabia: A bibliometric analysis. Journal of International Society of Preventive and Community Dentistry, 13(6), 433–442. https://doi.org/10.4103/jispcd.JISPCD_126_23
4. AlRyalat, S. A. S., Malkawi, L. W., & Momani, S. M. (2019). Comparing bibliometric analysis using PubMed, Scopus, and Web of Science databases. JoVE (Journal of Visualized Experiments), (152), e58494. https://doi.org/10.3791/58494
5. Alsalamah, A., & Callinan, C. (2021). The Kirkpatrick model for training evaluation: bibliometric analysis after 60 years (1959–2020). Industrial and Commercial Training, 54 (1), 36–63. https://doi.org/10.1108/ICT-12-2020-0115
6. Alsharif, A. H., Salleh, N. Z. M., Baharun, R., Alsharif, Y. H., & Abuhassna, H. (2021). A Bibliometric Analysis of Neuromarketing: Current Status, Development, and Future Directions. International Journal of Academic Research in Accounting, Finance and Management Sciences, 11(3), 828–847. https://doi.org/10.6007/IJARAFMS/v11-i3/11673
7. Aria, M., & Cuccurullo, C. (2017). Bibliometrix: An R tool for comprehensive science mapping analysis. Journal of Informetrics, 11(4), 959–975. https://doi.org/10.1016/j.joi.2017.08.007
8. Brück, O. (2023). A bibliometric analysis of geographic disparities in the authorship of leading medical journals. Communications Medicine, 3(1), 178. https://doi.org/10.1038/s43856-023-00418-2

9. Campbell, K., Taylor, V., & Douglas, S. (2019). Effectiveness of online cancer education for nurses and allied health professionals: a systematic review using the Kirkpatrick evaluation framework. Journal of Cancer Education, 34, 339–356. https://doi.org/10.1007/s13187-017-1308-2
10. Dizon, A. G. (2023). Historical development of the CIPP curriculum evaluation model. History of Education, 52(1), 109–128. https://doi.org/10.1080/0046760X.2022.2098390
11. Donthu, N., Kumar, S., Mukherjee, D., Pandey, N., & Lim, W. M. (2021). How to conduct a bibliometric analysis: An overview and guidelines. Journal of Business Research, 133, 285–296. https://doi.org/10.1016/j.jbusres.2021.04.070
12. Dorri, S., Akbari, M., & Sedeh, M. D. (2016). Kirkpatrick’s evaluation model for in-service training on cardiopulmonary resuscitation. Iranian Journal of Nursing and Midwifery Research, 21(5), 493–497. https://doi.org/10.4103/1735-9066.193396
13. Ellegaard, O. (2018). The application of bibliometric analysis: disciplinary and user aspects. Scientometrics, 116(1), 181–202. https://doi.org/10.1007/s11192-018-2765-z
14. Gandomkar, R. (2018). Comparing Kirkpatrick’s original and new models with the CIPP evaluation model. Journal of Advances in Medical Education and Professionalism, 6(2), 94. https://doi.org/10.30476/jamp.2018.41014
15. Ghorbani, B.D. (2024). Bibliometrix: Science Mapping Analysis with R Biblioshiny Based on Web of Science in Applied Linguistics. In: Meihami, H., Esfandiari, R. (eds.), A Scientometric Research Perspective in Applied Linguistics. Springer, Cham. https://doi.org/10.1007/978-3-031-51726-6_8
16. Gifari, M. K., & Arifin, Z. (2024). Literature review: Evaluation of the Kurikulum Merdeka using the CIPP model. Inovasi Kurikulum, 21(3), 1747-1762. https://doi.org/10.17509/jik.v21i3.71095.
17. Glänzel, W. (2001). National characteristics in international scientific co-authorship relations. Scientometrics, 51, 69–115. https://doi.org/10.1023/A:1010512628145 Hicks, D., & Melkers, J. (2013). Bibliometrics as a tool for research evaluation. In Handbook on the theory and practice of program evaluation (pp. 323–349). Edward Elgar Publishing. https://doi.org/10.4337/9780857932402.00019
18. Hussain, A. (2024). Bibliometric Analysis of Global Policy: A Scholarly Journal of Durham University, UK. Access: An International Journal of Nepal Library Association, 3, 65–75. https://doi.org/10.3126/access.v3i1.69421
19. Huynh, Q. T., & Guyen, N. T. (2020). A probabilistic method for managing common risks in software project scheduling based on the program evaluation review technique. International Journal of Information Technology Project Management (IJITPM), 11(3), 77–94. https://doi.org/10.4018/IJITPM.2020070105
20. Jacobs, D., & Ingwersen, P. (2000). A bibliometric study of the publication patterns in the sciences of South African scholars 1981–96. Scientometrics, 47(1), 75–93. https://doi.org/10.1023/A:1005617825947
21. Jibril, S., & Bagceci, B. (2024). Comparative evaluation of the 12th-grade chemistry curriculum according to the CIPP evaluation model in “Türkiye and Nigeria”. J Res Edu, 2(1), 01–27. https://doi.org/10.33140/JRE
22. Khalil, G. M., & Crawford, C. A. G. (2015). A bibliometric analysis of US-based research on the behavioral risk factor surveillance system. American Journal of Preventive Medicine, 48(1), 50–57. https://doi.org/10.1016/j.amepre.2014.08.021
23. Kirkpatrick, D. (1996). Great ideas revisited. Techniques for Evaluating training programs. Revisiting Kirkpatrick’s four- Level model. Training and Development, 50, 54–59.
24. Kirkpatrick, D., & Kirkpatrick, J. (2006). Evaluating training programs: The four levels. Berrett-Koehler Publishers.
25. La Duke, P. (2017). How to evaluate training: Using the Kirkpatrick Model. Professional Safety, 62(8). Retrieved from https://www.proquest.com/scholarly-journals/how-evaluate-training-using-kirkpatrick-model/docview/1928996197/se-2
26. Lee, H., & Song, Y. (2021). Kirkpatrick model evaluation of accelerated second-degree nursing programs: a scoping review. Journal of Nursing Education, 60(5), 265–271. https://doi.org/10.3928/01484834-20210420-05
27. Mendez-Vasquez, R. I., & Suñen-Pinyol, E. (2015). Reproducibility, consensus, and reliability in bibliometrics. In ISSI.
28. Nawaz, F., Ahmad, W., & Khushnood, M. (2022). Kirkpatrick model and training effectiveness: a meta-analysis, 1982 to 2021. Business & Economic Review, 14(2), 35–56. https://doi.org/10.22547/BER/14.2.2
29. Nerur, S. P., Rasheed, A. A., & Natarajan, V. (2008). The intellectual structure of the strategic management field: An author co‐citation analysis. Strategic Management Journal, 29(3), 319–336. https://doi.org/10.1002/smj.659
30. Nouraey, P., Al-Badi, A., Riasati, M. J., & Maata, R. L. (2020). Educational program and curriculum evaluation models: a mini-systematic review of recent trends. Universal J Educ Res, 8(9), 4048–4055. https://doi.org/10.13189/ujer.2020.080930
31. Paul, S., Burman, R. R., & Singh, R. (2024). Training effectiveness evaluation: Advancing a Kirkpatrick model-based composite framework. Evaluation and Program Planning, 107, 102494. https://doi.org/10.1016/j.evalprogplan.2024.102494
32. Price, D. D. S. (1976). A general theory of bibliometric and other cumulative advantage processes. Journal of the American Society for Information Science, 27(5), 292-306. https://doi.org/10.1002/asi.4630270505
33. Pritchard, A., & Wittig, G. R. (1981). Bibliometrics: a bibliography and index, Vol. 1. ALLM Books.
34. Quinton, M. L., Tidmarsh, G., Parry, B. J., & Cumming, J. (2022). A Kirkpatrick model process evaluation of reactions and learning from my Strengths Training for life™. International Journal of Environmental Research and Public Health, 19(18), 11320. https://doi.org/10.3390/ijerph191811320
35. Reio Jr, T. G., Rocco, T. S., Smith, D. H., & Chang, E. (2017). A critique of Kirkpatrick's evaluation model. New Horizons in Adult Education and Human Resource Development, 29(2), 35–53. https://doi.org/10.1002/nha3.20178
36. Roemer, R. C., & Borchardt, R. (2015). Meaningful metrics: A 21st-century librarian's guide to bibliometrics, altmetrics, and research impact. Amer Library Assn.
37. Suryana, D., Husna, A., & Mahyuddin, N. (2023). CIPP Evaluation Model: Analysis of education implementation in PAUD based on government policy on implementation of learning during the COVID-19 pandemic. Jurnal Obsesi: Jurnal Pendidikan Anak Usia Dini, 7(4), 4386–4396. https://doi.org/10.31004/obsesi.v7i4.3722
38. Tungtawee, C., & Chano, J. (2024). Bibliometric analysis using VOSviewer with Publish or Perish of curriculum evaluation using the CIPP model. Indonesian Journal of Multidisciplinary Research, 4(1), 163-170. https://doi.org/10.17509/ijomr.v4i1.69011
39. Uşun, S. (2024). Eğitimde program değerlendirmesi: süreçler, yaklaşımlar ve modeller. Anı Yayıncılık.
40. van Eck, N.J., & Waltman, L. (2010). Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics, 84, 523–538. https://doi.org/10.1007/s11192-009-0146-3
41. Vemić, J. (2007). Employee training and development, and the learning organization. Facta universitatis-series: Economics and Organization, 4(2), 209–216.
42. Waltman, L., Van Eck, N. J., & Noyons, E. C. (2010). A unified approach to mapping and clustering of bibliometric networks. Journal of Informetrics, 4(4), 629–635. https://doi.org/10.1016/j.joi.2010.07.002
43. Zhao, W., Liu, Z., Wang, T., Yin, X., Sun, Y., Zhang, X., & Yang, H. (2023). Assessment of a training project on English as a medium of instruction (EMI) using the Kirkpatrick model. BMC Medical Education, 23(1), 271. https://doi.org/10.1186/s12909-023-04204-5