İşletmelerde E-öğrenme Faaliyetlerinin Etkinliğinin Ölçüm Modelleri ve Karşılaştırılması: Bir Literatür İncelemesi

Öz

Günümüzde sürekli gelişen teknoloji ile beraber yaşamın her alanında yaygınlaşan dijital uygulamalar, işletmelerin eğitim faaliyetleri bünyesinde e-öğrenme faaliyetlerinin hızla yaygınlaşmasına neden olmuştur. E-öğrenme faaliyetleri, eğitim programlarının elektronik ortamda ya da platformlar aracılığıyla sunulmasının yanı sıra öğrenme ve gelişim sistemlerinin İK uygulamaları ve diğer yönetim sistemleriyle dijital entegrasyonunu, dijital öğrenme yönetim sistemleri ve platformlarını (Learning Management System) ve sonrasında geliştirilen öğrenme deneyim sistemlerini (Learning Experiences Management) kapsamaktadır. E-öğrenme faaliyetlerinin yaygınlaşması, e-öğrenme faaliyetlerinin etkinliğinin ölçülmesini hem akademisyenler hem de uygulamacılar açsından önemli bir konu haline getirmiştir. Eğitim etkinliğinin ölçülmesine yönelik geçmişte geliştirilen modellerin, yöntem ve tekniklerin yeni e-öğrenme faaliyetleri çerçevesinde nasıl kullanılabileceği ya da geliştirileceği sorusu bu çalışmanın temel sorunsalını oluşturmaktadır. Bu çalışma, belirtilen temel probleme yönelik bilgi birikimine katkı sağlamayı amaçlamakta ve “E-öğrenme faaliyetlerinin etkinliğinin etkili bir şekilde nasıl ölçülebileceği” sorusuna yanıt aramaktadır. Bu bağlamda Prisma Protokolü kullanılarak e-öğrenme faaliyetlerinin etkinliğinin ölçümünde (E-ÖEÖ) kullanılan modellere ilişkin sistematik literatür incelemesi yapılmıştır. Literatürde yer alan çalışmalar, e-öğrenme faaliyetlerinin etkinliğinin ölçümünde kullanılan modellerin yapısı ve odaklanılan etkinlik alanları ve kuruluş kapsamında değerlendirilmiştir. Sistematik literatür incelemesinin bulguları analiz edilerek bu çalışmada sunulmuştur. E-öğrenme faaliyetlerinin etkinliğinin, e-öğrenme portalı, e-öğrenme eğitim sistemi ve e-öğrenme programı bağlamında üç unsur doğrultusunda ölçümlendiği ve e-öğrenme portalının etkinliğinin ölçümüne önem verildiği ortaya konmuştur. Aynı zamanda ilgili literatürde E-ÖEÖ’de eğitim kuruluşlarının ön planda olduğu ancak işletmelere yönelik çalışmaların kısıtlı olduğu anlaşılmıştır.

Anahtar Kelimeler

• Elektronik Öğrenme
• E-öğrenme
• E-öğrenme Etkinliği

MEASUREMENT MODELS OF THE EFFECTIVENESS OF E-LEARNING ACTIVITIES IN BUSINESSES AND THESE COMPARISON: A LITERATURE REVIEW

Öz

In today's world, digital applications that are becoming widespread in every area of life along with the constantly developing technology have caused e-learning activities to become widespread rapidly within the businesses' educational activities. E-learning activities are not only the delivery of educational programs electronically or through platforms but also include the digital integration of learning and development systems with HR applications and other management systems, and digital learning management systems and platforms (Learning Management System) and learning experience systems (Learning Experiences Management). The widespread adoption of e-learning activities has made the measurement of e-learning activities' effectiveness a crucial issue for both academics and practitioners. The fundamental issue of this study is how past models, methods, and techniques for tracking educational effectiveness can be applied to or inform the development of new e-learning activities. This study aims to contribute to the accumulate knowledge aimed at addressing the stated problem and seeks to provide answers to the question of “How the effectiveness of e-learning activities can be measured effectively.” In this context, a systematic literature review was conducted on the models used for measuring the effectiveness of e-learning activities using the PRISMA Protocol. Studies in the literature have been evaluated in terms of the structure of the models used for measuring the effectiveness of e-learning activities, the areas of effectiveness they focus on, and the organizational context. The findings of the systematic literature review have been analyzed and presented in this study. It has been demonstrated that the effectiveness of e-learning activities is measured through three elements: the e-learning portal, the e-learning training system, and the e-learning program, with a particular emphasis on measuring the effectiveness of the e-learning portal. Additionally, it has been found that the relevant literature prioritizes educational institutions in in measuring the effectiveness of e-learning applications, while studies focusing on businesses are limited.

Anahtar Kelimeler

• Electronic Learning
• E-learning
• E-learning Effectiveness

Kaynakça

1. Aali, M., Narenji Thani, F., Keramati, M. R. ve Garavand, A. (2020). A Model for Effectiveness of E-learning at University, Journal of Information Technology Management, 12(4), 121-140.
2. Aguti, B., Wills, G. B. ve Walters, R. J. (2014). An Evaluation of The Factors That Impact on The Effectiveness of Blended E-learning within Universities, In International conference on information society,117-121.
3. Al-Fraihat D., Joy M., Masa’deh R. ve Sinclair, J. (2020). Evaluating E-learning Systems Success: An Empirical Study. Computers in Human Behavior 102, 67–86.
4. Alkhalaf, S., Drew, S. ve Nguyen, A. (2012). Validation of the IS Impact Model for Measuring the Impact of E-learning Systems in KSA Universities: Student Perspective, International Journal of Advanced Computer Science and Applications, 3(5), 73-78.
5. Al-sabawy, A. Y., Cater-Steel, A. ve Soar, J. (2011). Measuring E-Learning System Success (Research in Progress). In Proceedings of the 15th Pacific Asia conference on information systems (PACIS 2011). University of Southern Queensland.
6. Al-sabawy, A. Y. (2013). Measuring E-Learning Systems Success, Doctoral Dissertation, University of Southern Queensland, Australia.
7. Alkinani, E. A. ve Alzahrani, A. I. (2021). Evaluating The Usability and Effectiveness of Madrasati Platforms As A Learning Management System in Saudi Arabia for Public Education, International Journal of Computer Science & Network Security, 21(6), 275-285.
8. Aparicio, M., Bacao, F. ve Oliveira, T. (2016). An E-Learning Theoretical Framework, Educational Technology & Society, 19 (1), 292–307.

9. Ariawan, P. W., Giri, K. W. Made ve Divayana, G. H. D. (2019). The Cipp-Saw Evaluation Model Design in Measurement The Effectıveness of E-learnıng at Health Universities in Bali, Journal of Theoretical and Applied Information Technology, 97(21).
10. Arsić, S., Ruso, J., Milošević, I., Rakić, A., Glogovac, M. ve Filipović, J. (2023). The Quality Indicators of E-learning: Business vs. Education. Croatian Journal of Education, 25(4), 1103-1137.
11. Attwell, G. (2006). Evaluating E-learning: A Guide to the Evaluation of E-learning. Evaluate Europe Handbook Series, 2(2), 1610-0875.
12. Baklizi, M. ve Alghyaline, S. (2011, May). Evaluation of E-learning websites in Jordan Universities based on ISO/IEC 9126 Standard. In 2011 IEEE 3rd International Conference on Communication Software and Networks, 71-73.
13. Bele, J. L. (2015). Business Intelligence in e-Learning. Proceedıngs of 2015 Internatıonal Conference On Interactıve Collaboratıve Learnıng (ICL), 20-24 September, Italy, 369-375.
14. Benaida, M. (2024). A Framework Model for Exploring Factors for Measuring E-Learning Systems and Its Relevant Outcomes via AHP, International Journal of Engineering Pedagogy (iJEP), 14 (5), 123-148.
15. Bing, W. ve Song, T. (2015). An Evaluation Model on Enterprise E-learning System. In 3rd International Conference on Mechatronics, Robotics and Automation, Atlantis Press, 971-974.
16. Bersin, J. (2019). Learning Experience Platform (LXP) Market Grows Up: Now too Big to Ignore, Erişim: 6.07.2024, https://joshbersin.com/2019/03/learning-experience-platform-lxp-market-grows-up-now-too-big-to-ignore/
17. Büyüközkan, Gülçin., Ruan, Da. ve Feyzioglu, Orhan (2007). Evaluating E-learning Web Site Quality in a Fuzzy Environment, International Journal of Intelligent Systems, 22, 567-586.
18. Clark, R. C. ve Mayer, R. E. (2008). E-learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning, San Francisco: Pfeiffer, Second Edition.
19. Clark, R. C. ve Mayer, R. E. (2024). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning, New Jersey: John Wiley & Sons, Fifth Edition.
20. Chao, R. J. ve Chen, Y. H. (2009). Evaluation of the Criteria and Effectiveness of Distance E-learning with Consistent Fuzzy Preference Relations, Expert Systems with Applications, 36(7), 10657-10662.
21. Chopra, Gaurav, Madan, Pankaj ve Jaisingh, Piyush (2019). Effectiveness of E-learning Portal From Students' Perspective a Structural Equation Model (SEM) Approach, Interactive Technology and Smart Education,16(2), 94-116.
22. Choudhury, G. B. ve Sharma, V. (2019). Review and Comparison of Various Training Effectiveness Evaluation Models for R & D Organization Performance, PM World Journal, 3 (2), 1-13.
23. Corbeil J. ve Corbeil M. (2015). E-learning: Past, Present, and Future, In: Khan BH, Ally M (Eds) International Handbook of E-learning Volume 1 Theoretical Perspectives and Research, New York: Routledge, 51–64.
24. Davis, F. D. (1985). A Technology Acceptance Model for Empirically Testing New End-User Information Systems: Theory and Results. Doctoral Dissertation. Cambridge, MA: MIT Sloan School of Management, December 20.
25. DeLone, W. H ve McLean E.R. (2003). The DeLone and McLean Model of Information Success: A Ten-Year Update, Journal of Management Information Systems, 19(4), 9-30.
26. DeLone, W. H. ve McLean, E. R. (2016). Information systems success measurement, Foundations and Trends in Information Systems, 2(1), 1–116.
27. De Witt, C. ve Karolyi, H. (2021). Anforderungen an ein Next Generation LMS zur Unterstützung von Personalisierung aus bildungswissenschaftlicher Perspektive. eleed, Iss. 14. Erişim: 06.07.2024, https://www.eleed.de/archive/se2021/5284/index_html
28. Diana ve Suharjito. (2013). Evaluation of E-learning Application System Performance Based On The Users' Perceptions, In Proceedings of 2013 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE), 772-775.
29. Divayana, D. G. H., Ariawan, I. ve Giri, M. K. W. (2021). CIPP-SAW Application as an Evaluation Tool of E-Learning Effectiveness, International Journal of Modern Education & Computer Science, 13(6).
30. Drozdova, A. A. ve Guseva, A. I. (2017). Modern Technologies of E-learning and its Evaluation of Efficiency, Procedia-Social and Behavioral Sciences, 237, 1032-1038.
31. Drueke, B., Mainz, V., Lemos, M., Wirtz, M. A. ve Boecker, M. (2021). An Evaluation of Forced Distance Learning and Teaching Under Pandemic Conditions Using The Technology Acceptance Model, Frontiers in Psychology, 12, 701347.
32. Emilova, P. (2016). E-Learning in Business Organisations–New Concepts, Technologies and Models. Бизнес управление, 26(3), 21-44.
33. Engelbrecht, E. (2003). A Look at E-learning Models: Investigating Their Value for Developing an E-learning Strategy, Progressio, 25(2), 38–47.
34. Eroğlu, U. (2006). İşletmelerde Eğitim Faaliyetlerinin Etkinliğinin Ölçümüne İlişkin Bir Model Önerisi, Yönetim Bilimleri Dergisi, 4 (2), 111-132.
35. Eroğlu, U. (2021). İnsan Kaynaklarının Eğitimi ve Gelişimi. A. Ş. Örnek (Ed.), İnsan Yönetimi Strateji, Politika ve Uygulamalar Kapsamında Davranışsal Bir Yaklaşım içinde, (s. 317-364). Ankara: Nobel Akademi Yayıncılık.
36. Ersöz, B., Bülbül, H. İ. ve Sağıroğlu, Ş. (2023). Using LXP for Green Deal: A New Approach, 12th International Conference on Renewable Energy Research and Applications (ICRERA), September, Oshawa, 524-529.
37. Farid, S., Ahmad, R., Alam, M., Akbar, A. ve Chang, V. (2018). A Sustainable Quality Assessment Model for The Information Delivery In E-learning Systems, Information Discovery and Delivery, 46(1), 1-25.
38. Fırat, M. (2023). Integrating AI Applications into Learning Management Systems to Enhance E-learning. Instructional Technology and Lifelong Learning, 4 (1), 1-14.
39. Gülbahar, Yasemin (2009). E-öğrenme. Ankara: Pegem Akademi.
40. Gürsoy, M. (2022). TEGEP Kurumsal Eğitim Araştırması, İstanbul: Eğitim ve Gelişim Platformu Derneği.
41. Hamtini, M. Thair (2008). Evaluating E-learning Programs: An Adaptation of Kirkpatrick's Model to Accommodate E-learning Environments, Journal of Computer Science, 4 (8), 693-698.
42. Hadoussa, S. (2020). Evaluation of E-learning System on Higher Education institutions İn KSA: A Survey at Saudi Electronic University, International Journal of Technology Enhanced Learning, 12(2), 180-199.
43. Hassanzadeh, A., Kanaani, F. ve Elahi, S. (2012). A Model for Measuring E-learning Systems Success in Universities, Expert systems with Applications, 39(12), 10959-10966.
44. Henderson, A. J. (2003). The E-learning Question and Answer Book: A Survival Guide for Trainers and Business Managers, New York: AMACOM.
45. Horton, William (2001). Evaluating E-learning, American Society for Training & Development (ASTD), USA.
46. İleri, G. (2021). Şirket akademilerinin etkinliğinin ölçümü ve değerlendirilmesine yönelik bir araştırma, yüksek lisans tezi, Bahçeşehir Üniversitesi, Sosyal Bilimler Enstitüsü / İnsan Kaynakları Yönetimi Ana Bilim Dalı.
47. Jones, J. G. ve Knezek, G. A. (1995). Categorizing Distance Learning Systems: Discovering Successful Ingredients.
48. J. D. Tinsley ve T. J. van Weert. (Eds), Abstracts of the World Conference on Computers in Education VI içinde (s. 157-161). Birmingham, UK: International Federation of Information Processing.
49. Jundillah, M. L., Suseno, J. E. ve Surarso, B. (2019). Evaluation of E-learning Websites Using The Webqual Method and Importance Performance Analysis, In E3S Web of Conferences,125.
50. Keksela, O. S., Skvortsova, V. N., Sukhushina, E. V., Rudneva, E. L. ve Spichenko, T. A. (2016). E-learning as a Modern Resource of Education, In SHS Web of Conferences, Research Paradigms Transformation in Social Sciences, Rusya, 28, 1-4.
51. Kim, S. W. ve Lee, M. G. (2008). Validation of an Evaluation Model for Learning Management Systems, Journal of Computer Assisted Learning, 24(4), 284-294.
52. Kirkpatrick, D. (1996). Great Ideas Revisited, Training and Development, Jaunary.
53. Lanzilotti, Rosa., Ardito, Carmelo., Costabile, F. Maria ve De Angeli, Antonella (2006). eLSE Methodology: a Systematic Approach to the eLearning Systems Evaluation, Educational Technology & Society, 9 (4), 42-53.
54. Lee, B.-C., Yoon, J.-O. ve Lee, I. (2009). Learners’ Acceptance of E-learning in South Korea: Theories and Results. Computers & Education, 53(4), 1320–1329.
55. Lee, Yi-Hsuan., Hsieh, Yi-Chuan ve Ma, Chun-Yuan (2011). A Model of Organizational Employees’ E-learning Systems Acceptance”. Knowledge-Based Systems, 24, 355–366.
56. Loukis, E., Georgiou, S. ve Pazalo, K. (2007). A Value Flow Model for the Evaluation of an E-learning Service, European Conference on Information Systems ECIS, 369-382.
57. Martínez-Caro, E., Juan Gabriel Cegarra-Navarro ve Gabriel CepedaCarrión (2015). An Application of the Performance-Evaluation Model for E-learning Quality in Higher Education, Total Quality Management & Business Excellence, 26:5-6, 632-647.
58. Mahdavi Ardestani, S. F., Adibi, S., Golshan, A. ve Sadeghian, P. (2023). Factors Influencing The Effectiveness of E-learning in Healthcare: A Fuzzy ANP Study, In Healthcare,11(14), 2-15.
59. Masadeh, R. E., Almajali, D., Majali, T. E., Hanandeh, A. ve Al-Radaideh, A. (2022). Evaluating E-learning Systems Success in The New Normal, International Journal of Data and Network Science, 6(4), 1033-1042.
60. Mehregan, Mohammad Reza, Jamporazmey, Mona, Hosseinzadeh, Mahnaz, ve Mehrafrouz, Mohsen (2011). Proposing an Approach For Evaluating E-learning by Integrating Critical Success Factor and Fuzzy AHP, 2011 International Conference on Innovation, Management and Service IPEDR.
61. Miyazoe, T. (2008). LMS-Based EFL Blended Learning: Blackboard vs. Moodle. K. Bradford Watts, T. Muller ve M.
62. Swanson (Eds.), JALT2007 Conference Proceedings içinde (s. 745-754). Tokyo: JALT.
63. Muhammad, A. H., Siddique, A., Youssef, A. E., Saleem, K., Shahzad, B., Akram, A. ve Al-Thnian, A. B. S. (2020). A
64. Hierarchical Model to Evaluate The Quality of Web-Based E-learning Systems, Sustainability, 12(10), 4071.
65. Mohammed, H. J., Kasim, M. M. ve Shaharanee, I. N. (2018). Evaluation of E-learning Approaches Using AHP-TOPSIS Technique, Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 10(1-10), 7-10.
66. Mosakhani, M. ve Jamporazmey, M. (2010, September). Introduce Critical Success Factors (CSFs) of Elearning for Evaluating E-Learning Implementation Success. In 2010 International Conference on Educational and Information Technology.
67. Nagy, V. ve Duma, L. (2023). Measuring efficiency and effectiveness of knowledge transfer in e-learning. Heliyon, 9(7), 1-15.
68. Ngabiyanto; Nurkhin, A. ve Mukhibad, H. (2021). E-Learning Evaluation Using General Extended Technology Acceptance Model Approach at Schools in COVID-19 Pandemic, European Journal of Educational Research, 10(3), 1171-1180.
69. Ouajdouni, A., Chafik, K.,ve Boubker, O. (2021). Measuring E-learning Systems Success: Data From Students of Higher Education İnstitutions İn Morocco, Data in Brief, 35, 106807.
70. Özbay, Ö. (2015). Dünyada ve Türkiye’de Uzaktan Eğitimin Güncel Durumu. Uluslararası Eğitim Bilimleri Dergisi, (5), 376-394.
71. Özkan, Sevgi, Köseler, Refika ve Baykal, Nazife (2008). Evaluating Learning Management Systems: Hexagonal E-Learning Assessment Model (HELAM), European and Mediterranean Conference on Information Systems (EMCIS2008) May 25-26 2008.
72. Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., ... ve Moher, D. (2021). The PRISMA 2020 Statement: an Updated Guideline For Reporting Systematic Reviews, BMJ, 372(71), 1-9.
73. Pruengkarn, R., Praneetpolgrang, P. ve Srivihok, A. (2005). An Evaluation Model for E-learning Websites in Thailand University. In Fifth IEEE International Conference on Advanced Learning Technologies, Temmu, Taiwan, 161-162.
74. Pavlovic, I., Cukusic, M. ve Jadric, M. (2016). Evaluating the Effectiveness of e-Learning Projects in Croatian Companies, In Central European Conference on Information and Intelligent Systems, Faculty of Organization and Informatics Varazdin, 151-250.
75. Pour, M. J., Hosseinzadeh, M., Azar, M. B. ve Taheri, F. (2017). Developing a New Framework for Evaluating E-learning Systems: Integrating BSC and FAHP. Kybernetes, 46(8), 1303-1324.
76. Pulley, M. Lynn (1994). Navigating the Evaluation Rapids, Training and Development, September, 48(9), 19-24.
77. Ramayasa, I. P. (2015). Evaluation Model of Success and Acceptance of E-learning, Journal of Theoretical and Applied Information Technology, 82(3), 462.
78. Rismayani, I ve Soetikno, Y. J. W. (2020). Using WebQual 4.0 for Measuring Quality of E-learning Services During COVID-19 Pandemic, 8th International Conference on Cyber and IT Service Management (CITSM),1-7.
79. Rokhman, F., Mukhibad, H., Bagas Hapsoro, B. ve Nurkhin, A. (2022). E-learning Evaluation During The COVID-19 Pandemic Era Based on The Updated of Delone and Mclean Information Systems Success Model, Cogent Education, 9(1), 1-25.
80. Rosenberg, J., Mark (1987). Evaluating Training Programs for Decision Making. Leslie S. May, Carol A. Moore,
81. Stephen J. Zammıt (Edt.), Evaluating Business And Industry Training içinde (s. 158-159). Boston: Kluver Academic Publisher.
82. Sandjojo, N. ve Wahyuningrum, T. (2015). Measuring E-learning Systems Success: Implementing D&M is Success Model, In 2015 4th International Conference on Interactive Digital Media (ICIDM),1-6.
83. Sardjono, W., Rahmasari, A. ve Lusia, E. (2022). Evaluation Model of E-learning Systems Implementation by Using Factor Analysis and Regression Analysis Method, In 2022 International Conference on Information Management and Technology,73-78.
84. Salter, S. M., Karia, A., Sanfilippo, F. M. ve Clifford, R. M. (2014). Effectiveness of E-learning in Pharmacy Education, American Journal of Pharmaceutical Education, 78(4), 83.
85. Sheldon, L. (2020). The Multiplayer Classroom: Designing Coursework as a Game. Boca Raton: CRC Press.
86. Smart, W.J. (2009). Information System Success: Evaluation of a Carbon Accounting and Sequestration Sysystem. (Doctoral Dissertation), Southern Cross University, Australia.
87. Stufflebeam, L. Daniel (2003). The CIPP Model for Evaluation, Annual Conference of the Oregon Program Evaluators Network, Erişim: 21.04.2024 https://goeroendeso.wordpress.com/wp-content/uploads/2009/01/cipp-modeloregon10-031.pdf
88. Sun, P.C., Tsai, R.J., Finger, G., Chen, Y.Y. ve Yeh, D. (2008). What Drives a Successful E-learning? An Empirical Investigation of the Critical Factors Influencing Learner Satisfaction, Computers & Education, 50,1183-1202.
89. Serbănescu, L., ve Chircu, S. (2014). The efficiency of E-learning Activities in Training Mentor Teachers, In Proceedings of International Conference e-Learning-IADIS, Lisabona, Portugalia.
90. Thane, S. (2023). Knowledge of E-learning is a Must in 21st Century Digital Education Era, International Journal Advance of Social Science and Education (IJASSE), 1(1), 51-62.
91. Trentin, G (1997). Telematics and On-line Teacher Training: the POLARIS project, Journal of Computer Assisted Learning, 13, 261–270.
92. Tseng, M. L., Lin, R. J., ve Chen, H. P. (2011). Evaluating the Effectiveness of E‐learning System in Uncertainty, Industrial Management & Data Systems, 111(6), 869-889.
93. Venkatesh, V. ve Davis, F.D. (2000). “A theoretical extension of the technology acceptance model: Four longitudinal field studies,” Manage. Sci., vol. 46, no. 2, pp. 186–204
94. Wang, Yi-Shun., Wang, Hsiu-Yuan. ve Shee, Daniel Y (2007). Measuring E-learning Systems Success in an Organizational Context: Scale Development and Validation, Computers in Human Behavior, 23, 1792–1808.
95. Wang, C. S. ve Lin, S. L. (2019). How Instructors Evaluate an E-learning System? An Evaluation Model Combining Fuzzy AHP With Association Rule Mining. Journal of Internet Technology, 20(6), 1947-1959.
96. Weigert, D. ve Behrendt, F. (2022). Conceptual Framework of a Learning Experience Platform (LXP) to Strengthen AI Competence by Linking Simulation Technologies and AI, International Conference on Modelling and Applied Simulation (MAS), 19-21 September, Roma.
97. White, M. A. (1983). Synthesis of Research on Electronic Learning. Educational Leadership, 40(8), 13-15.
98. Wiepcke, C. (2011). 360-degree Evaluation of E-learning Measures: Taking Into Account Phases, Levels, Stakeholders and Methods. Ubiquitous Learning, 3(1), 57.
99. Zainudin, M., Utami, A. D., Widyaningrum, R. ve Amin, A. K. (2023). The Evaluation of Research-based Learning on ‘Moodle’Learning Management System Using CIPP Models, Pegem Journal of Education and Instruction, 13(4), 149-157.
100. Zareisaroukolaei, M., Shams, G. ve RezaeiZadeh, M. (2024). Effectiveness Evaluation Indicators of Organizational E-learning Courses, Computers in Human Behavior Reports, 15, 1-17.
101. Zhang, W. ve Cheng, Y. (2012). Quality Assurance in E-Learning: PDPP Evaluation Model and its Application. International Review of Research in Open and Distributed Learning, 13(3), 65–82.