The Evaluation of Learning Management Systems by Using Fuzzy AHP, Fuzzy TOPSIS and an Integrated Method: A Case Study

Year 2019, Volume: 20 Issue: 2, 195 - 218, 01.04.2019

Yasemin Altun Turker Kasim Baynal , Turgay Turker

https://doi.org/10.17718/tojde.557864

Cited By: 9

Abstract

Distance Education has become an important educational model with the development of information and communication technologies in the world. It has also become increasingly used model in Turkey. In our country, the number of universities and educational organizations offering distance education programs increase day by day. Distance education is performed by some special software called Learning Management System (LMS). There are both commercial LMSs and several open-source LMSs that have cost advantages. It is difficult to select an appropriate LMS for many institutions to meet their standards. The decision making and selection process is crucial, because LMS functions are different from each other, and they all have various features. Because of this, in this study, it is targeted to help and facilitate the decision-making process of LMS for institutions. In this study, Fuzzy AHP (Analytic Hierarchy Process), Fuzzy TOPSIS (Technique for Order Preference by Similarity to Ideal Solution) and an integrated method were used for evaluating criteria and deciding alternatives. To help accomplish this aim, questionnaire forms was designed and used for data collection. These methods were compared with each other. Results were evaluated. According to results among different criteria, the most important criteria for selecting LMS alternatives was found Content Management and Development (96.3%) and between the alternatives Moodle was found the best alternative with the degree of 36.3%.

Keywords

Fuzzy AHP, fuzzy TOPSIS, learning management system, distance education

References

• Alshomrani, S. (2012). Evaluation of technical factors in distance learning with respect to open-source LMS. Asian Transactions on Computers, 2, 11-17. Aydin, C. C., & Birogul, S. (2008). E-Ogrenmede acik kaynak kodlu ogretim yonetim sistemleri ve Moodle [Open source learning management systems in e-learning and Moodle]. Bilisim Teknolojileri Dergisi, 1, 31-36. Bakhoum, E. S. & Brown, D. C. (2013). A hybrid approach using AHP–TOPSIS–entropy methods for sustainable ranking of structural materials. International Journal of Sustainable Engineering, 6(3), 212-224. Browne, T., Jenkins, M., & Walker, R. (2006). A longitudinal perspective regarding the use of VLEs by higher education institutions in the United Kingdom. Interactive Learning Environments, 14(2), 177-192. Cavus, N. (2010). The evaluation of Learning Management Systems using an artificial intelligence fuzzy logic algorithm. Advances in Engineering Software, 41, 248-254. Chang, D. Y. (1996). Applications of the extent analysis method on fuzzy AHP. European Journal of Operational Research, 95(3), 649-655. Chao, R. J., & Chen, Y. H. (2009). Evaluation of the criteria and effectiveness of distance e-learning with consistent fuzzy preference relations. Expert Systems with Applications, 36, 10657-10662. Chen, S., & Li, Y. (2009). A research of fuzzy AHP approach in evaluating distance education system alternatives. In Proceedings of the 1st International Workshop on Education Technology and Computer Science, 741-745. Chen, C. T. (2000). Extensions of the TOPSIS for Group Decision-Making under Fuzzy Environment. Fuzzy Sets and Systems, 114, 1-9. Chen, L. H., & Hung, C. C. (2010). An integrated fuzzy approach for the selection of outsourcing manufacturing partners in pharmaceutical R&D. International Journal of Production Research, 48(24), 7483-7506. Colace F., & De Santo, M. (2008). Evaluation models for e-learning platforms and the AHP approach: A case study. [Accessed 02 November 2015]. Available at: <http://tir.ipsitransactions.org/2011/January/Paper%2004.pdf>. Demirel, T., Musdal, H., Demirel, N. C., & Yucenur, G. N. (2009). Multi-criteria evaluation of land cover policies using fuzzy AHP and fuzzy ANP: The case of Turkey. Human and Ecological Risk Assessment, 15(4), 746-764. Chan, F. T. S., & Kumar, N. (2007). Global supplier development considering risk factors using fuzzy extended AHP-based approach. Omega, 35(4), 417-431. Goh, H. H., Kok, B. C., Yeo, H. T., Lee, S. W., & Zin, A. A. M. (2013). Combination of TOPSIS and AHP in load shedding scheme for large pulp mill electrical system. International Journal of Electrical Power & Energy Systems, 47, 198-204. Hassanzadeh, A., Kanaani, F., & Elahi, S. (2012). A model for measuring e-learning systems success in universities. Expert Systems with Applications, 39, 10959-10966. Hwang, C. L., & Yoon, K. (1981). Multiple attribute decision making: methods and applications: a state-of-the-art survey. New York: Springer-Verlag. Itmazi, J., Gea, M. M., Paderewski, P., & Gutierrez, F. L. (2005). A Comparison and Evaluation of Open-Source Learning Management Systems. In Proceedings of the IADIS International Conference Applied Computing, 80-86. Kahraman, C., Cebeci, U., & Ruan, D. (2004). Multi-attribute comparison of catering service companies using fuzzy AHP: The case of Turkey. International Journal of Production Economics, 87, 171-184. Kanuka, H., & Anderson, T. (2007). Ethical issues in qualitative e-learning research. International Journal of Qualitative Methods, 6(2), 20-39. Kaya, M. (2012). Distance education systems used in universities of Turkey and Northern Cyprus, Procedia-Social and Behavioral Sciences, 31, 676-680. Lonn, S., & Teasley, S. D. (2009). Saving time or innovating practice: investigating perceptions and uses of Learning Management Systems. Computers & Education, 53, 686-694. Majumdar, A., Sarkar, B., & Majumdar, P. K. (2005), Determination of quality value of cotton fibre using hybrid AHP-TOPSIS method of multi-criteria decision-making. The Journal of the Textile Institute, 96(5), 303-309. Mahmoodzadeh, S., Shahrabi, J., Pariazar, M., & Zaeri, M. S., (2007). Project selection by using fuzzy AHP and TOPSIS technique. International Journal of Social, Human Science and Engineering, 1(6), 302-307. Moore, M. G., & Kearsley, G. (2005). Distance education: a systems view, (2nd ed.). Canada: Wadsworth Publishing. Olcott, D. (2003). Accreditation in the digital era: preparing for the new distance education landscape. In S. Panda (Ed.), Planning and management in distance education (pp. 253-261). London: Kogan Page Limited. Ozan, O. (2008). Ogrenme yonetim sistemlerinin degerlendirilmesi [Evaluation of learning management systems]. XIII. Turkiye’de Internet Konferansi Bildirileri, [Accessed 12 September 2015]. Available at: <http://inet-tr.org.tr/inetconf13/kitap/ozan_inet08.pdf>. Razmi, J., Songhori, M. J., Khakbaz, M. H. (2009). An integrated fuzzy group decision making/fuzzy linear programming (FGDMLP) framework for supplier evaluation and order allocation. International Journal of Advanced Manufacturing Technology, 43, 590-607. Reis, Z. A., Baktir, H. O., Celik, B., Erkoc, M. F., Ozcakir, F. C., Ozdemir, S., & Sahin, K. (2012). Acik kaynak kodlu ogrenme yonetim sistemleri uzerine bir karsilastirma calismasi [A comparison study on open source learning management systems]. Egitim ve Ogretim Arastirmalari Dergisi, 1, 42-58. Saaty, T. L. (1977). A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology, 15(3), 234-281. Saaty, T. L. (1980). The Analytic Hierarchy Process. New York: McGraw-Hill. Shee, D. Y., & Wang Y. S. (2008). Multi-criteria evaluation of the web based e-learning system: A methodology based on learner satisfaction and its applications. Computers & Education, 50, 894-905. Shyjith, K., Ilangkumaran, M., & Kumanan, S. (2008). Multi-criteria decision-making approach to evaluate optimum maintenance strategy in textile industry. Journal of Quality in Maintenance Engineering, 14(4), 375-386. Stella, A., & Gnanam, A. (2004). Quality assurance in distance education: the challenges to be addressed. Higher Education, 47, 143-160. Sun, P. C., Tsai, R. J., Finger, G., Chen, Y. Y., & Yeh, D. (2008). What drives a successful e-learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers & Education, 50, 1183-1202. Tsaur, S. H., Chang, T. Y., & Yen, C. H. (2002). The Evaluation of Airline Service Quality by Fuzzy MCDM. Tourism Management, 23, 107-115. Van Laarhoven, P. J. M., & Pedrycz, W. (1983). A fuzzy extension of Saaty’s priority theory. Fuzzy Sets and System, 11(3), 199-227. Wang, W. T., & Wang, C. C. (2009). An empirical study of instructor adoption of web-based learning systems. Computers & Education, 53, 761-774. Yu, X., Guo, S., Guo, J., & Huang, X. (2011). Rank B2C e-commerce websites in e-alliance based on AHP and fuzzy TOPSIS. Expert Systems with Applications, 38, 3550-3557. Zadeh, L. A. (1965). Fuzzy Sets. Information and Control, 8, 338-353. Zheng, G., Zhu, N., Tian, Z., Chen, Y., & Sun, B. (2012). Application of a trapezoidal fuzzy AHP method for work safety evaluation and early warning rating of hot and humid environments. Safety Science, 50, 228–239.