A systematic literature review on multi-criteria decision making in higher education

Year 2023, Volume: 10 Issue: 1, 12 - 28, 20.03.2023

Fatma Şeyma Yüksel Ayşe Nilgün Kayadelen Figen Antmen

https://doi.org/10.21449/ijate.1104005

Cited By: 1

Abstract

The three components that form the basis of the educational process are the teacher, the learner, and the environment. These three components are affected by the developing and changing technology as a result of globalization considerably. Teaching and learning techniques should be updated and connected with these developments; new tools are therefore needed to make the necessary updates. Determination and application of the new tools include many decisions. Decision-makers can make more effective decisions using Multi-Criteria Decision-Making Techniques (MCDM), a complex decision-making tool that includes both quantitative and qualitative factors at present time. This study aimed to determine which MCDM methods are used in studies conducted in higher education, which is one of the most important development level indicators of countries, and to present a systematic literature review of MCDM method applications. The study was conducted in three stages: first, known electronics were searched until the end of 2021 using keywords; then, all studies were listed in a systematic taxonomy, and in the last stage, Thematic Network Analysis was used to evaluate the development of MCDM studies in the higher education area. It is determined that the Analytical Hierarchy Process (AHP) method is the most widely used method in higher education in MCDM applications. It was observed that the most common use of MCDM applications in higher education is e-learning as well. This study aims to be a guide for all researchers and practitioners who will study in both higher education and the MCDM areas.

Keywords

Multi-Criteria Decision-Making Techniques, Higher Education, Literature Review.

Supporting Institution

-

Project Number

-

Thanks

-

References

• Altunok, T., Özpeynirci, O., Kazancoglu, Y., & Yilmaz, R. (2010). Comparatives of multicriteria decisions making methods for postgraduate student selection. Egitim Arastirmalari-Eurasian Journal of Educational Research, 40, 1-15.
• Aly, M.F., Attia, H.A., & Mohammed, A.M. (2014). Prioritizing faculty of engineering education performance by using AHP-TOPSİS and balanced scorecard approach. International Journal of Engineering Science and Innovative Technology, 3(1), 11-23.
• Anggrainingsih, R., Umam, M.Z., & Setiadi, H. (2018). Determining e-learning success factor in higher education based on user perspective using Fuzzy AHP. MATEC Web Conferences. 154, 03011. https://doi.org/10.1051/matecconf/201815403011
• Aytaç, S., & Bayram, N. (2001). Üniversite gençliğinin iş ve eş seçimindeki etkin kriterlerinin analitik hiyerarşi süreci (AHP) ile analizi [Analysis of university youth's effective criteria for job and spouse selection by analytical hierarchy process (AHP)]. Öneri Dergisi, 4(16), 89-100. https://doi.org/10.14783/maruoneri.727643
• Badri, M.A., & Abdulla, M.H. (2004). Awards of excellence in institutions of higher education: an AHP approach. International Journal of Educational Management, 18(4), 224-242. https://doi.org/10.1108/09513540410538813
• Bali, O., & Gencer, C. (2005). AHP Bulanık AHP ve Bulanık Mantıkla Kara Harp Okuluna öğretim elemanı seçimi [Ahp, Fuzzy Ahp, and Fuzzy Logic Selection of Academic Staff to Turkish Military Academy]. Kara Harp Okulu Savunma Bilimleri Dergisi, 4, 24-43.
• Begicevic, N., & Divjak, B. (2006). Validation of theoretical model for decision making about e-learning implementation. Journal of Information and Organizational Sciences, 30(2), 171-184.
• Begicevic, N., Divjak, B., & Hunjak, T. (2007). Development of AHP based-model for decision making on e-learning implementation. Journal of Information and Organizational Sciences, 31, 13-24.
• Bo, L., Xuning P., & Bingquan B. (2009). Modeling of network education effectiveness evaluation in fuzzy analytic hierarchy process. International Conference on Networking and Digital Society, 2, 198–200. ICNDS’09, IEEE.
• Cakir, E., & Ozdemir, M. (2018). Altı sigma projelerinin bulanık copras yöntemiyle değerlendirilmesi: Bir üretim işletmesi örneği [Evaluation of six sigma projects with fuzzy copras method: An example of a manufacturing company]. Verimlilik Dergisi, 1, 7-39.
• Cebi, A., & Karal, H. (2017). An application of fuzzy analytic hierarchy process (FAHP) for evaluating students’ Projects. Educational Research and Reviews, 12(3), 120-132. https://doi.org/10.5897/ERR2016.3065
• Celikten, M., Gilic, F., Celikten., & Yildirim, A. (2019). Örgüt yönetiminde karar verme süreci: Bitmeyen bir tartışma [Decision making process in organization management: An endless discussion]. Mersin Üniversitesi Eğitim Fakültesi Dergisi, 15(2), 581-592.
• Chao, R.J., & Chen, Y.H. (2009). Evaluation of the criteria and effectiveness of distance e-learning with consistent fuzzy preference relations. Expert Systems with Applications, 36, 10657-10662. https://doi.org/10.1016/j.eswa.2009.02.047
• Chen, J.F., Hsieh, H.N., & Do, Q, H. (2015). Evaluating teaching performance based on fuzzy AHP and comprehensive evaluation approach. Applied Soft Computing, 28, 100-108. https://doi.org/10.1016/j.asoc.2014.11.050
• Chen, J.K., & Chen, I.S. (2010). Using a novel conjunctive MCDM approach based on DEMATEL, fuzzy ANP, and TOPSIS as an innovation support system for Taiwanese higher education. Expert Systems with Applications, 37, 1981 1990. https://doi:10.1016/j.eswa.2009.06.079
• Chen, Y., & Yang, M. (2010). Study and construct an online self-learning evaluation system model based on the AHP method. 2nd IEEE International Conference on Information and Financial Engineering (ICIFE), 54–58.
• Chena, J.F. Hsieha, H.N. & Do, Q.H. (2015). Evaluating teaching performance based on fuzzy AHP and comprehensive evaluation approach. Applied Soft Computing, 28, 100-108. https://doi.org/10.1016/j.asoc.2014.11.050
• Chi, H.K., Yeh, H.R., & Liao, L.-H., (2008). Applying fuzzy analytic hierarchy process to explore the university organizational performance in Taiwan. The Journal of Human Resource and Adult Learning, 4(1), 39–46.
• Choi, C.R., & Jeong, H.Y. (2019). Quality evaluation for multimedia contents of e-learning systems using the ANP approach on a high-speed network. Multimedia Tools and Applications, 78, 28853-28875. https://doi.org/10.1007/s11042-019-7351-8 Cicekli, U.G., & Karacizmeli, A. (2013). Bulanık analitik hiyerarşi süreci ile basarılı öğrenci seçimi: Ege Universitesi İktisadi ve İdari Bilimler Fakültesi örneği [Successful student selection with fuzzy analytic hierarchy process: The example of Ege University Faculty of Economics and Administrative Sciences]. Ege Strategic Research Journal, 4(1), 77-103. https://doi.org/10.18354/esam.81730
• Colace, F., Santo, M.D., & Pietrosanto, A. (2006). Evaluation models for e-learning platform: An AHP approach. Proceedings of Frontiers in Education. 1 6. https://doi.org/10.1109/FIE.2006.322312
• Drake, P.R. (1998). Using the analytic hierarchy process in engineering education. Int. J. Engng Ed., 14, 191-196.
• Dundar, S. (2008). Ders seçiminde analitik hiyerarşi prosesi uygulaması [Analytical Hierarchy Process application in course selection]. Suleyman Demirel Universitesi Iktisadi ve Idari Bilimler Fakultesi Dergisi 13(2), 217-226.
• Ertugrul, I., & Karakasoglu, N. (2007). Fuzzy TOPSIS method for academic member selection in engineering faculty. Innovations in E-learning, Instruction Technology, Assesment, and Engineering Education, 151-156.
• Feng, Y.J., Lu, H., & Bi, K. (2004). An AHP/DEA method for measurement of the efficiency of R&D management activities in universities. Intl. Trans. In Op. Res., 11, 181-191. https://doi.org/10.1111/j.1475-3995.2004.00450.x
• Garg, R., & Jain, D. (2017). Fuzzy multi-attribute decision-making evaluation of e-learning websites using FAHP, COPRAS, VIKOR, WDBA. Decision Science Letters, 6, 351-364. http://dx.doi.org/10.5267/j.dsl.2017.2.003
• Garg, R. (2017). Optimal selection of E-learning websites using multiattribute decision-making approaches. J. Multi Criteria Decision Analysis, 24, 187 196. https://doi.org/10.1002/mcda.1612
• Garg, R., Kumar, R., & Garg, S. (2019). MADM-Based parametric selection and ranking of e-learning websites using fuzzy COPRAS. IEEE Trans. Educ., 62(1), 11–18. https://doi.org/10.1109/TE.2018.2814611
• Ghosh, D., & Pal, A. (2017). Analysis of faculty teaching using a multi-criteria decision-making approach. International Journal of Engineering & Technology, 7, 74-78. https://doi.org/10.14419/ijet.v7i2.28.12884
• Giannoulis, C., & Ishizaka, A. (2010). A Web-based decision support system with ELECTRE III for a personalized ranking of British universities. Decision Support Systems, 48, 488-497. https://doi.org/10.1016/j.dss.2009.06.008
• Grandzol, J.R. (2005). Improving the faculty selection process in higher education: A case for the analytic hierarchy process. IR Applications, 6, 13.
• Gupta, R., Garg, T.K., Gupta, S., & Goel, A. (2010). Decision Analysis Approach for Quality in Technical Education. Global Journal of Human Social Science, 10(1), 14-18.
• Ho, W., Dey, P.K., & Higson, H.E. (2006). Multiple criteria decision-making techniques in higher education. International Journal of Educational Management, 20(5), 319-337. https://doi.org/10.1108/09513540610676403
• Holton, G. (1975). On the role of themata in scientific thought. Science, 188(4186), 328-334.
• Hsu, C.M., Yeh, Y.C., & Yen, J. (2009). Development of design criteria and evaluation scale for web-based learning platforms. International Journal of Industrial Ergonomics, 39, 90-95. https://doi.org/10.1016/j.ergon.2008.08.006
• Jain, D., Garg, R., & Bansal, A. (2016). Selection and ranking of E-learning websites using weighted distance-based approximation. Journal of Computer Education, 3(2), 193-207. https://doi.org/10.1007/s40692-016-0061-6
• Jesus. E.N., Rodrigues, J.C., & Antunes, C.H. (2007). A multicriteria decision support system for housing evaluation. Decision Support Systems, 43, 779 790. https://doi.org/10.1016/j.dss.2006.03.014
• Jie, C. (2010). Evaluation and modeling of online courses using fuzzy AHP. 2010 International Conference on Computer and Information Application, 232-235.
• Kabak, M., Ozceylan, E., Dagdeviren, M., & Genc T. (2017). Evaluation of distance education websites: a hybrid multicriteria approach. Turkish Journal of Electrical Engineering & Computer Sciences, 25, 2809–2819. https://doi.org/10.3906/elk-1512-271
• Kara, M., & Karaca, Y. (2010). Üniversite öğrencilerin işletme bölümünü seçmelerinde etkili olan öncelikli faktörlerin analitik hiyerarşi prosesi metodu ile analizi: Bozok Üniversitesi İktisadi ve İdari Bilimler Fakültesinde bir uygulama [Analysis of the priority factors that affect university students' choice of business administration with the analytical hierarchy process method: An application in Bozok University Faculty of Economics and Administrative Sciences]. Organizasyon ve Yonetim Bilimleri Dergisi, 2(1), 133-140.
• Kiarazm, A., & Koohkan, F. (2013). Performance evaluation in higher education institutes with the use of combinative model AHP and BSC. Journal of Basic and Applied Scientific Research, 3(4), 940-944
• Kousalya, P., Ravindranath, V., & Vizayakumar, K. (2006). Student absenteeism in engineering colleges: Evaluation of alternatives using AHP. Journal of Applied Mathematics and Decision Sciences 2006. 1–26. https://doi.org/10.1155/JAMDS/2006/58232
• Kurilovas, E., & Zilinskiene, I. (2013). New MCEQLS AHP method for evaluating the quality of learning scenarios. Technological and Economic Development of Economy, 19(1), 78-92. https://doi.org/10.3846/20294913.2012.762952
• Kurilovas, E., & Serikoviene, S. (2013). New MCEQLS TFN method for evaluating quality and reusability of learning objects. Technological and Economic Development of Economy, 19(4), 706-723. https://doi.org/10.3846/20294913.2013.837112
• Kwak N.K., & Lee C. (1998). A multicriteria decision-making approach to university resource allocations and information infrastructure planning. European Journal of Operational Research, 110(2), 234-242. https://doi.org/10.1016/S0377-2217 (97)00262-2
• Lee, S.H. (2010). Using fuzzy AHP to develop intellectual capital evaluation model for assessing their performance contribution in a university. Expert Systems with Applications, 37, 4941-4947. https://doi.org/10.1016/j.eswa.2009.12.020
• Lesmes, D., Castillo, M., & Zarama, R. (2009). Application of The Analytic Network Process (ANP) to Establish Weights in Order to Re-Accredit a Program of a University. Proceedings of the International Symposium on the Analytic Hierarchy Process, 29.
• Li, W., Gao, X., & Fu, G. (2012). Fuzzy comprehensive assessment of network environment and learning quality combined with the analytic hierarchy process. 2nd International Conference on Consumer Electrics, Communications and Networks, 2600-2603, IEEE.
• Liberatore, M.J., & Nydick, R.L. (1997). Group Decision Making in Higher Education Using the Analytic Hierarchy Process. Research in Higher Education, 38, 593–614. https://doi.org/10.1023/A:1024948630255
• Lin, H.F. (2010). An application of fuzzy AHP for evaluating course website quality. Computers & Education, 54, 877-888. https://doi.org/10.1016/j.compedu.2009.09.017
• Mehregan, M.R., Jamporazmey, M., Hosseinzadeh, M., & Mehrafrouz, M. (2011a). Proposing an approach for evaluating e-learning by integrating critical success factor and fuzzy AHP. International Conference on Innovation, Management and Service, Singapore.
• Mehregan, M.R., Jamporazmey, M., Hosseinzadeh, M., & Mehrafrouz, M. (2011b). Application of fuzzy analytic hierarchy process in ranking modern educational systems’ success criteria. International Journal of e-Education, 1(4), 299-304.
• Melon, M.G., Beltran, P.A., & Cruz, M.C.G. (2008). An AHP-based evaluation procedure for Innovative Educational Projects: A face-to-face vs. computer-mediated case study. Omega, 36, 754-765. https://doi.org/10.1016/j.omega.2006.01.005
• Mendoza, G.A., Prabhub, R. (2000). Multiple criteria decision-making approaches to assessing forest sustainability using criteria and indicators: A case study. Forest Ecology and Management, 131, 107-126.
• Mohammed, H.J., Kasim, M.M., & Shaharanee, I.N. (2018). Evaluating of e-learning approaches using AHP-TOPSIS technique, Journal of Telecommunication, Electronic and Computer Engineering, 10, 1-10.
• Mondal, K., & Pramanik, S. (2014). Neutrosophic Sets and Systems, 6, 28-34.
• Murakoshi H., Kawarasaki T., & Ochimizu K. (2001). Comparison using AHP Web-based learning with classroom learning, Proceedings of Symposium on Applications and the Internet Workshops, 67-73. https://doi.org/10.1109/SAINTW.2001.998212
• Mustaffa, W.S.W., Shokory, S.M., & Kamis, H. (2006). The Analytical Hierarchy Process: Multi-Criteria Decision Making for Promoting Academic Staff in Higher Education. The Journal of Global Business Management, 2(2).
• Nagpal, R., Mehrotra, D., Sharma, A., & Bhatia, P. (2013). ANFIS method for usability assessment of the website of an educational institute. World Applied Sciences Journal, 23(11), 1489–1498. https://doi.org/10.5829/ idosi.wasj.2013.23.11.790
• Nagpal, R., Mehrotra, D., Bhatia, P.K., & Sharma, A. (2015). FAHP approach to rank educational websites on usability. International Journal of Computing and Digital Systems, 4(4), 251–260. http://dx.doi.org/10.12785/IJCDS/040404
• Naveed, Q.N., Qureshi, M.R.N., Alsayed, A.O., Muhammad, A., Sanober, S. & Shah, A. (2017). Prioritizing barriers of E-learning for effective teaching-learning using fuzzy analytic hierarchy process (FAHP). 4th IEEE International Conference on Engineering Technologies and Applied Sciences (ICETAS), 1-8.
• Naveed, Q.N., Qureshi, M.R., Tairan, N., Mohammad, A., & Shaikh, A. (2020). Evaluating critical success factors in implementing e-learning system using multi-criteria decision-making. PLoS ONE, 15(5), https://doi.org/10.1371/journal.pone.0231465
• Nilashi, M., & Janahmadi, N. (2012). Assessing and prioritizing affecting factors in e-learning websites using the AHP method and fuzzy approach. Information and Knowledge Management, 2(1), 46-61.
• Nikoomaram, H., Mohammadi, M., Javad Taghipouria, M., & Taghipourian, Y. (2009). Training performance evaluation of administration sciences instructors by fuzzy MCDM approach. Contemporary Engineering Sciences, 2(12), 559–575.
• Omurbek, N., Karaatli, M., & Yetim, T. (2014). Analitik hiyerarsi surecine dayali TOPSIS ve VIKOR yöntemleri ile ADIM universitelerinin değerlendirilmesi [Evaluation of ADIM universities with TOPSIS and VIKOR methods based on analytical hierarchy process]. Selcuk Universitesi Sosyal Bilimler Dergisi, Dr.Mehmet YILDIZ special issues. 189-207.
• Opricovic, S., (1998). Multicriteria Optimization of Civil Engineering Systems [Doctoral dissertation, Faculty of Civil Engineering].
• Ozdemir, M.S., & Gasimov R.N. (2004). The analytic hierarchy process and multiobjective 0-1 faculty course assignment. European Journal of Operational Research, 157, 398-408. https://doi.org/10.1016/S0377-2217(03)00189-9
• Ozkul, A.E., Girginer, N., & Ozturk, Z.K. (2007). Multi-Criteria Evaluation of Distance Education Implementation Models using Analytic Hierarchy Process, Proceedings of the 21st Annual Conference Empowering Asia through Partnership in Open and Distance Learning, 87.
• Ozturk, Z.K. (2014). Using a multi-criteria decision making approach for open and distance learning system selection. Anadolu University Journal of Science and Technology– An Applied Sciences and Engineering, 15(1), 1-14.
• Paksoy, S. (2015). Ülke göstergelerinin vikor yöntemi ile değerlendirilmesi [Evaluation of Country Indicators by Vikor Method]. Ekonomik ve Sosyal Araştırmalar Dergisi, 11(2), 153-169.
• Perez Vergara, I.G., Arias Sa´nchez, J.A., Poveda-Bautista, R., & Diego-Mas J.A. (2020). Improving distributed decision making in inventory management: A combined ABC-AHP approach supported by teamwork. Complexity, 3–5, 1–13.
• Politis, Y., & Siskos, Y. (2004). Multicriteria methodology for the evaluation of a Greek engineering department. European Journal of Operational Research, 156, 223-240. https://doi.org/10.1016/S0377-2217(02)00902-5
• Ray, S. (2007). Selecting a doctoral dissertation supervisor: analytical hierarchy approach to the multiple criteria problem. International Journal of Doctoral Studies, 2, 23-32. https://doi.org/10.28945/55
• Roy, B. (2005). Paradigms and challenges. In J. Figueira, S. Greco, & M. Ehrgott (Eds.), Multiple criteria decision analysis: State-of-the-art surveys (pp. 3–24). Springer.
• Saaty, T. L. (1999). Basic theory of the analytic hierarchy process: How to make a decision. Revista de la Real Academia de Ciencias Exactas Fisicas y Naturales, 93(4), 395-423.
• Saaty, T.L., & Ramanujam, V. (1983). An objective approach to faculty promotion and tenure by the analytic hierarchy process. Research in High Education, 18, 311 331. https://doi.org/10.1007/BF00979603
• Sagir, M., & Ozturk, Z.K. (2010). Exam scheduling: Mathematical modeling and parameter estimation with the Analytic Network Process approach. Mathematical and Computer Modelling, 52, 930-941. https://doi.org/10.1016/j.jallcom.2011.02.170
• Shee, D.Y., & Wang, Y.S. (2008). Multi-criteria evaluation of the web-based e-learning system: A methodology based on learner satisfaction and its applications. Computer & Education, 50. 894-905. https://doi.org/10.1016/j.compedu.2006.09.005
• Soba, M. (2012). Universite öğrencilerinin performanslarinin akademisyenler tarafından analitik hiyerarşi sureci ile değerlendirilmesi [Evaluation of university students' performances by academics through the analytical hierarchy process]. Electronic Journal of Social Sciences, 11(42), 368-381.
• Subbaiah, K.V., Shekhar, N.C., & Kandukuri, N.R. (2014). Integrated DEA/TOPSIS approach for the evaluation and ranking of engineering education institutions-a case study. International Journal of Management Science and Engineering Management, 9(4), 249-264. https://doi.org/10.1080/17509653.2014.902758
• Syamsuddin, I. (2012). Fuzzy multi-criteria evaluation framework for E-learning software quality. Academic Research International, 2(1), 139-147.
• Tekindal, B., & Erumit, A.K. (2007). Analitik hiyerarşi sureci (AHS) ve bulanık AHS yöntemlerinin yüksek lisans öğrencisi seçimi problemi üzerinde karşılaştırılması [Comparison of analytical hierarchy process (AHS) and fuzzy AHP methods on graduate student selection problem]. Gazi Üniversitesi Endüstriyel Sanatlar Eğitim Fakültesi Dergisi, 21, 14-37.
• Tezergil, S. (2016). Vikor yöntemi ile Türk bankacılık sektörünün performans analizi [Evaluation of Country Indicators by Vikor Method]. Marmara Üniversitesi İktisadi ve İdari Bilimler Dergisi, 38(1), 357-373. https://doi.org/10.14780/iibd.92056
• Timor, M., (2011). Analitik hiyerarşi prosesi [Analytical hierarchy process]. Türkmen Kitabevi.
• Turki, A., & Duffuaa, S. (2003). Performance measures for academic departments. International Journal of Educational Management, 17(7), 330 338. https://doi.org/10.1108/IJEM-09-2014-0129
• Tzeng, G.H., Chiang, C.H., & Li, C.W. (2007). Evaluating intertwined effects in learning programs: A novel hybrid MCDM model based on factor analysis and DEMATEL. Expert Systems with Applications, 32, 1028 1044. https://doi.org/10.1016/j.eswa.2006.02.004
• Wu, H.Y., Chen, J.K., Chen, I.S., & Zhuo, H.H. (2012). Ranking universities based on performance evaluation by a hybrid MCDM model. Measurement, 45, 856-880. https://doi.org/10.1016/j.measurement.2012.02.009
• Yazdani, B.O., Yaghoubi, E.S., & Giri, E.S. (2011). Factors affecting the empowerment of employees (an empirical study). European Journal of Social Sciences, 20(2), 267-274.
• Yigit, T., Isik, A.H., & Ince, M. (2014). Web-based learning object selection software using analytical hierarchy process. IET Software, 8(4), 174–183. https://doi.org/10.1049/iet-sen.2013.0116
• Zare, M., Pahl, C., Rahnama, H., Nilashi, M., Mardani, A., Ibrahim, O., & Ahmadi, H. (2016). Multi-criteria decision-making approach in e-learning: A systematic review and classification. Applied Soft Computing, 45, 108 128. https://doi.org/10.1016/j.asoc.2016.04.020