A systematic literature review on multi-criteria decision making in higher education

Öz

The three components that form the basis of the educational process are the teacher, the learner, and the environment. These three components are affected by the developing and changing technology as a result of globalization considerably. Teaching and learning techniques should be updated and connected with these developments; new tools are therefore needed to make the necessary updates. Determination and application of the new tools include many decisions. Decision-makers can make more effective decisions using Multi-Criteria Decision-Making Techniques (MCDM), a complex decision-making tool that includes both quantitative and qualitative factors at present time. This study aimed to determine which MCDM methods are used in studies conducted in higher education, which is one of the most important development level indicators of countries, and to present a systematic literature review of MCDM method applications. The study was conducted in three stages: first, known electronics were searched until the end of 2021 using keywords; then, all studies were listed in a systematic taxonomy, and in the last stage, Thematic Network Analysis was used to evaluate the development of MCDM studies in the higher education area. It is determined that the Analytical Hierarchy Process (AHP) method is the most widely used method in higher education in MCDM applications. It was observed that the most common use of MCDM applications in higher education is e-learning as well. This study aims to be a guide for all researchers and practitioners who will study in both higher education and the MCDM areas.

Anahtar Kelimeler

• Higher education
• Literature review.
• Multi-Criteria Decision-Making Techniques

References

1. Altunok, T., Özpeynirci, O., Kazancoglu, Y., & Yilmaz, R. (2010). Comparatives of multicriteria decisions making methods for postgraduate student selection. Egitim Arastirmalari-Eurasian Journal of Educational Research, 40, 1-15.
2. Aly, M.F., Attia, H.A., & Mohammed, A.M. (2014). Prioritizing faculty of engineering education performance by using AHP-TOPSİS and balanced scorecard approach. International Journal of Engineering Science and Innovative Technology, 3(1), 11-23.
3. Anggrainingsih, R., Umam, M.Z., & Setiadi, H. (2018). Determining e-learning success factor in higher education based on user perspective using Fuzzy AHP. MATEC Web Conferences. 154, 03011. https://doi.org/10.1051/matecconf/201815403011
4. Aytaç, S., & Bayram, N. (2001). Üniversite gençliğinin iş ve eş seçimindeki etkin kriterlerinin analitik hiyerarşi süreci (AHP) ile analizi [Analysis of university youth's effective criteria for job and spouse selection by analytical hierarchy process (AHP)]. Öneri Dergisi, 4(16), 89-100. https://doi.org/10.14783/maruoneri.727643
5. Badri, M.A., & Abdulla, M.H. (2004). Awards of excellence in institutions of higher education: an AHP approach. International Journal of Educational Management, 18(4), 224-242. https://doi.org/10.1108/09513540410538813
6. Bali, O., & Gencer, C. (2005). AHP Bulanık AHP ve Bulanık Mantıkla Kara Harp Okuluna öğretim elemanı seçimi [Ahp, Fuzzy Ahp, and Fuzzy Logic Selection of Academic Staff to Turkish Military Academy]. Kara Harp Okulu Savunma Bilimleri Dergisi, 4, 24-43.
7. Begicevic, N., & Divjak, B. (2006). Validation of theoretical model for decision making about e-learning implementation. Journal of Information and Organizational Sciences, 30(2), 171-184.
8. Begicevic, N., Divjak, B., & Hunjak, T. (2007). Development of AHP based-model for decision making on e-learning implementation. Journal of Information and Organizational Sciences, 31, 13-24.

9. Bo, L., Xuning P., & Bingquan B. (2009). Modeling of network education effectiveness evaluation in fuzzy analytic hierarchy process. International Conference on Networking and Digital Society, 2, 198–200. ICNDS’09, IEEE.
10. Cakir, E., & Ozdemir, M. (2018). Altı sigma projelerinin bulanık copras yöntemiyle değerlendirilmesi: Bir üretim işletmesi örneği [Evaluation of six sigma projects with fuzzy copras method: An example of a manufacturing company]. Verimlilik Dergisi, 1, 7-39.
11. Cebi, A., & Karal, H. (2017). An application of fuzzy analytic hierarchy process (FAHP) for evaluating students’ Projects. Educational Research and Reviews, 12(3), 120-132. https://doi.org/10.5897/ERR2016.3065
12. Celikten, M., Gilic, F., Celikten., & Yildirim, A. (2019). Örgüt yönetiminde karar verme süreci: Bitmeyen bir tartışma [Decision making process in organization management: An endless discussion]. Mersin Üniversitesi Eğitim Fakültesi Dergisi, 15(2), 581-592.
13. Chao, R.J., & Chen, Y.H. (2009). Evaluation of the criteria and effectiveness of distance e-learning with consistent fuzzy preference relations. Expert Systems with Applications, 36, 10657-10662. https://doi.org/10.1016/j.eswa.2009.02.047
14. Chen, J.F., Hsieh, H.N., & Do, Q, H. (2015). Evaluating teaching performance based on fuzzy AHP and comprehensive evaluation approach. Applied Soft Computing, 28, 100-108. https://doi.org/10.1016/j.asoc.2014.11.050
15. Chen, J.K., & Chen, I.S. (2010). Using a novel conjunctive MCDM approach based on DEMATEL, fuzzy ANP, and TOPSIS as an innovation support system for Taiwanese higher education. Expert Systems with Applications, 37, 1981 1990. https://doi:10.1016/j.eswa.2009.06.079
16. Chen, Y., & Yang, M. (2010). Study and construct an online self-learning evaluation system model based on the AHP method. 2nd IEEE International Conference on Information and Financial Engineering (ICIFE), 54–58.
17. Chena, J.F. Hsieha, H.N. & Do, Q.H. (2015). Evaluating teaching performance based on fuzzy AHP and comprehensive evaluation approach. Applied Soft Computing, 28, 100-108. https://doi.org/10.1016/j.asoc.2014.11.050
18. Chi, H.K., Yeh, H.R., & Liao, L.-H., (2008). Applying fuzzy analytic hierarchy process to explore the university organizational performance in Taiwan. The Journal of Human Resource and Adult Learning, 4(1), 39–46.
19. Choi, C.R., & Jeong, H.Y. (2019). Quality evaluation for multimedia contents of e-learning systems using the ANP approach on a high-speed network. Multimedia Tools and Applications, 78, 28853-28875. https://doi.org/10.1007/s11042-019-7351-8 Cicekli, U.G., & Karacizmeli, A. (2013). Bulanık analitik hiyerarşi süreci ile basarılı öğrenci seçimi: Ege Universitesi İktisadi ve İdari Bilimler Fakültesi örneği [Successful student selection with fuzzy analytic hierarchy process: The example of Ege University Faculty of Economics and Administrative Sciences]. Ege Strategic Research Journal, 4(1), 77-103. https://doi.org/10.18354/esam.81730
20. Colace, F., Santo, M.D., & Pietrosanto, A. (2006). Evaluation models for e-learning platform: An AHP approach. Proceedings of Frontiers in Education. 1 6. https://doi.org/10.1109/FIE.2006.322312
21. Drake, P.R. (1998). Using the analytic hierarchy process in engineering education. Int. J. Engng Ed., 14, 191-196.
22. Dundar, S. (2008). Ders seçiminde analitik hiyerarşi prosesi uygulaması [Analytical Hierarchy Process application in course selection]. Suleyman Demirel Universitesi Iktisadi ve Idari Bilimler Fakultesi Dergisi 13(2), 217-226.
23. Ertugrul, I., & Karakasoglu, N. (2007). Fuzzy TOPSIS method for academic member selection in engineering faculty. Innovations in E-learning, Instruction Technology, Assesment, and Engineering Education, 151-156.
24. Feng, Y.J., Lu, H., & Bi, K. (2004). An AHP/DEA method for measurement of the efficiency of R&D management activities in universities. Intl. Trans. In Op. Res., 11, 181-191. https://doi.org/10.1111/j.1475-3995.2004.00450.x
25. Garg, R., & Jain, D. (2017). Fuzzy multi-attribute decision-making evaluation of e-learning websites using FAHP, COPRAS, VIKOR, WDBA. Decision Science Letters, 6, 351-364. http://dx.doi.org/10.5267/j.dsl.2017.2.003
26. Garg, R. (2017). Optimal selection of E-learning websites using multiattribute decision-making approaches. J. Multi Criteria Decision Analysis, 24, 187 196. https://doi.org/10.1002/mcda.1612
27. Garg, R., Kumar, R., & Garg, S. (2019). MADM-Based parametric selection and ranking of e-learning websites using fuzzy COPRAS. IEEE Trans. Educ., 62(1), 11–18. https://doi.org/10.1109/TE.2018.2814611
28. Ghosh, D., & Pal, A. (2017). Analysis of faculty teaching using a multi-criteria decision-making approach. International Journal of Engineering & Technology, 7, 74-78. https://doi.org/10.14419/ijet.v7i2.28.12884
29. Giannoulis, C., & Ishizaka, A. (2010). A Web-based decision support system with ELECTRE III for a personalized ranking of British universities. Decision Support Systems, 48, 488-497. https://doi.org/10.1016/j.dss.2009.06.008
30. Grandzol, J.R. (2005). Improving the faculty selection process in higher education: A case for the analytic hierarchy process. IR Applications, 6, 13.
31. Gupta, R., Garg, T.K., Gupta, S., & Goel, A. (2010). Decision Analysis Approach for Quality in Technical Education. Global Journal of Human Social Science, 10(1), 14-18.
32. Ho, W., Dey, P.K., & Higson, H.E. (2006). Multiple criteria decision-making techniques in higher education. International Journal of Educational Management, 20(5), 319-337. https://doi.org/10.1108/09513540610676403
33. Holton, G. (1975). On the role of themata in scientific thought. Science, 188(4186), 328-334.
34. Hsu, C.M., Yeh, Y.C., & Yen, J. (2009). Development of design criteria and evaluation scale for web-based learning platforms. International Journal of Industrial Ergonomics, 39, 90-95. https://doi.org/10.1016/j.ergon.2008.08.006
35. Jain, D., Garg, R., & Bansal, A. (2016). Selection and ranking of E-learning websites using weighted distance-based approximation. Journal of Computer Education, 3(2), 193-207. https://doi.org/10.1007/s40692-016-0061-6
36. Jesus. E.N., Rodrigues, J.C., & Antunes, C.H. (2007). A multicriteria decision support system for housing evaluation. Decision Support Systems, 43, 779 790. https://doi.org/10.1016/j.dss.2006.03.014
37. Jie, C. (2010). Evaluation and modeling of online courses using fuzzy AHP. 2010 International Conference on Computer and Information Application, 232-235.
38. Kabak, M., Ozceylan, E., Dagdeviren, M., & Genc T. (2017). Evaluation of distance education websites: a hybrid multicriteria approach. Turkish Journal of Electrical Engineering & Computer Sciences, 25, 2809–2819. https://doi.org/10.3906/elk-1512-271
39. Kara, M., & Karaca, Y. (2010). Üniversite öğrencilerin işletme bölümünü seçmelerinde etkili olan öncelikli faktörlerin analitik hiyerarşi prosesi metodu ile analizi: Bozok Üniversitesi İktisadi ve İdari Bilimler Fakültesinde bir uygulama [Analysis of the priority factors that affect university students' choice of business administration with the analytical hierarchy process method: An application in Bozok University Faculty of Economics and Administrative Sciences]. Organizasyon ve Yonetim Bilimleri Dergisi, 2(1), 133-140.
40. Kiarazm, A., & Koohkan, F. (2013). Performance evaluation in higher education institutes with the use of combinative model AHP and BSC. Journal of Basic and Applied Scientific Research, 3(4), 940-944
41. Kousalya, P., Ravindranath, V., & Vizayakumar, K. (2006). Student absenteeism in engineering colleges: Evaluation of alternatives using AHP. Journal of Applied Mathematics and Decision Sciences 2006. 1–26. https://doi.org/10.1155/JAMDS/2006/58232
42. Kurilovas, E., & Zilinskiene, I. (2013). New MCEQLS AHP method for evaluating the quality of learning scenarios. Technological and Economic Development of Economy, 19(1), 78-92. https://doi.org/10.3846/20294913.2012.762952
43. Kurilovas, E., & Serikoviene, S. (2013). New MCEQLS TFN method for evaluating quality and reusability of learning objects. Technological and Economic Development of Economy, 19(4), 706-723. https://doi.org/10.3846/20294913.2013.837112
44. Kwak N.K., & Lee C. (1998). A multicriteria decision-making approach to university resource allocations and information infrastructure planning. European Journal of Operational Research, 110(2), 234-242. https://doi.org/10.1016/S0377-2217 (97)00262-2
45. Lee, S.H. (2010). Using fuzzy AHP to develop intellectual capital evaluation model for assessing their performance contribution in a university. Expert Systems with Applications, 37, 4941-4947. https://doi.org/10.1016/j.eswa.2009.12.020
46. Lesmes, D., Castillo, M., & Zarama, R. (2009). Application of The Analytic Network Process (ANP) to Establish Weights in Order to Re-Accredit a Program of a University. Proceedings of the International Symposium on the Analytic Hierarchy Process, 29.
47. Li, W., Gao, X., & Fu, G. (2012). Fuzzy comprehensive assessment of network environment and learning quality combined with the analytic hierarchy process. 2nd International Conference on Consumer Electrics, Communications and Networks, 2600-2603, IEEE.
48. Liberatore, M.J., & Nydick, R.L. (1997). Group Decision Making in Higher Education Using the Analytic Hierarchy Process. Research in Higher Education, 38, 593–614. https://doi.org/10.1023/A:1024948630255
49. Lin, H.F. (2010). An application of fuzzy AHP for evaluating course website quality. Computers & Education, 54, 877-888. https://doi.org/10.1016/j.compedu.2009.09.017
50. Mehregan, M.R., Jamporazmey, M., Hosseinzadeh, M., & Mehrafrouz, M. (2011a). Proposing an approach for evaluating e-learning by integrating critical success factor and fuzzy AHP. International Conference on Innovation, Management and Service, Singapore.
51. Mehregan, M.R., Jamporazmey, M., Hosseinzadeh, M., & Mehrafrouz, M. (2011b). Application of fuzzy analytic hierarchy process in ranking modern educational systems’ success criteria. International Journal of e-Education, 1(4), 299-304.
52. Melon, M.G., Beltran, P.A., & Cruz, M.C.G. (2008). An AHP-based evaluation procedure for Innovative Educational Projects: A face-to-face vs. computer-mediated case study. Omega, 36, 754-765. https://doi.org/10.1016/j.omega.2006.01.005
53. Mendoza, G.A., Prabhub, R. (2000). Multiple criteria decision-making approaches to assessing forest sustainability using criteria and indicators: A case study. Forest Ecology and Management, 131, 107-126.
54. Mohammed, H.J., Kasim, M.M., & Shaharanee, I.N. (2018). Evaluating of e-learning approaches using AHP-TOPSIS technique, Journal of Telecommunication, Electronic and Computer Engineering, 10, 1-10.
55. Mondal, K., & Pramanik, S. (2014). Neutrosophic Sets and Systems, 6, 28-34.
56. Murakoshi H., Kawarasaki T., & Ochimizu K. (2001). Comparison using AHP Web-based learning with classroom learning, Proceedings of Symposium on Applications and the Internet Workshops, 67-73. https://doi.org/10.1109/SAINTW.2001.998212
57. Mustaffa, W.S.W., Shokory, S.M., & Kamis, H. (2006). The Analytical Hierarchy Process: Multi-Criteria Decision Making for Promoting Academic Staff in Higher Education. The Journal of Global Business Management, 2(2).
58. Nagpal, R., Mehrotra, D., Sharma, A., & Bhatia, P. (2013). ANFIS method for usability assessment of the website of an educational institute. World Applied Sciences Journal, 23(11), 1489–1498. https://doi.org/10.5829/ idosi.wasj.2013.23.11.790
59. Nagpal, R., Mehrotra, D., Bhatia, P.K., & Sharma, A. (2015). FAHP approach to rank educational websites on usability. International Journal of Computing and Digital Systems, 4(4), 251–260. http://dx.doi.org/10.12785/IJCDS/040404
60. Naveed, Q.N., Qureshi, M.R.N., Alsayed, A.O., Muhammad, A., Sanober, S. & Shah, A. (2017). Prioritizing barriers of E-learning for effective teaching-learning using fuzzy analytic hierarchy process (FAHP). 4th IEEE International Conference on Engineering Technologies and Applied Sciences (ICETAS), 1-8.
61. Naveed, Q.N., Qureshi, M.R., Tairan, N., Mohammad, A., & Shaikh, A. (2020). Evaluating critical success factors in implementing e-learning system using multi-criteria decision-making. PLoS ONE, 15(5), https://doi.org/10.1371/journal.pone.0231465
62. Nilashi, M., & Janahmadi, N. (2012). Assessing and prioritizing affecting factors in e-learning websites using the AHP method and fuzzy approach. Information and Knowledge Management, 2(1), 46-61.
63. Nikoomaram, H., Mohammadi, M., Javad Taghipouria, M., & Taghipourian, Y. (2009). Training performance evaluation of administration sciences instructors by fuzzy MCDM approach. Contemporary Engineering Sciences, 2(12), 559–575.
64. Omurbek, N., Karaatli, M., & Yetim, T. (2014). Analitik hiyerarsi surecine dayali TOPSIS ve VIKOR yöntemleri ile ADIM universitelerinin değerlendirilmesi [Evaluation of ADIM universities with TOPSIS and VIKOR methods based on analytical hierarchy process]. Selcuk Universitesi Sosyal Bilimler Dergisi, Dr.Mehmet YILDIZ special issues. 189-207.
65. Opricovic, S., (1998). Multicriteria Optimization of Civil Engineering Systems [Doctoral dissertation, Faculty of Civil Engineering].
66. Ozdemir, M.S., & Gasimov R.N. (2004). The analytic hierarchy process and multiobjective 0-1 faculty course assignment. European Journal of Operational Research, 157, 398-408. https://doi.org/10.1016/S0377-2217(03)00189-9
67. Ozkul, A.E., Girginer, N., & Ozturk, Z.K. (2007). Multi-Criteria Evaluation of Distance Education Implementation Models using Analytic Hierarchy Process, Proceedings of the 21st Annual Conference Empowering Asia through Partnership in Open and Distance Learning, 87.
68. Ozturk, Z.K. (2014). Using a multi-criteria decision making approach for open and distance learning system selection. Anadolu University Journal of Science and Technology– An Applied Sciences and Engineering, 15(1), 1-14.
69. Paksoy, S. (2015). Ülke göstergelerinin vikor yöntemi ile değerlendirilmesi [Evaluation of Country Indicators by Vikor Method]. Ekonomik ve Sosyal Araştırmalar Dergisi, 11(2), 153-169.
70. Perez Vergara, I.G., Arias Sa´nchez, J.A., Poveda-Bautista, R., & Diego-Mas J.A. (2020). Improving distributed decision making in inventory management: A combined ABC-AHP approach supported by teamwork. Complexity, 3–5, 1–13.
71. Politis, Y., & Siskos, Y. (2004). Multicriteria methodology for the evaluation of a Greek engineering department. European Journal of Operational Research, 156, 223-240. https://doi.org/10.1016/S0377-2217(02)00902-5
72. Ray, S. (2007). Selecting a doctoral dissertation supervisor: analytical hierarchy approach to the multiple criteria problem. International Journal of Doctoral Studies, 2, 23-32. https://doi.org/10.28945/55
73. Roy, B. (2005). Paradigms and challenges. In J. Figueira, S. Greco, & M. Ehrgott (Eds.), Multiple criteria decision analysis: State-of-the-art surveys (pp. 3–24). Springer.
74. Saaty, T. L. (1999). Basic theory of the analytic hierarchy process: How to make a decision. Revista de la Real Academia de Ciencias Exactas Fisicas y Naturales, 93(4), 395-423.
75. Saaty, T.L., & Ramanujam, V. (1983). An objective approach to faculty promotion and tenure by the analytic hierarchy process. Research in High Education, 18, 311 331. https://doi.org/10.1007/BF00979603
76. Sagir, M., & Ozturk, Z.K. (2010). Exam scheduling: Mathematical modeling and parameter estimation with the Analytic Network Process approach. Mathematical and Computer Modelling, 52, 930-941. https://doi.org/10.1016/j.jallcom.2011.02.170
77. Shee, D.Y., & Wang, Y.S. (2008). Multi-criteria evaluation of the web-based e-learning system: A methodology based on learner satisfaction and its applications. Computer & Education, 50. 894-905. https://doi.org/10.1016/j.compedu.2006.09.005
78. Soba, M. (2012). Universite öğrencilerinin performanslarinin akademisyenler tarafından analitik hiyerarşi sureci ile değerlendirilmesi [Evaluation of university students' performances by academics through the analytical hierarchy process]. Electronic Journal of Social Sciences, 11(42), 368-381.
79. Subbaiah, K.V., Shekhar, N.C., & Kandukuri, N.R. (2014). Integrated DEA/TOPSIS approach for the evaluation and ranking of engineering education institutions-a case study. International Journal of Management Science and Engineering Management, 9(4), 249-264. https://doi.org/10.1080/17509653.2014.902758
80. Syamsuddin, I. (2012). Fuzzy multi-criteria evaluation framework for E-learning software quality. Academic Research International, 2(1), 139-147.
81. Tekindal, B., & Erumit, A.K. (2007). Analitik hiyerarşi sureci (AHS) ve bulanık AHS yöntemlerinin yüksek lisans öğrencisi seçimi problemi üzerinde karşılaştırılması [Comparison of analytical hierarchy process (AHS) and fuzzy AHP methods on graduate student selection problem]. Gazi Üniversitesi Endüstriyel Sanatlar Eğitim Fakültesi Dergisi, 21, 14-37.
82. Tezergil, S. (2016). Vikor yöntemi ile Türk bankacılık sektörünün performans analizi [Evaluation of Country Indicators by Vikor Method]. Marmara Üniversitesi İktisadi ve İdari Bilimler Dergisi, 38(1), 357-373. https://doi.org/10.14780/iibd.92056
83. Timor, M., (2011). Analitik hiyerarşi prosesi [Analytical hierarchy process]. Türkmen Kitabevi.
84. Turki, A., & Duffuaa, S. (2003). Performance measures for academic departments. International Journal of Educational Management, 17(7), 330 338. https://doi.org/10.1108/IJEM-09-2014-0129
85. Tzeng, G.H., Chiang, C.H., & Li, C.W. (2007). Evaluating intertwined effects in learning programs: A novel hybrid MCDM model based on factor analysis and DEMATEL. Expert Systems with Applications, 32, 1028 1044. https://doi.org/10.1016/j.eswa.2006.02.004
86. Wu, H.Y., Chen, J.K., Chen, I.S., & Zhuo, H.H. (2012). Ranking universities based on performance evaluation by a hybrid MCDM model. Measurement, 45, 856-880. https://doi.org/10.1016/j.measurement.2012.02.009
87. Yazdani, B.O., Yaghoubi, E.S., & Giri, E.S. (2011). Factors affecting the empowerment of employees (an empirical study). European Journal of Social Sciences, 20(2), 267-274.
88. Yigit, T., Isik, A.H., & Ince, M. (2014). Web-based learning object selection software using analytical hierarchy process. IET Software, 8(4), 174–183. https://doi.org/10.1049/iet-sen.2013.0116
89. Zare, M., Pahl, C., Rahnama, H., Nilashi, M., Mardani, A., Ibrahim, O., & Ahmadi, H. (2016). Multi-criteria decision-making approach in e-learning: A systematic review and classification. Applied Soft Computing, 45, 108 128. https://doi.org/10.1016/j.asoc.2016.04.020