PSI-Entropi-Marcos entegrasyonu ile moodle öğrenim yönetim sistemi için öğrencilerin performans düzeylerinin kullanılabilirlik kriterleri açısından değerlendirilmesi

Year 2022, Volume: 28 Issue: 4, 588 - 603, 31.08.2022

Muhammet Yorulmaz Gülin Feryal Can

Abstract

Çalışmada, Moodle Öğrenim Yönetim Sistemi (ÖYS)’ni kullanırken, belirlenen amaçlara ulaşma kapsamında, son kullanıcı performanslarının belirlenerek karşılaştırılması amaçlanmıştır. Buna göre, birden fazla kullanılabilirlik kriteri dikkate alınarak, 18 kullanıcı Moodle ÖYS’yi kullanım performansları açısından önceliklendirilmiştir. Bu doğrultuda, kullanılabilirlik kriterlerinin önem ağırlıklarının belirlenmesinde, Tercih Seçim İndeksi (Preference Selection Index-PSI) ve Entropi entegrasyonu, son kullanıcıların önceliklendirilmesinde ise, Uzlaşma Çözümüne Göre Alternatiflerin Ölçülmesi ve Sıralaması (Measurement of Alternatives and Ranking according to Compromise Solution-MARCOS) yöntemlerinden faydalanılmıştır. PSI yöntemi, Entropi yöntemi ile entegre edilerek, hem son kullanıcıların kriterlere göre ortaya çıkan performans düzeylerindeki, hem de kriterlere ait tercih değişim değerlerindeki belirsizlik dikkate alınarak, yeni bir hibrit ağırlıklandırma yöntemi önerilmiştir. Önerilen bu yöntem, üç boyutlu bir başlangıç karar matrisi için uygulanmıştır. Böylece, alternatifler ve alternatiflerin kriterlere göre aldıkları değerlerden oluşan, iki boyutlu, geleneksel başlangıç karar matrisi geliştirilerek, daha esnek bir hale getirilmiştir. Analizde ölçülebilen (nesnel) kullanılabilirlik kriterleri dikkate alınmış ve bu kriterler, önceden belirlenen amaçlar olarak tanımlanan görevlerin, Moodle ÖYS üzerinde, kullanıcılar tarafından gerçekleştirilmesi ile Morae V3 programı tarafından ölçülmüştür. Bununla birlikte, önerilen PSI-Entropi entegrasyonu ile nesnel kriterler için elde edilen ağırlıklar, MARCOS yönteminde kullanılarak son kullanıcıların performans düzeylerine göre sıralamaları gerçekleştirilmiştir.

Keywords

Kullanılabilirlik, Moodle ÖYS, Son kullanıcı, PSI, Entropi, MARCOS

Evaluation of performance levels of students for moodle learning management system in terms of usability Criteria with PSI-Entropy-Marcos integration

Abstract

The study, it is aimed to determine and compare the end-user performances within the scope of achieving the determined objectives while using the Moodle Learning Management System (LMS). Accordingly, considering multiple usability criteria, 18 users were prioritized in terms of their performances in using Moodle LMS. In this direction, Preference Selection Index (PSI) and Entropy integration was used to determine the importance weights of usability criteria, and the Measurement of Alternatives and Ranking according to Compromise Solution (MARCOS) method was used to prioritize the end-users. A new hybrid weighting method has been proposed by integrating the PSI method with the Entropy method, taking into account both the uncertainty in the performance values of the end-users according to the criteria and the preference change values of the criteria. This proposed method is applied for a three-dimensional initial decision matrix. Thus, the traditional two-dimensional initial decision matrix, which consists of the alternatives and the values that the alternatives take according to the criteria, has been developed and made more flexible. The objective criteria taken into account in the analysis were measured by the Morae V3 program, with the tasks defined as predetermined goals being performed by the users on the Moodle LMS. In addition, the criteria weights obtained from the proposed PSI-Entropy integration were used in the MARCOS method to rank the end-users according to their performance levels.

Keywords

Usability, Moodle LMS, End user, PSI, Entropy, MARCOS

References

• [1] DataReportal. “Digital 2021 Global Digital Overview”. https://datareportal.com/reports/digital-2021-globaldigital-overview (20/08/2021).
• [2] İnal Y, Güner H. “Understanding software developers’ awareness and knowledge about user experience and usability”. Pamukkale Üniversitesi Mühendislik Bilimleri Dergisi, 22(5), 384-389, 2016.
• [3] Moodle.org. “Moodle Statistics”. https://moodle.net/stats/ (20/08/2021).
• [4] Attri R, Grover S. “Application of preference selection index method for decision making over the design stage of production system life cycle”. Journal of King Saud University-Engineering Sciences, 27(2), 207-216, 2015.
• [5] Bakir M, Atalık Ö. “Entropi ve aras yöntemleriyle havayolu işletmelerinde hizmet kalitesinin değerlendirilmesi”. Journal of Business Research-Turk, 10(1), 617-638, 2018.
• [6] Perçin S, Sönmez Ö. “Bütünleşik entropi̇ ağırlık ve topsis yöntemleri̇ kullanılarak türk sigorta ş ̇ irketler ̇ i̇ni̇n performansının ölçülmesi̇”. Uluslararası İktisadi ve İdari İncelemeler Dergisi,(18), EYI Special Issue, 565-582, 2018.
• [7] Stevic Z, Brkovic N. “A novel ıntegrated FUCOM-MARCOS model for evaluation of human resources in a transport company”. Logistics, 4(4), 2-14, 2020.
• [8] Thao NX. “Similarity measures of picture fuzzy sets based on entropy and their application in MCDM”. Pattern Analysis and Applications, 23(3), 1203-1213, 2020.
• [9] Li H, Wang W, Fan L, Li Q, Chen X. “A novel hybrid MCDM model for machine tool selection using fuzzy DEMATEL , entropy weighting and later defuzzification VIKOR”. Applied Soft Computing Journal, 91, 1-14, 2020.
• [10] Shubhra S, Dhiren G, Behera K, Goswami SS, Behera DK. “Solving material handling equipment selection problems in an ındustry with the help of entropy ıntegrated COPRAS and ARAS MCDM techniques”. Process Integration and Optimization for Sustainability, 5(4), 947-973, 2021.
• [11] Salehi V, Zarei H, Shirali GA, Hajizadeh K. “An entropybased TOPSIS approach for analyzing and assessing crisis management systems in petrochemical industries”. Journal of Loss Prevention in the Process Industries, 67, 1-8, 2020.
• [12] Yazdani M, Torkayesh AE, Santibanez-Gonzalez EDR, Otaghsara SK. “Evaluation of renewable energy resources using integrated Shannon Entropy-EDAS model”. Sustainable Operations and Computers, 1, 35-42, 2020.
• [13] Chodha V, Dubey R, Kumar R, Singh S, Kaur S. “Selection of industrial arc welding robot with TOPSIS and entropy MCDM techniques”. Materials Today: Proceedings Journal, 2021. https://doi.org/10.1016/j.matpr.2021.04.487.
• [14] Seker S, Aydin N. “Hydrogen production facility location selection for Black Sea using entropy based TOPSIS under IVPF environment”. Internationa ournal of Hydrogen Energy, 45(32), 15855-15868, 2020.
• [15] Lam WS, Lam WH, Jaaman SH, Liew, KF. “Performance evaluation of construction companies using integrated entropy-fuzzy vikor model”. Entropy, 23(3), 1-16, 2021.
• [16] Sahoo SK, Choudhury BB. “Optimal selection of an electric power wheelchair using an integrated COPRAS and EDAS approach based on Entropy weighting technique”. Decision Science Letter, 11(1), 21-34, 2022.
• [17] Chaurasiya R, Jain D, “Pythagorean fuzzy entropy measure-based complex proportional assessment technique for solving multi-criteria healthcare waste treatment problem”. Granular Computing, 2022. https://doi.org/10.1007/s41066-021-00304-z.
• [18] Deveci M, Öner SC, Ciftci ME, Özcan E, Pamucar D. “Interval type-2 hesitant fuzzy Entropy-based WASPAS approach for aircraft type selection”. Applied Soft Computing, 114, 1-14, 2022.
• [19] Ulutaş A. “Stacker selection with PSI and WEDBA Methods”. International Journal of Contemporary Economics and Administrative Sciences, 10(2), 493-504, 2020.
• [20] Akbulut OY. “Gri entropi temelli PSI ve ARAS ÇKKV yöntemleriyle Türk mevduat bankalarının performans analizi”. Finans Ekonomi ve Sosyal Araştırmalar Dergisi, 5(2), 171-187, 2020.
• [21] [Ulutaş A, Popovic G, Radanov P, Stanujkic D, Karabasevic D. “A new hybrid fuzzy psi-piprecia-cocoso mcdm based approach to solving the transportation company selection problem”. Technological and Economic Development of Economy, 27(5), 1227-1249, 2021.
• [22] Chen Z, Zhong P, Liu M, Sun H, Shang K. “A novel hybrid approach for product concept evaluation based on rough numbers, shannon entropy and TOPSIS-PSI”. Journal of Intelligent & Fuzzy Systems, 40, 12087-12099, 2021.
• [23] Amin M, Irawati N, Sinaga HDE, Retnosari D, Maulani J, Raja HDL. “Decision support system analysis for selecting a baby cream product with preference selection ındex (PSI) baby sensitive skin under 3 year”. Virtual Conference on Engineering, Science and Technology (ViCEST) 2020, Kuala Lumpur, Malaysia, 12-13 August 2020.
• [24] Reddy PV, Meenakshi Reddy R, Srinivasa Rao P, Mohana Krishnudu D, Saikumar Reddy RV, Eswara Kumar A. “Parameters Selection for Enhanced Mechanical and Wear Properties of Natural Fiber Reinforced Hybrid Composites Using PSI Technique”. Journal of Naural Fibers, 2021. https://doi.org/10.1080/15440478.2021.1993484.
• [25] Reddy PV, Reddy BV. “Multi-objective Tubular Hydroforming Process Parametric Optimization using TOPSIS and PSI techniques”. PREPRINT, 2021. https://doi.org/10.21203/rs.3.rs-866901/v1.
• [26] Stankovic M, Stevic Z, Das DK, Subotic M, Pamucar D. “A New Fuzzy MARCOS Method for Road Traffic Risk Analysis”. Mathematics, 8(3), 2-18, 2020.
• [27] Puška A, Stojanović I, Maksimović A. “Project management software evaluation by usıng the measurement of alternatıves and (MARCOS) method”. Operational Research in Engineering Sciences: Theory and Applications, 3(1), 89-102, 2020.
• [28] Ilieva G, Yankova T, Hadjieva V, Doneva R, Totkov G. “Cloud service selection as a fuzzy multi-criteria problem”. TEM Journal, 9(2), 484-495, 2020.
• [29] Badi I, Pamucar D. “Supplier selection for steelmaking company by using combıned grey-marcos methods”. Decision Making: Applications in Management and Engineering, 3(2), 37-47, 2020.
• [30] Chattopadhyay R, Chakraborty S, Chakraborty S. “An integrated D-Marcos method for supplier selectıon in an iron and steel industry”. Decision Making: Applications in Management and Engineering, 3(2), 49-69, 2020.
• [31] Gong X, Yang M, Du P. “Renewable energy accommodation potential evaluation of distribution network: A hybrid decision-making framework under interval type-2 fuzzy environment”. Journal of Cleaner Production, 286, 1-21, 2021.
• [32] Pamucar D, Iordache M, Deveci M. “A new hybrid fuzzy multi-criteria decision methodology model for prioritizing the alternatives of the hydrogen bus development : A case study from Romania”. International Journal of Hydrogen Energy, 46(57), 29616-29637, 2021.
• [33] Ulutaş A, Karabasevic D, Popovic G, Stanujkic D, Nguyen PT, Karaköy Ç. “Development of a novel ıntegrated CCSDITARA-MARCOS decision-making approach for stackers selection in a logistics system”. Methematics, 8(1672), 1-15, 2020.
• [34] Biswas S. “Measuring performance of healthcare supply Chains in India: a comparative analysis of multi-criteria decision making methods”. Decision Making: Applications in Management and Engineering, 3(2), 162-189, 2020.
• [35] Khoshabi P, Nejati, E, Ahmadi, SF, Chegini A, Makui A, Ghousi R. “Developing a multi-criteria decision making approach to compare types of classroom furniture considering mismatches for anthropometric measures of university students”. PLoS One, 15(9), 1-25, 2020.
• [36] Graf S, List B. “An Evaluation of open source e-learning platforms stressing adaptation issues”. Proceedings of the Fifth IEEE International Conference on Advanced Learning Technologies (ICALT’05), Kaohsiung, Taiwan, 5-8 July 2005.
• [37] Kakasevski G, Mihajlov M, Sime A, Chungurski S. “Evaluating Usability in Learning Management System Moodle”. Proceedings of the ITI 2008 30th Internatioanl Confefernce on Information Technology Interfaces, Cavtat/Dubrovnik, Croatia, 23-26 June, 2008.
• [38] Kirner TG, Custódio CDA, Kirner, C. “Usability Evaluation Of The Moodle System From The Teachers’ Perspective”. Brazil IADIS Internataional Conference ELearning, Amsterdam, The Netherlands, 22-25 July, 2008.
• [39] Tee SS, Wook TSMT, Zainudin S. “User testing for moodle application”. International Journal of Software Engineering and its Applications , 7(5), 243-252, 2013.
• [40] Ivanović M, Putnik Z, Komlenov Ž, Welzer T, Hölbl M, Schweighofer T. “Usability and privacy aspects of moodle: Students’ and teachers’ perspective”. Informatica, 37(3), 221-230, 2013.
• [41] Unal Z, Unal A. “Investigating and comparing user experiences of course management systems: BlackBoard vs. moodle”. Journal of Interactive Learning Research, 25(1), 101-123, 2014.
• [42] Senol L, Gecili H, Onay Durdu P. “Usability evaluation of a moodle based learning management system”. World Conference on Educational Multimedia, Tampere, Finland, 23-26 June 2014.
• [43] Farmanesh P, Samani AA, Magusa G. “Heuristic evaluation of the usability of learning management system (Moodle) at Eastern Mediterranean University”. International Journal of Scientific Research in Information Systems and Engineering, 2(1), 22-36, 2016.
• [44] Hasan L. “Usability problems on desktop and mobile ınterfaces of the moodle learning management system (LMS)”. ICEBA 2018: Proceedings of the 2018 International Conference on E-Business and Applications, Da Nang, Viet Nam, 23-25 February 2018.
• [45] Suner A. “Evaluation of internet assisted biostatistics course with Moodle”. Ege Journal of Medicine, 57(4), 201-211, 2018.
• [46] Başaran S, Khalleefah R, Mohammed H. “Usability evaluation of open source learning management systems”. International Journal of Advanced Computer Science and Applications, 11(6), 400-410, 2020.
• [47] Aliyu OA, Arasanmi C, Ekundayo S. “Do demographic characteristics moderate the acceptance and use of the Moodle learning system among business students ?”. International Journal of Education and Development using Information and Communication Technology, 15(1), 165-178, 2019.
• [48] Melton J. “The LMS moodle: A usability evaluation”. Languages Issues, 11(12), 1-24, 2006.
• [49] Yorulmaz M, Can GF. “Moodle öğrenme yönetim sistemi sürümlerinin öğrenci perspektifinden karşılaştırmalı kullanılabilirlik analizi”. Journal of Turkish Operations Management, 1(4), 336-356, 2020.
• [50] Radwan NM, Senousy MB, Alaa El Din MR. “Neutrosophic AHP multi criteria decision making method applied on the selection of learning management system”. International Journal of Advancenets in Computing Technology, 8(5),95-105, 2016.
• [51] Ayouni S, Menzli LJ, Hajjej F, Maddeh M. “A Hybrid fuzzy DEMATEL-AHP/VIKOR method for LMS selection”. ECEL 18th European Conference on e-Learning, Copenhagen, Denmark, 07-08 November 2019.
• [52] Artsın M, Günal K. “Öğrenme yönetim sistemi seçiminde kullanılacak kriterlerin belirlenmesine yönelik çok ölçütlü karar verme yöntemi önerisi”. Açıköğretim Uygulamaları ve Araştırmaları Dergisi, 7(2),87-108, 2021.
• [53] Maniya K, Bhatt MG. “A selection of material using a novel type decision-making method: preference selection index method”. Materials and Design, 31(4), 1785-1789, 2010.
• [54] Shannon CE. “A mathematical theory of communication”. The Bell System Technical Journal, 27(3), 379-423, 1948.
• [55] Nielsen J. Usability Engineering. San Francisco, USA, Morgan Kaufmann, 1993.
• [56] Hart SG. “Nasa-Task Load Index (NASA-TLX); 20 Years Later”. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, San Fransisco, USA, 16-20 October 2006.