Usability Evaluation of Massive Open Online Courses (MOOC) Websites with the Cognitive Walkthrough

Year 2021, Volume: 14 Issue: 4, 377 - 389, 31.10.2021

Elif Bilge Şahin , Pınar Onay Durdu

Abstract

Nowadays, Massive Open Online Courses (MOOC) has emerged as one of the widely preferred learning methods. In these systems, students can access learning resources through websites, regardless of time and place. Ensuring the usability of these systems is essential for their effectiveness. Within the scope of this study, it was aimed to evaluate the usability of three popular MOOC websites, which were Udemy, Udacity and Coursera. Cognitive walkthrough, an expert-based evaluation method, was used for usability evaluation. Actions to be performed with these systems were identified under four categories which were the system design and access, educational resources, course, profile, and help for MOOC websites. Usability problems were determined based on the responses generated by the system while performing these actions. The identified usability problems were related to the lack of visibility of the necessary functions in the interface, deficiencies in the interface design, and some missing functions. Among the three MOOC websites examined, fewer usability problems were identified on the Coursera website, while the largest number of problems was identified on the Udacity website. In addition, Recommendations have been made to improve the design of these systems and to increase the usability of MOOC websites in general in addressing the identified problems.

Keywords

usability, usability evaluation, cognitive walkthrough method, massive open online course

Bilişsel Gezinti ile Kitlesel Açık Çevrimiçi Ders (KAÇD) Web Sitelerinin Kullanılabilirliğinin Değerlendirilmesi

Abstract

Günümüzde Kitlesel açık çevrimiçi dersler (KAÇD) yaygın olarak tercih edilen öğrenme yöntemlerinden biri olarak karşımıza çıkmaktadır. Bu sistemlerde öğrenciler zaman ve mekândan bağımsız olarak öğrenme kaynaklarına web siteleri aracılığıyla erişebilmektedirler. Bu erişimler sırasında sistemlerin etkinliği için kullanılabilir olmaları oldukça önem taşımaktadır. Bu çalışma kapsamında, üç popüler KAÇD web sitesi olan Udemy, Udacity ve Coursera’nın kullanılabilirliklerinin değerlendirilmesi hedeflenmiştir. Kullanılabilirlik değerlendirmesi için uzman temelli bir değerlendirme yöntemi olan bilişsel gezinti kullanılmıştır. KAÇD web sitelerinde gerçekleştirilebilecek eylemler sistem tasarımı ve erişimi, eğitim kaynakları, kurs, profil ve yardım kategorileri altında belirlenmiştir. Bu eylemler gerçekleştirilirken sistem tarafından oluşan yanıtlara dayalı olarak kullanılabilirlik problemleri belirlenmiştir. Belirlenen kullanılabilirlik problemlerinin; arayüzde yer alan gerekli fonksiyonların görünür olmamaları, arayüz tasarımındaki yetersizlikler ve bazı eksik fonksiyonlar ile ilgili olduğu tespit edilmiştir. İncelenen üç KAÇD web sitesi arasında Coursera web sitesinde daha az sayıda kullanılabilirlik problemi belirlenirken en fazla sayıda problem Udacity web sitesinde tespit edilmiştir. Tespit edilen problemlere yönelik olarak bu sistemlerin tasarımlarını iyileştirmek ve genel olarak KAÇD web sitelerinin kullanılabilirliğini artırmak için öneriler sunulmuştur.

Keywords

kullanılabilirlik, kullanılabilirlik değerlendirmesi, bilişsel gezinti yöntemi, kitlesel açık çevrimiçi ders

References

• S. Çoban, “Uzaktan ve teknoloji destekli eğitimin gelişimi”, XVII. Türkiye'de İnternet Konferansı, Eskişehir, 2012.
• M. Kırık, “Uzaktan Eğitimin Tarihsel Gelişimi ve Türkiye’deki durumu”, Marmara University Journal of Communication, 21, 73-94, 2014.
• C. M. Stracke, “How can e-learning and MOOCs reveal and exploit the hidden treasures of open research and open education?”, Proceedings of European Distance and E-Learning Network Conference, 11, 2014.
• D. Cormier, G. Siemens, “Through the open door: Open courses as research, learning, and engagement”, EDUCAUSE Review, 45(4), 30-39, 2010.
• C. O. Rodriguez, “MOOCs and the AI-Stanford Like Courses: Two Successful and Distinct Course Formats for Massive Open Online Courses”, European Journal of Open, Distance and E-Learning, 2012.
• M. H. Baturay, “An overview of the world of MOOCs”, Procedia-Social and Behavioral Sciences, 174, 427-433, 2015.
• R. Schuwer, J. Darco, Institutional MOOC Strategies in Europe Status Report Based on a Mapping Survey Conducted, European Association of Distance Teaching Universities, 2015.
• R. Kop, A. Hill, “Connectivism: learning theory of the future or vestige of the past”, International Review of Research in Open and Distance Learning, 2008.
• I., De Waard, “mMOOC design: Ubiquitous, open learning in the cloud”, Handbook of mobile learning, Editör: Berge, Z.L., Muilenburg, L.Y., New York & London: Routledge, 356-368, 2013.
• A. Atik, A. Ata, “Alternatif Dijital Eğitim Platformu Olarak Kitlesel Çevrimiçi Açık Ders (Mooc) Uygulamaları”, Social Sciences, 13(4): 144-154, 2018.
• J. Nielsen, Designing Web Usability: The Practice of Simplicity. New Riders Publishing, Indianapolis. 2000.
• M. Shubina, Usability evaluation of MOOC platforms, Bachelor’s Thesis, Haaga -HeliaUniversity of Applied Science, 2016.
• C. Katsanos, N. Tselios, M. Xenos, “Perceived usability evaluation of Learning Management Systems: a first step towards standardization of the System Usability Scale”, 16th Panhellenic Conference on Informatics, 302–307, 2012.
• A. Granić, “Experience with usability evaluation of e-learning systems”, Universal Access in the Information Society, 7(4), 209–221, 2008.
• B. Wong, T.T. Nguyen, E. Chang, N. Jayaratna, “Usability metrics for e-learning”, In Workshop on human computer interface for semantic web and web applications, Italy: Springer-Verlag Berlin Heidelberg, 2889, 235–252, 2003.
• Internet: International Organization for Standardization (ISO), Ergonomics of Human-System Interaction- Part 210: Human Centred Design for Interactive Systems, https://www.iso.org/standard/77520.html, 2019.
• J. Nielsen, Usability Engineering, Academic Press, Boston, 1993.
• K. Çağıltay, Teoriden Pratiğe İnsan Bilgisayar Etkileşimi ve Kullanılabilirlik Mühendisliği, Seçkin Yayıncılık, Ankara, Türkiye, 2011.
• J. Nielsen, R. Molich, “Heuristic evaluation of user interfaces”, In Proceedings of the SIGCHI conference on Human factors in computing systems, 249-256,1990.
• C. Lewis, P. Polson, C. Wharton, J. Rieman, “Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces”, In Proceedings of the ACM CHI’90 Conference, 235-242,1990.
• C. Lewis, C. Wharton,"Cognitive walkthroughs", Handbook of Human-Computer Interaction, Editör: Helander M., Landauer T.K., Prabhu P., Amsterdam:Elsevier, 717-732, 1997.
• P. Polson, C. Lewis, J. Rieman, C. Wharton, “Cognitive walkthroughs: A method for theory-based evahtation of user interfaces”, International Journal of ManMachine Studies, 36(5), 741–773, 1992.
• C. Wharton, J. Bradford, R. Jeffries, M. Franzke,, “Applying cognitive walkthroughs to more complex user interfaces: Experiences, issues, and recommendations”, Proceedings of the ACM CHI 92 Human Factors in Computing Systems Conference, 381-388, 1992.
• C. Wharton, J. Rieman, C. Lewis, P. Polson, "The cognitive walkthrough method: A practitioner’s guide", Usability Inspection Methods , Editör: Nielsen J., Mack R.L., New York: John Wiley, 105-140, 1994.
• S. Riihiaho, Experiences with usability evaluation methods, License thesis, Helsinki University of Technology, 2000.
• A. Tsironis, C. Katsanos, M. Xenos, “Comparative usability evaluation of three popular MOOC platforms”, Global Engineering Education Conference, 608-612, 2016.
• Ş. E. İşgör, B. O. Turan, “Mobil Ortamlarda Kitlesel Açık Çevrimiçi Derslerin (KAÇD) Kullanılabilirliğinin Değerlendirilmesi”, Mersin University Journal of the Faculty of Education, 13(2), 595-608, 2017.
• O. Korableva, T. Durand, O. Kalimullina, I. Stepanova, “Studying user satisfaction with the MOOC platform interfaces using the example of coursera and open education platforms”, Paper presented at the ACM International Conference Proceeding Series, 26- 30, 2019.
• E. Hildebrand, J. M. Bekki, B. L. Bernstein, C. J. Harrison, ‘‘Online learning environment design: A heuristic evaluation”, In Proc. 120th ASEE Annual Conference Exposition, 1–11, 2013.
• S. G. Jiménez-González, R. Mendoza-González, H. Luna-García, C. A. Martínez-Sandoval, F. Macias-Ruvalcaba, M. A. M. Rocha, I. De La Parra González, “Heuristic Approach to Evaluate Basic Types of Interactions-Communications in MOOCs”, Proceedings of the 6th Mexican Conference on Human-Computer Interaction, 32–38, 2016.
• F. Iniesto, An Investigation Into The Accessibility Of Massive Open Online Courses (MOOCs), PhD Thesis, The Open University, 2020.
• A. Ntourmas, N. Avouris, S. Daskalaki, Y. Dimitriadis, “Evaluation of a Massive Online Course Forum: Design Issues and Their Impact on Learners’ Support”, Human-Computer Interaction – INTERACT, 197-206, 2019.
• H. Al-Dossari, “A Heuristic Based Approach for Usability Evaluation of Academic Portals”, International Journal of Computer Science and Information Technology, 9(3), 15–30, 2017.
• Internet: Udemy, https://www.udemy.com, 27.11.2020.
• Internet: Udacity, https://www.udacity.com, 27.11.2020.
• Internet: Coursera, https://www.coursera.org/, 27.11.2020.
• D. Sideris, A. Tsironis, M. Xenos, “Comparative evaluation of MOOC technologies: the case of Hellenic Open University”, 7th International Conference on Education and New Learning Technologies, 5543–5552, 2015.
• E. Usselman, E.M. Borycki, A.W. Kushniruk, “The evaluation of electronic perioperative nursing documentation using a cognitive walkthrough approach”, Studies in Health Technology and Informatics, 208, 331–336, 2015.
• K. Jordan, “Initial trends in enrolment and completion of massive open online courses”, International Review of Research in Open and Distributed Learning, 15( 1), 2014.