How usable is Coursera? A usability analysis through eye-tracking and authentic tasks
Abstract
Massive Open Online Courses (MOOCs) are considered learning environments that eliminate many learning barriers. Online courses in MOOCs have become an opportunity for everyone during the lockdown of the COVID-19 outbreak. However, usability issues may cause problems such as high dropout rates and lack of learner's motivation. Therefore, in this study, the usability of Coursera, one of the most known MOOCs in the world, was evaluated. The evaluation was performed with ISO 9241-11 standard. The environment's effectiveness, efficiency, and satisfaction were evaluated with the authentic tasks requested to be done in Coursera. Additionally, the findings were supported by eye-tracking metrics such as fixation duration, fixation counts, heat maps, and gaze plots. Twelve individuals (six females, six males) participated in the authentic tasks, and three individuals (two females, one male) participated in the eye-tracking phase. Results of the study revealed that most participants successfully performed the authentic tasks and are generally satisfied with the usability of the environment. However, considering eye-tracking findings and Coursera Usage Satisfaction Survey, some usability problems such as inadequate language support and the difficulty of using the search feature emerged. In the end, possible reasons were discussed, and the suggestions were presented for usability improvements.
Keywords
usability , coursera , mooc , eye tracking
References
- Abran, A., Khelifi, A., Suryn, W., & Seffah, A. (2003). Usability meanings and interpretations in ISO standards. Software quality journal, 11(4), 325-338.
- Becker, S. A., & Mottay, F. E. (2001). A global perspective on web site usability. IEEE software, 18(1), 54-61.
- Conole, G. G. (2013). MOOCs as disruptive technologies: strategies for enhancing the learner experience and quality of MOOCs. Revista de Educación a Distancia, (39).
- Dix, A., Finlay, J., Abowd, G. D., & Beale, R. (2004). Human-computer interaction. Edingburh: Pearson Prentice Hall.
- Espada, J. P., Rodríguez, C. C., García-Díaz, V., & Crespo, R. G. (2014). Method for analysing the user experience in MOOC platforms. In Computers in Education (SIIE), International Symposium, 2014 (pp. 157-162). IEEE.
- Gütl, C., Rizzardini, R. H., Chang, V., & Morales, M. (2014). Attrition in MOOC: Lessons learned from drop-out students. In International Workshop on Learning Technology for Education in Cloud, 2014 (pp. 37-48). Springer International Publishing.
- Hone, K. S., & El Said, G. R. (2016). Exploring the factors affecting MOOC retention: A survey study. Computers & Education, 98, 157-168.
- International Organization for Standardization (ISO) (2018). ISO 9241-11: Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs)’, Part 11: Guidance on Usability Specification and Measures. Technical Report. Retrieved January 05, 2020 from https://www.iso.org/standard/63500.html
- Mackness, J., Mak, S. F. J., & Williams, R. (2010). The ideals and reality of participating in a MOOC. In L. Dirckinck-Holmfeld, V. Hodgson, C. Jones, M. de Laat, D. McConnell, & T. Ryberg (Eds.), Proceedings of the Seventh International Conference on Networked Learning (pp. 266-275).
- Mackness, J., Waite, M., Roberts, G., & Lovegrove, E. (2013). Learning in a small, task-oriented, connectivist MOOC: Pedagogical issues and implications for higher education. International Review of Research in Open & Distance Learning, 14, 140-159.
