Usability Evaluation of University Websites: A Mixed-Methods Case Study from a Multi-Stakeholder Perspective

Muhammed Çağrı Aksu

doi:10.53850/joltida.1800362

Usability Evaluation of University Websites: A Mixed-Methods Case Study from a Multi-Stakeholder Perspective

Abstract

This study evaluates the usability of a state university website using a mixed-methods approach that integrates quantitative and qualitative evidence to provide a comprehensive understanding of user experiences. A total of 60 participants (students, academic staff, and administrative staff) completed five representative tasks and responded to the Turkish System Usability Scale (SUS-TR), while qualitative data were gathered through open-ended questions and observational notes. Quantitative findings indicated a good level of perceived usability (SUS-TR: M = 76.46) together with strong task performance (overall success rate = 86.3% and mean completion time = 44 s). Between-group analyses also revealed meaningful differences across age cohorts: Generation Z reported lower usability perceptions and showed a greater tendency to abandon the website and rely on external platforms (e.g., search engines or social media) when information was difficult to locate. Qualitative thematic analysis highlighted recurring usability barriers related to navigation complexity, ineffective internal search, and content structure/placement, helping to explain observed delays and external resource usage. Based on the integrated findings, recommendations include simplifying navigation structures, improving site-wide search (including common abbreviations), reorganizing content in line with user expectations, and considering supportive features such as a chatbot for frequently sought information. Overall, the study emphasizes the importance of user-centered design in higher education websites and demonstrates how generational differences can shape online interaction behaviors and usability outcomes.

Keywords

Ethical Statement

This study received ethical approval from Artvin Coruh University Scientific Research and Publication Ethics Committee (Decision No: E-18457941-050.99-175301, dated April 16, 2025). Additionally, necessary fieldwork permissions were obtained from the university administration. Informed consent was acquired from all participants prior to their participation in the research.

Thanks

I would like to sincerely thank the students and staff of Artvin Çoruh University who voluntarily participated in this study. I also gratefully acknowledge the administrative approval and support provided by the university's management.

References

Abuqaddom, I., Alazzam, H., Hudaib, A., & Al-Zaghoul, F. (2019). A measurable website usability model: Case Study University of Jordan. 10th international conference on information and communication systems (ICICS), Irbid, Jordan.
Acala, A., & Talirongan, H. (2023). Assessing user satisfaction and usability of a university portal: A quantitative study utilizing the computer system usability questionnaire (CSUQ). Psychology and Education: A Multidisciplinary Journal, 14(4), 408-415. http://dx.doi.org/10.5281/zenodo.8426291
Adekunle, A., Alao, A., & Akande, O. (2022). A systematic review on usability evaluation for university websites. International Journal of Computer Applications Technology and Research, 11(02), 22-28. http://dx.doi.org/10.7753/IJCATR1102.1003
Akgül, Y. (2021). Accessibility, usability, quality performance, and readability evaluation of university websites of Turkey: A comparative study of state and private universities. Universal Access in the Information Society, 20(1), 157-170. https://doi.org/10.1007/s10209-020-00715-w
Albaghli, R., Beidas, A., & Attar, N. (2024). Eyes on higher education: Evaluating web usability in Kuwaiti private universities using eye-tracking and SUPR-Q metrics. Journal of Engineering Research. https://doi.org/10.1016/j.jer.2024.12.001
Allison, R., Hayes, C., McNulty, C. A., & Young, V. (2019). A comprehensive framework to evaluate websites: literature review and development of GoodWeb. JMIR formative research, 3(4), e14372.
Bangor, A., Kortum, P., & Miller, J. (2009). Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies, 4(3), 114-123. https://uxpajournal.org/determining-what-individual-sus-scores-mean-adding-an-adjective-rating-scale/
Baraković, S., Skorin-Kapov, L., & Husić, J. B. (2024). The impact of QoE factors on the perception of constructs comprising information quality, usability, and aesthetics in mobile web browsing. International Journal of Human–Computer Interaction, 40(22), 6996-7018. https://doi.org/10.1080/10447318.2023.2260982

Baskın, Z. P. (2022). Üniversite web sitelerinin tasarımı ve kullanılabilirlik açısından değerlendirilmesi [Evaluation of university websites in terms of design and usability]. Sanat ve Tasarım Dergisi(29), 97-114. https://doi.org/10.18603/sanatvetasarim.1137911
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa
Brechon, S. (2012). Gender matters: a guide to attracting male and female clients to your website. Journal of Aesthetic Nursing, 1(6), 318-321. https://doi.org/10.12968/joan.2012.1.6.318
Brooke, J. (1996). SUS-A quick and dirty usability scale. In P. W. Jordan, B. Thomas, I. L. McClelland, & B. Weerdmeester (Eds.), Usability evaluation in industry. CRC Press.
Chan, T. C., Leow, M. C., & Ong, L. Y. (2022). A practical usability study framework using the SUS and the affinity diagram: A case study on the online roadshow website. Pertanika Journal of Science & Technology, 30(2), 1439 - 1455. http://dx.doi.org/10.47836/pjst.30.2.33
Creswell, J. W., & Creswell, J. D. (2014). Research design: qualitative, quantitative and mixed methods approaches. Sage Publications.
Csontos, B., & Heckl, I. (2025). Five years of changes in the accessibility, usability, and security of Hungarian government websites. Universal Access in the Information Society. https://doi.org/10.1007/s10209-025-01223-5
De Angelis, M., Volpi, L., Giusino, D., Pietrantoni, L., & Fraboni, F. (2025). Acceptability and usability of a digital platform promoting mental health at work: A qualitative evaluation. International Journal of Human–Computer Interaction, 41(2), 1246-1259. https://doi.org/10.1080/10447318.2024.2313892
Demirkol, D., & Şeneler, Ç. (2018). A Turkish translation of the system usability scale: the SUS-TR [Sistem Kullanılabilirlik Ölçeğinin Türkçeye Çevirisi: SUS-TR]. Uşak Üniversitesi Sosyal Bilimler Dergisi, 11(3), 237-253. http://dx.doi.org/10.29217/uujss.495
Dimock, M. (2019). Defining generations: Where Millennials end and Generation Z begins. https://www.pewresearch.org/short-reads/2019/01/17/where-millennials-end-and-generation-z-begins/
Fakrudeen, M. (2025). Evaluation of the accessibility and usability of university websites: A comparative study of the Gulf region. Universal Access in the Information Society, 24, 1883-1898. https://doi.org/10.1007/s10209-024-01160-9
Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255-274. https://doi.org/10.3102/01623737011003255
Hamarashid, H., & Saeed, S. (2017). Usability testing on Sulaimani Polytechnic University website. International Journal of Multidisciplinary and Current Research, 5, 650-653. https://ijmcr.com/wp-content/uploads/2017/06/Paper22650-653.pdf
ISO. (2018). ISO 9241-11:2018 ergonomics of human-system interaction—part 11: usability: definitions and concepts. Retrieved July 24, 2025 from https://www.iso.org/standard/63500.html
Jawad, H. (2020). Usability analysis of university websites in Pakistan. Lahore Garrison University Research Journal of Computer Science and Information Technology, 4(2), 13-29. http://dx.doi.org/10.54692/lgurjcsit.2020.0402131
Jilani, M. A., Sheikh, A., Shah, F. A., & Saqlain, S. M. (2025). Usability evaluation of academic library websites: a systematic literature review. Information Discovery and Delivery. https://doi.org/10.1108/IDD-09-2024-0144
Kortum, P., & Oswald, F. L. (2018). The impact of personality on the subjective assessment of usability. International Journal of Human–Computer Interaction, 34(2), 177-186. https://doi.org/10.1080/10447318.2017.1336317
Kuhail, M. A., Al Katheeri, H., Negreiros, J., Seffah, A., & Alfandi, O. (2023). Engaging students with a chatbot-based academic advising system. International Journal of Human–Computer Interaction, 39(10), 2115-2141. https://doi.org/10.1080/10447318.2022.2074645
Kvasnicová, T., & Kremeňová, I. (2015). The use of eye tracking to assess the usability of university website. CBU International Conference Proceedings, Prague, Czech Republic.
Lewis, J. R. (1995). IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. International Journal of Human‐Computer Interaction, 7(1), 57-78. https://doi.org/10.1080/10447319509526110
Lynch, K. R., Schwerha, D. J., & Johanson, G. A. (2013). Development of a weighted heuristic for website evaluation for older adults. International Journal of Human–Computer Interaction, 29(6), 404-418. https://doi.org/10.1080/10447318.2012.715277
Macakoğlu, Ş. S., Peker, S., & Medeni, İ. T. (2023). Accessibility, usability, and security evaluation of universities’ prospective student web pages: A comparative study of Europe, North America, and Oceania. Universal Access in the Information Society, 22, 671-683. https://doi.org/10.1007/s10209-022-00869-9
Moran, K. (2019). Usability (User) Testing 101. Retrieved July 26, 2019 from https://www.nngroup.com/articles/usability-testing-101/
Nielsen, J. (1993). Usability engineering. Morgan Kaufmann.
Nielsen, J. (1994). Usability inspection methods (J. Nielsen & R. L. Mack, Eds.). John Wiley & Sons.
Norman D. A. (2013). The design of everyday things. MIT Press.
Oliveira, C., Maia, M., Vairinhos, M., Pereira, A., Oliveira, I., & Vagos, P. (2022). A social anxiety mobile intervention for college students attending therapy: A usability and acceptability study. International Journal of Human–Computer Interaction, 38(14), 1333-1344. https://doi.org/10.1080/10447318.2021.2002042
Perrig, S. A. C., von Felten, N., Vollenwyder, B., & Opwis, K. (2024). Development and psychometric validation of a positively worded German version of the System Usability Scale (SUS). International Journal of Human–Computer Interaction, 1-21. https://doi.org/10.1080/10447318.2024.2434720
Roncal-Belzunce, V., Gutiérrez-Valencia, M., Martínez-Velilla, N., & Ramírez-Vélez, R. (2025). System Usability Scale for gamified e-learning courses: cross-cultural adaptation and measurement properties of the Spanish version. International Journal of Human–Computer Interaction, 1-11. https://doi.org/10.1080/10447318.2025.2499947
Schrepp, M., Thomaschewski, J., & Aufderhaar, K. (2019). Do women and men perceive user experience differently? International Journal of Interactive Multimedia and Artificial Intelligence, 5, 63-67. http://dx.doi.org/10.9781/ijimai.2019.03.005
Shay, M. (2017, January 12). Move over millennials: Generation Z is the retail industry's next big buying group. Forbes. https://www.forbes.com/sites/ibm/2017/01/12/move-over-millennials-generation-z-is-the-retail-industrys-next-big-buying-group/
Shneiderman, B. (2010). Designing the user interface: strategies for effective human-computer interaction. Pearson Education India.
Slyke, C. V., Comunale, C. L., & Belanger, F. (2002). Gender differences in perceptions of web-based shopping. Communications of the ACM, 45(8), 82–86. https://doi.org/10.1145/545151.545155
Sukmasetya, P., Setiawan, A., & Arumi, E. (2020). Usability evaluation of university website: A case study. Journal of Physics: Conference Series, 1517(1), 012071. https://doi.org/10.1088/1742-6596/1517/1/012071
Şengel, E., & Toklu, E. (2019). Öğrenci seçme ve yerleştirme merkezi (ÖSYM) web sitesinin kullanılabilirliği hakkında bir durum çalışması [A Case Study on the Usability of the Student Selection and Placement Center (ÖSYM) Website]. Electronic Turkish Studies, 14(3), 1929-1943. http://dx.doi.org/10.29228/TurkishStudies.22555
Tanışık, S. (2024). Üniversite web sitelerinin kullanılabilirlik sorunları üzerine bir araştırma: Başkent Üniversitesi örneği [A study on the usability problems of university websites: The case of Başkent University]. Bilim Eğitim Sanat ve Teknoloji Dergisi, 8(1), 22-43. https://dergipark.org.tr/tr/pub/bestdergi/issue/83266/1382025
Teddlie, C., & Yu, F. (2007). Mixed Methods Sampling:A Typology With Examples. Journal of Mixed Methods Research, 1(1), 77-100. https://doi.org/10.1177/1558689806292430
Wagner, N., Hassanein, K., & Head, M. (2014). The impact of age on website usability. Computers in Human Behavior, 37, 270-282. https://doi.org/10.1016/j.chb.2014.05.003
Yesmin, S., & Atikuzzaman, M. (2023). Usability testing of a website through different devices: a task-based approach in a public university setting in Bangladesh. Information Discovery and Delivery, 52(4), 365-377. https://doi.org/10.1108/idd-01-2023-0007
Zammarchi, G., Frigau, L., & Mola, F. (2021). Markov chain to analyze web usability of a university website using eye tracking data. Statistical Analysis and Data Mining: The ASA Data Science Journal, 14(4), 331-341. https://doi.org/10.1002/sam.11512

Details

Primary Language

English

Subjects

Information Systems User Experience Design and Development

Journal Section

Research Article

Authors

Muhammed Çağrı Aksu ^*
0000-0002-8577-4413
Türkiye

Publication Date

May 1, 2026

Submission Date

October 9, 2025

Acceptance Date

February 20, 2026

Published in Issue

Year 2026 Volume: 11 Number: 2

DOI

https://doi.org/10.53850/joltida.1800362

IZ

https://izlik.org/JA54AX59ZZ

Cite

RIS / Bibtex

APA

Aksu, M. Ç. (2026). Usability Evaluation of University Websites: A Mixed-Methods Case Study from a Multi-Stakeholder Perspective. Journal of Learning and Teaching in Digital Age, 11(2), 98-116. https://doi.org/10.53850/joltida.1800362

AMA

1.Aksu MÇ. Usability Evaluation of University Websites: A Mixed-Methods Case Study from a Multi-Stakeholder Perspective. JOLTIDA. 2026;11(2):98-116. doi:10.53850/joltida.1800362

Chicago

Aksu, Muhammed Çağrı. 2026. “Usability Evaluation of University Websites: A Mixed-Methods Case Study from a Multi-Stakeholder Perspective”. Journal of Learning and Teaching in Digital Age 11 (2): 98-116. https://doi.org/10.53850/joltida.1800362.

EndNote

Aksu MÇ (May 1, 2026) Usability Evaluation of University Websites: A Mixed-Methods Case Study from a Multi-Stakeholder Perspective. Journal of Learning and Teaching in Digital Age 11 2 98–116.

IEEE

[1]M. Ç. Aksu, “Usability Evaluation of University Websites: A Mixed-Methods Case Study from a Multi-Stakeholder Perspective”, JOLTIDA, vol. 11, no. 2, pp. 98–116, May 2026, doi: 10.53850/joltida.1800362.

ISNAD

Aksu, Muhammed Çağrı. “Usability Evaluation of University Websites: A Mixed-Methods Case Study from a Multi-Stakeholder Perspective”. Journal of Learning and Teaching in Digital Age 11/2 (May 1, 2026): 98-116. https://doi.org/10.53850/joltida.1800362.

JAMA

1.Aksu MÇ. Usability Evaluation of University Websites: A Mixed-Methods Case Study from a Multi-Stakeholder Perspective. JOLTIDA. 2026;11:98–116.

MLA

Aksu, Muhammed Çağrı. “Usability Evaluation of University Websites: A Mixed-Methods Case Study from a Multi-Stakeholder Perspective”. Journal of Learning and Teaching in Digital Age, vol. 11, no. 2, May 2026, pp. 98-116, doi:10.53850/joltida.1800362.

Vancouver

1.Muhammed Çağrı Aksu. Usability Evaluation of University Websites: A Mixed-Methods Case Study from a Multi-Stakeholder Perspective. JOLTIDA. 2026 May 1;11(2):98-116. doi:10.53850/joltida.1800362