MIXED-METHODS EVALUATION OF THE USABILITY OF A STUDENT INFORMATION SYSTEM-INTEGRATED EXAM ASSESSMENT AND VERIFICATION SYSTEM
Öz
Exam assessment processes in higher education require usable systems that support not only scoring but also verification, record integrity, and efficient workflow management. This study evaluated the usability of a Student Information System (SIS)-integrated exam assessment and verification system using a mixed-methods approach. Twenty participants with different levels of prior system experience completed a task-based usability test, the Turkish version of the System Usability Scale (SUS-TR), and semi-structured interviews analyzed through reflexive thematic analysis. Findings were integrated through a joint display. The mean SUS score was 86.88 and the mean task success score was 93.50. However, filtering records by academic unit had one of the lowest success rates (85%), the highest number of errors (12), and the highest mean perceived difficulty (2.25). Perceived task difficulty was negatively associated with SUS score (ρ = −.592, p = .006). Qualitative findings highlighted SIS-based verification, integrated access to student information, duplicate-record identification, and digital optical-form access as valued features, while clearer error communication, process-step guidance, interface labeling, and first-use support emerged as improvement needs. Overall, high usability coexisted with localized workflow difficulties.
Anahtar Kelimeler
Usability, Student Information System, Exam Assessment System, Optical Form, Mixed Methods, Thematic Analysis
Destekleyen Kurum
This research was supported by the Scientific Research Projects Coordination Unit of Artvin Çoruh University. Project No: 2025.F13.02.03.
Etik Beyan
The study was conducted with the approval of the Artvin Çoruh University Ethics Committee (E-18457941-050.99-216576). Participants voluntarily took part in the study, and all data collection processes were carried out in accordance with principles of confidentiality and anonymity.
Kaynakça
- Akcakaya, A., Diri, S., & Şahin, S. (2023). Kocaeli Üniversitesi Öğrenci Bilgi Sistemi Mobil Uygulamasının Kullanılabilirliği [Usability of Kocaeli University Student Information System Mobile Application]. Yalvaç Akademi Dergisi, 8(1), 65–74. https://doi.org/10.57120/yalvac.1257634
- Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
- Braun, V., & Clarke, V. (2021). One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qualitative Research in Psychology, 18(3), 328–352. https://doi.org/10.1080/14780887.2020.1769238
- de Elias, E. M., Tasinaffo, P. M., & Hirata, R., Jr. (2021). Optical mark recognition: Advances, difficulties, and limitations. SN Computer Science, 2, 367. https://doi.org/10.1007/s42979-021-00760-z
- Demirkol, D., & Şeneler, Ç. (2018). A Turkish translation of the System Usability Scale: The SUS-TR [Sistem Kullanılabilirlik Ölçeğinin Türkçeye Çevirisi: SUS-TR]. Uşak Üniversitesi Sosyal Bilimler Dergisi, 11(3), 237–253. https://dergipark.org.tr/tr/pub/usaksosbil/article/480920
- Demirkol, D., & Seneler, C. (2019). Evaluation of Student Information System (SIS) in terms of user emotion, performance and perceived usability: A Turkish university case (An empirical study). Procedia Computer Science, 158, 1033–1051. https://doi.org/10.1016/j.procs.2019.09.145
- Demirkol, D., Seneler, C., Daim, T., & Shaygan, A. (2020). Measuring perceived usability of university students towards a student information system (SIS): A Turkish university case. Technology in Society, 62, 101281. https://doi.org/10.1016/j.techsoc.2020.101281
- Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013). Achieving integration in mixed methods designs—Principles and practices. Health Services Research, 48(6 Pt 2), 2134–2156. https://doi.org/10.1111/1475-6773.12117
- Guetterman, T. C., Fetters, M. D., & Creswell, J. W. (2015). Integrating quantitative and qualitative results in health science mixed methods research through joint displays. The Annals of Family Medicine, 13(6), 554–561. https://doi.org/10.1370/afm.1865
- Hertzum, M. (2026). System Usability Scale: A meta-analysis of how SUS relates to workload, task time, and error rate. International Journal of Human–Computer Interaction. Advance online publication. https://doi.org/10.1080/10447318.2026.2625260
