Comparison of Three Travel Websites According to Cognitive Load and Usability Metrics Büşra Nur Yetkin1*, N. Fırat Özkan2 , Merve Sert3 and

Abstract

It is crucial that a website reflects a quality user interface (UI) that brings out the holiday enthusiasm for users, while still remaining user friendly and a simple area that is easily accessible. To achieve this, it is important that user experience designers measure the efficiency of the interface interaction between human and machine and ensure continuous interface development. In order to test interfaces and find differences, 3 different travel web sites was selected with experiments performed with 38 participants. Tested travel website interfaces are Trivago, ETS Tour and Odamax. Participants performed 6 different tasks during the experiment and after these tasks, a think aloud protocol was implemented and NASA-Task Load Index (NASA-TLX), IBM Computer System Usability Questionnaire (IBM CSUQ) questionnaires were applied. In order to analyze the differences between interfaces, task completion time, number of errors, success rate, IBM CSUQ questionnaire average and NASA TLX questionnaire average were collected as data. ANOVA analysis was performed between interfaces using IBM SPSS Statistics 22 program and Independent T test was used to test usability by gender. According to ANOVA results, task completion time, number of errors, success rate, IBM CSUQ survey score average and NASA TLX survey score average change significantly as interface changes. Respectively, Trivago, Odamax, ETS Tour has the highest cognitive load in the measurements. According to the results of independent T tests, there is no difference between men and women in terms of interface usability.

Keywords

• user experience
• user testing
• anova
• interface
• nasa-tlx
• ibm csuq

References

1. [1] Hamurcu, A. “The Role of Industrial Designers in the Birth and Development of User Experience Design in Turkey.” National Design Research Conference, pp. 333-347, 2014, Orta Doğu Teknik Üniversitesi.
2. [2] Ulusam Seçkiner, S., and Toraman, N. “A new model for determining cognitive workload. " Journal of Engineering Sciences and Design, 5(1), pp 365-381, 2017.
3. [3] Tracy, J. P. and Albers, M. J. “Measuring cognitive load to test the usability of web sites.” In Annual Conference-society for technical communication, 53, p. 256, 2006.
4. [4] Hart, S. “Nasa-Task Load Index (NASA-TLX); 20 Years Later.” Proceedings Of The Human Factors And Ergonomics Society Annual Meeting, 50(9), pp 904-908, 2006.
5. [5] Conway, D., Dick, I., Li, Z., Wang, Y. and Chen, F. “The Effect of Stress on Cognitive Load Measurement.” In Human-Computer Interaction–INTERACT 2013 pp. 659-666. Springer Berlin Heidelberg, 2013.
6. [6] Rizzo, L., Delany, S., Dondio, P., and Longo, L. “Modeling Mental Workload via Rule-based Expert System: a Modeling Mental Workload via Rule-based Expert System: a Comparison with NASA-TLX & Workload Profile.” Dublin: ARROW TU DUBLIN, 2016.
7. [7] Şeker, A. “Using outputs of NASA_TLX for Building a Mental Workload Expert System.” Gazi University Journal Of Science, 27(4), pp. 1131-1142, 2014.
8. [8] Delice E. “Assessment of mental workloads of emergency physicians using nasa-rtlx method: an application study.” Atatürk University Journal of Economic and Administrative Sciences, 30(3), pp. 645-662, 2016.

9. [9] Flemisch, F., and Onken, R. “Open a Window to the Cognitive Work Process! Pointillist Analysis of Man-Machine Interaction.” Cognition, Technology & Work, 4(3), pp. 160-170, 2002.
10. [10] Wanyan, X., Zhuang, D., and Zhang, H. “Improving pilot mental workload evaluation with combined measures. Bio-Medical Materials And Engineering", 24, pp. 2283-2290, 2014.
11. [11] Střelák, D., Škola, F., and Liarokapis, F. “ Examining User Experiences in a Mobile Augmented Reality Tourist Guide.” Proceedings Of The 9Th ACM International Conference On Pervasive Technologies Related To Assistive Environments, 1-8, 2016.
12. [12] Beck, E., Christiansen, M., Kjeldskov, J., Kolbe, N., and Stage, J. “Experimental Evaluation of Techniques for Usability Testing of Mobile Systems in a Laboratory Setting.” In Proceedings of OzCHI, Brisbane, Australia CHISIG, 2003.
13. [13] Carstens D.S, and Patterson P. “Usability study of travel websites.” J. Usability Studies 1, 1, pp. 47–61, 2007.
14. [14] Groth A., Haslwanter D. “Efficiency, effectiveness, and satisfaction of responsive mobile tourism websites: a mobile usability study.” Information Technology & Tourism 16(2): pp. 201-228, 2016.
15. [15] Rusli, N., Hassan, S., and Liau, N. “Usability Analysis of Students Information System in a Public University.” Journal Of Emerging Trends In Engineering And Applied Sciences, 4(6), pp. 806-810, 2013.
16. [16] Nielson, J. “Usability Engineering.” 1993.