Comparing Glasses-Based and Remote Eye Tracking for Mobile Banking Usability Assessment

Abstract

Mobile banking applications, one of the digital marketing tools, enable users to perform banking transactions quickly and easily. The way to create usable mobile applications is through detailed user experience tests. In user experience tests, eye tracking technique, which can provide reliable and detailed results, stands out. The purpose of this study is to compare the metrics and data outputs of two eye tracking techniques used to measure user experience: glasses-based (TobiiPro) and remote (Realeye) with a webcam. Eleven participants with mobile banking application experience were asked to perform two different tasks. During the procedure, in addition to the eye tracking technique, the system usability scale (SUS) was used as a method, and user opinions were also utilized as part of the think aloud protocol. As the output of the task-based procedure, findings on the mobile application experience were obtained, while the results of the data collected from different eye tracking techniques were also obtained comparatively. While close values were obtained in the average fixation time, it was found that the fixation amounts differed. Due to the differences in the findings between the techniques, it is recommended that further research should be standardized refresh rates across both eye-tracking techniques and test a larger sample to validate these findings.

Keywords

• Digital Marketing
• Eye Tracking
• User Experience
• Mobile Banking Application

Mobil Bankacılık Kullanılabilirlik Değerlendirmesinde Gözlük Tabanlı ve Uzaktan Göz Takibinin Karşılaştırılması

Abstract

Dijital pazarlama araçlarından biri olan mobil bankacılık uygulamaları, kullanıcıların bankacılık işlemlerini hızlı ve kolay bir şekilde yapmasına olanak sağlamaktadır. Kullanılabilir mobil uygulamalar yapmanın yolu detaylı kullanıcı deneyimi testlerinden geçmektedir. Kullanıcı deneyimi testlerinde güvenilir ve detaylı sonuç alınabilen göz takibi tekniği öne çıkmaktadır. Bu araştırmanın amacı, kullanıcı deneyiminin ölçülmesinde kullanılan, gözlük bazlı (TobiiPro) ve web kamerası ile uzaktan (Realeye) olmak üzere, iki göz takibi tekniğinin metriklerinin ve veri çıktılarının karşılaştırılmasıdır. Mobil bankacılık uygulaması deneyimi olan 11 katılımcıdan iki farklı görevi yerine getirmeleri istenmiştir. Prosedür esnasında yöntem olarak göz takibi tekniğinin yanı sıra, sistem kullanılabilirlik ölçeği (SUS) kullanılmış ve yüksek sesle düşünme protokolü kapsamında kullanıcı görüşlerinden de yararlanılmıştır. Görev temelli prosedür çıktısı olarak, mobil uygulama deneyimine yönelik bulgulara ulaşılırken aynı zamanda farklı göz takibi tekniklerinden toplanan verilerin sonuçları da karşılaştırmalı olarak elde edilmiştir. Ortalama odaklanma süresinde yakın değerler elde edilirken odaklanma miktarlarının farklılık gösterdiği bulgulanmıştır. Teknikler arasında meydana gelen bulgu farklılıklarından dolayı, her iki göz izleme tekniğinde de yenileme hızlarının standartlaştırılması ve daha büyük bir örneklem üzerinde test yapılması önerilmektedir.

Keywords

• Dijital Pazarlama
• Göz Takibi
• Kullanıcı Deneyimi
• Mobil Bankacılık Uygulaması

References

1. Abubakar, H. I., Hashim, N. L., and Hussain, A. (2016). Usability evaluation model for mobile banking applications interface: Model evaluation process using experts’ panel. Journal of Telecommunication, Electronic and Computer Engineering, 8(10), 53-57.
2. Al Amin, M., Sultana, N., Saha, T., Islam, S. M., and Kashem, M. A. (2021). Customer's attitude toward mobile banking usage: A case study in Bangladesh. The Journal of Asian Finance, Economics and Business, 8(2), 419-426.
3. Alhejji, S., Albesher, A., Wahsheh, H., and Albarrak, A. (2022). Evaluating and comparing the usability of mobile banking applications in Saudi Arabia. Information, 13(12), 559.
4. Başar, M. S., and Sinan, K. U. L. (2020). Evaluation of mobile banking apps in terms of usability with analytical hierarchy process. Bilgi Yönetimi, 3(1), 39-49.
5. Bevan, N. (2009, August). What is the difference between the purpose of usability and user experience evaluation methods. In Proceedings of the Workshop UXEM (Vol. 9, No. 1, pp. 1-4).
6. Blair-Early, A., and Zender, M. (2008). User interface design principles for interaction design. Design Issues, 24(3), 85-107.
7. Burton, L., Albert, W., and Flynn, M. (2014, September). A comparison of the performance of webcam vs. infrared eye tracking technology. In Proceedings of the human factors and ergonomics society annual meeting (Vol. 58, No. 1, pp. 1437-1441). Sage CA: Los Angeles, CA: SAGE Publications.
8. Cameron, G., Cameron, D., Megaw, G., Bond, R., Mulvenna, M., O’Neill, S., ... and McTear, M. (2019). Assessing the usability of a chatbot for mental health care. In Internet science: INSCI 2018 international workshops, st. Petersburg, Russia, october 24–26, 2018, revised selected papers 5 (pp. 121-132). Springer International Publishing.

9. Cellary, A. (2023). Understanding eye-tracking technology: A beginner’s guide. Retrieved from https://www.realeye.io/blog/post/understanding-eye-tracking-technology-a-beginners-guide.
10. Chandran, R. (2014). Pros and cons of mobile banking. International Journal of Scientific and Research Publications, 4(10), 1-5.
11. Chauhan, S., Akhtar, A., and Gupta, A. (2022). Customer experience in digital banking: a review and future research directions. International Journal of Quality and Service Sciences, 14(2), 311-348.
12. Djamasbi, S. (2014). Eye tracking and web experience. AIS Transactions on Human-Computer Interaction, 6(2), 37-54.
13. Dos Santos, R. D. O. J., de Oliveira, J. H. C., Rocha, J. B., and Giraldi, J. D. M. E. (2015). Eye tracking in neuromarketing: A research agenda for marketing studies. International Journal of Psychological Studies, 7(1), 32.
14. Dospinescu, O., and Percă-Robu, A. E. (2017). The analysis of e-commerce sites with eye-tracking technologies. BRAIN. Broad Research in Artificial Intelligence and Neuroscience, 8(3), 85-100.
15. Duchowski, A. T., and Duchowski, A. T. (2017). Eye tracking methodology: Theory and practice. Springer. Ehinger, B. V., Groß, K., Ibs, I., and König, P. (2019). A new comprehensive eye-tracking test battery concurrently evaluating the Pupil Labs glasses and the EyeLink 1000. PeerJ, 7, e7086.
16. Gatsou, C. (2015). The importance of metaphors for user interaction with mobile devices. In Design, User Experience, and Usability: Users and Interactions: 4th International Conference, DUXU 2015, Held as Part of HCI International 2015, Los Angeles, CA, USA, August 2-7, 2015, Proceedings, Part II 4 (pp. 520-529). Springer International Publishing.
17. Hamid, K., Iqbal, M. W., Muhammad, H. A. B., Fuzail, Z., Ghafoor, Z. T., and Ahmad, S. (2022). Usability evaluation of mobile banking applications in digital business as emerging economy. International Journal of Computer Science and Network Security, 22(1), 250-260.
18. Harezlak, K., Kasprowski, P., and Stasch, M. (2014). Towards accurate eye tracker calibration–methods and procedures. Procedia Computer Science, 35, 1073-1081.
19. Harrison, R., Flood, D., and Duce, D. (2013). Usability of mobile applications: Literature review and rationale for a new usability model. Journal of Interaction Science, 1, 1-16.
20. Hoehle, H., and Venkatesh, V. (2015). Mobile application usability. MIS Quarterly, 39(2), 435-472.
21. Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., and Van de Weijer, J. (2011). Eye tracking: A comprehensive guide to methods and measures. oup Oxford.
22. Hussain, A., Abubakar, H. I., and Hashim, N. B. (2014, November). Evaluating mobile banking application: Usability dimensions and measurements. In Proceedings of the 6th international Conference on Information Technology and Multimedia (pp. 136-140). IEEE.
23. Hyökki, S. (2012). Eye tracking in user research. Interdisciplinary Studies Journal, 1(4), 65-77.
24. Hyzy, M., Bond, R., Mulvenna, M., Bai, L., Dix, A., Leigh, S., and Hunt, S. (2022). System usability scale benchmarking for digital health apps: Meta-analysis. JMIR mHealth and uHealth, 10(8), e37290.
25. ISO 9241−11. (1998). Retrieved from https://csrc.nist.gov/glossary/term/usability#:~:text=Definitions%3A,a%20specified%20context%20of%20use.
26. Jun, M., and Palacios, S. (2016). Examining the key dimensions of mobile banking service quality: An exploratory study. International Journal of Bank Marketing, 34(3), 307-326.
27. Malik, H. A. M., Muhammad, A., and Sajid, U. (2021). Analyzing usability of mobile banking applications in Pakistan. Sukkur IBA Journal of Computing and Mathematical Sciences, 5(2), 25-35.
28. Marcus, A. (1998). Metaphor design in user Interfaces. ACM SIGDOC Asterisk Journal of Computer Documentation, 22(2), 43-57.
29. Nagaraju, S. (2015). Mobile banking. Trends, Challenges and Innovations in Management-Volume III, 96.
30. Nielsen, J. (2000). Why you only need to test with 5 user. Retrieved from https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/.
31. Nielsen, J. (2012). How Many Test Users in A Usability Study? Retrieved from https://www.nngroup.com/articles/how-many-test-users.
32. Norman, D. A. (2005). Human-Centered Design Considered Harmful. Interactions, 12(4), 14-19.
33. Pappas, I., Sharma, K., Mikalef, P., and Giannakos, M. (2018). Visual aesthetics of e-commerce websites: An eye-tracking approach. In Proceedings of the 51st Hawaii International Conference on System Sciences.
34. Pradini, R. S., Kriswibowo, R., and Ramdani, F. (2019, September). Usability evaluation on the SIPR website uses the system usability scale and net promoter score. In 2019 International Conference on Sustainable Information Engineering and Technology (SIET) (pp. 280-284). IEEE.
35. Qu, Q. X., Zhang, L., Chao, W. Y., and Duffy, V. (2017). user experience design based on eye-tracking technology: A case study on smartphone apps. In Advances in Applied Digital Human Modeling and Simulation: Proceedings of the AHFE 2016 International Conference on Digital Human Modeling and Simulation, July 27-31, 2016, Walt Disney World®, Florida, USA (pp. 303-315). Springer International Publishing.
36. Schiessl, M., Duda, S., Thölke, A., and Fischer, R. (2003). Eye tracking and its application in usability and media research. MMI-Interaktiv Journal, 6(1), 41-50.
37. Shackel, B. (2009). Usability–Context, framework, definition, design and evaluation. Interacting with computers, 21(5-6), 339-346.
38. Statista. (2023). The leaders and laggards of mobile banking. Retrieved from https://www.statista.com/chart/27528/share-of-mobile-bankers-by-country/.
39. TBB. (2024). Dijital, internet ve mobil bankacılık raporu, Retrieved from https://www.tbb.org.tr/istatistiki-raporlar/2024-eylul-dijital-internet-ve-mobil-bankacilik-istatistikleri.
40. Țichindelean, M., Țichindelean, M. T., Cetină, I., and Orzan, G. (2021). A comparative eye tracking study of usability—towards sustainable web design. Sustainability, 13(18), 10415. https://doi.org/10.3390/su131810415
41. Tobii. (2024). Understanding tobii pro lab's eye tracking metrics. Retrieved from https://connect.tobii.com/s/article/understanding-tobii-pro-lab-eye-tracking-metrics?language=en_US.
42. Todua, N., and Gogitidze, N. (2021). Features of the use of digital marketing in the banking sector. The New Economist, 16(3/4), 7-15.
43. Wang, Q., Yang, S., Liu, M., Cao, Z., and Ma, Q. (2014). An eye-tracking study of website complexity from cognitive load perspective. Decision Support Systems, 62, 1-10.
44. Wang, Y. J., and Minor, M. S. (2008). Validity, reliability, and applicability of psychophysiological techniques in marketing research. Psychology and marketing, 25(2), 197-232.
45. Windasari, N. A., Kusumawati, N., Larasati, N., and Amelia, R. P. (2022). Digital-only banking experience: Insights from gen Y and gen Z. Journal of Innovation and Knowledge, 7(2), 100170.