Online Academic Resources With The Focus Of Eye Behaviors

Abstract

The aim of this study is to investigate the eye movements of participants during academic searching tasks. Since the Web of Science, Google Scholar, and Council of Higher Education Thesis Center are among the popular academic search platforms in Turkey, we focused on the usability of them. During the preliminary analysis, we explored the most problematic tasks, and then analyzed the eye movements. The problematic tasks were detected with respect to the unexpected differences of completion duration; the failure of completion status; unexpected number of clicks; high number of fixations; high fixation duration; unrelated fixation areas; and think aloud data. Tobii Studio 3.4.2 was utilized for both the collection and analysis of eye-tracking data. 21 academicians participated to the study. The results of the analysis indicated that due to interface design, the eye movements of participants tended to focus on unrelated parts of the screen. As a result, certain parameters of eye tracking demonstrated how participants’ eye movements tended to generate intense fixations, increased amount spent for unrelated areas, intense mouse clicks, and increased total amount spent for tasks. In addition to those parameters, users’ satisfaction levels showed a decrease as a result of unexpected experiences.

Keywords

• Usability
• Eye Tracking
• Eye Behaviour
• Online Search
• Online Academic Sources

Project Number

PYO.EGF.1904.15.005

Çevrimiçi Akademik Kaynaklarının Kullanımında Göz Davranışlarının Değerlendirmesi

Abstract

Bu çalışmanın amacı, çevrimiçi akademik kaynak kullanımında göz hareketlerini incelemektir. Bu kapsamda ülkemizde en yaygın kullanıma sahip çevrimiçi akademik kaynaklardan olan Web of Science, Google Akademik (Google Scholar) ve YÖK Tez Merkezi web sitelerinin kullanılabilirlik testi esnasında en çok problem yaşanan görevler belirlenmiş ve kullanıcıların bu görevleri gerçekleştirirken sergilediği göz hareketleri analiz edilmiştir. En çok problem yaşanan görevler, görevlerin tamamlanması gereken süre ile tamamlanma süreleri arasındaki farkın yüksek olmasına, görevlerin tamamlanma durumlarına, fare tıklama sayılarına, sabit bakış sayılarına, sabit bakış sürelerine, sabit bakış yerlerine ve kullanıcı yorumlarına bakılarak belirlenmiştir. Bu işlemler için Tobii Studio 3.4.2 programı kullanılmıştır. Araştırma, çevrimiçi akademik kaynakları kullanma konusunda en az 2 yıl tecrübesi olan 21 akademisyen ile gerçekleştirilmiştir. Çalışmada geçerlilik için veri çeşitlemesi sağlanarak nicel ve nitel veriler birlikte kullanılmıştır. Bulgular sonucunda ara yüz tasarımındaki sıkıntılardan dolayı kullanıcıların yanlış yerlerde ve daha yoğun sabit bakış sergilediği, bu durumun da sabit bakış süreleri, fare tıklama sayıları ve görev tamamlama sürelerini olumsuz etkilediği ve memnuniyetlerini düşürdüğü belirlenmiştir. Bu problemlerin sebepleri incelenmiş ve çözüm için önerilerde bulunulmuştur.

Keywords

• Kullanılabilirlik
• Göz İzleme
• Göz Davranışları
• Çevrimiçi Arama
• Çevrimiçi Akademik Kaynaklar

Supporting Institution

Ondokuz Mayıs Üniversitesi Bilimsel Araştırmalar Proje Birimi (BAP)

Project Number

PYO.EGF.1904.15.005

References

1. Agrawal, R., & Nyamful, C. (2016). Challenges of big data storage and management. Global Journal of Information Technology, 6(1), 1-10. https://doi.org/10.18844/gjit.v6i1.383
2. Alemdag, E., & Cagiltay, K. (2018). A systematic review of eye tracking research on multimedia learning. Computers & Education, 125, 413-428. http://doi.org/10.1016/j.compedu.2018.06.023
3. Bahsisoglu, H. K. (2006). E-kaynaklar kullanımının akademik çalışmalara sağladığı kolaylıklar. Akademik Bilişim Bilgi Tek IV 9-11. http://ab.org.tr/ab06/bildiri/133.doc. Accessed 05 May 2019
4. Beymer, D., Orton, P. Z., & Russell, D. M. (2007, September). An eye tracking study of how pictures influence online reading. In IFIP Conference on Human Computer Interaction Springer Berlin Heidelberg 2007 (pp. 456-460). Springer, Berlin, Heidelberg.
5. Brandt, D. S. (1996). Evaluating information on the internet. Computers in Libraries, 16(5), 44-46.
6. Buscher, G., Dumais, S. T., & Cutrell, E. (2010, July). The good, the bad, and the random: an eye-tracking study of ad quality in web search. In 33rd international ACM SIGIR conference on Research and development in information retrieval 2010 (pp. 42-49). https://doi.org/10.1145/1835449.1835459
7. Byerly, G. (2007). Look in their eyes-eye tracking, usability, and children. School Library Media Activities Monthly, 23(8), 30-32. Clark, R. C., Nguyen, F., Sweller, J., & Baddeley, M. (2006). Efficiency in learning: Evidence‐based guidelines to manage cognitive load. Performance Improvement, 45(9), 46-47. http://doi.org/10.1002/pfi.4930450920
8. Cutrell, E., & Guan, Z. (2007) What are you looking for?: an eye-tracking study of information usage in web search. In: SIGCHI conference on Human factors in computing systems (pp. 407-416). https://doi.org/10.1145/1240624.1240690

9. Çagiltay, K. (2011). İnsan bilgisayar etkileşimi ve kullanılabilirlik mühendisliği: Teoriden pratiğe. ODTÜ Yayıncılık, Ankara
10. Cetin, I., & Sendurur, E. (2016). Çevrimiçi akademik kaynakların kullanılabilirlik değerlendirmesi. Mehmet Akif Ersoy Üniversitesi Eğitim Fakültesi Dergisi, 1(40), 273-299. http://doi.org/10.21764/efd.04062
11. Dumas, J. S., & Redish, J. (1999). A practical guide to usability testing. Intellect Books.
12. Hertzum, M., Hansen, K. D., & Andersen, H. H. (2009). Scrutinising usability evaluation:does thinking aloud affect behaviour and mental workload?. Behaviour & Information Technology, 28(2), 165-181. https://doi.org/10.1080/01449290701773842
13. ISO 9241-11 (2018). Ergonomic requirements for office work with visual display terminals (vdts) part 11: Guidance on usability.
14. International Organization for Standardization, Geneva. https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en. Accessed 15 March 2019
15. ISO/IEC 25010 (2011). Systems and software engineering – systems and software quality requirements and evaluation – system and software quality models. International Organization for Standardization, Geneva. https://www.iso.org/obp/ui/#iso:std:iso-iec:25010:ed-1:v1:en. Accessed 15 March 2019
16. Kilic, E., & Karadeniz, S. (2004). Hiper ortamlarda öğrencilerin bilişsel yüklenme ve kaybolma düzeylerinin belirlenmesi. Kuram ve Uygulamada Egitim Yönetimi Dergisi, 10(4), 562-579.
17. Kovacs, D. K., & Elkordy, A. (2000). Collection development in cyberspace: building an electronic library collection. Library Hi Tech, 18(4), 335-361. https://doi.org/10.1108/07378830010360446
18. Land, S. M. (2000). Cognitive requirements for learning with open-ended learning environments. Educational Technology Research and Development, 48(3), 61-78. https://doi.org/10.1007/BF02319858
19. Lewandowski, D., Kerkmann, F., Rümmele, S., & Sünkler, S. (2018). An empirical investigation on search engine ad disclosure. Journal of the Association for Information Science and Technology, 69(3), 420-437. http://doi.org/10.1002/asi.23963
20. Lindgaard, G., & Chattratichart, J. (2007). Usability testing: what have we overlooked?. In Proceedings of the SIGCHI Conference on Human factors in Computing Systems (pp. 1415-1424). ACM. https://doi.org/10.1145/1240624.1240839
21. Lund, H. (2016). Eye tracking in library and information science: A literature review. Library Hi Tech, 34(4), 585–614. http://doi.org/10.1108/LHT-07-2016-0085
22. Murray, T. (2001, October). Characteristics and Affordances of Adaptive Hyperbooks. In WebNet (pp. 899-904).
23. Myers, B. A., & Rosson, M. B. (1992, May). Survey on user interface programming. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 195-202). ACM.
24. Nakatani, C., & Pollatsek, A. (2004). An eye movement analysis of “mental rotation” of simple scenes. Perception & psychophysics, 66(7), 1227-1245. https://doi.org/10.3758/BF03196848
25. Nielsen, J. (1994). Heuristic evaluation. In Nielsen, J., and Mack, R.L. (Eds.), Usability Inspection Methods, (413-414). John Wiley & Sons, New York, NY.
26. Nielsen, J. (2012). Usability 101: Introduction to usability. https://www.nngroup.com/articles/usability-101-introduction-to-usability/. Accessed 21 January 2019
27. Nielsen, J., & Landauer, T. K. (1993). A mathematical model of the finding of usability problems. In Proceedings of the INTERACT'93 and CHI'93 conference on Human factors in computing systems (pp. 206-213). ACM.
28. Norman, D. A. (1988). Designing everyday things. Currency-Doubleday, New York.
29. Pan, B., Hembrooke, H. A., Gay, G. K., Granka, L. A., Feusner, M. K., & Newman, J. K. (2004, March). The determinants of web page viewing behavior: an eye tracking study. In Proceedings of the 2004 symposium on Eye tracking research & applications (pp. 147-154). ACM.
30. Pernice, K., & Nielsen, J. (2009). Eyetracking methodology: How to conduct and evaluate usability studies using eyetracking. Berkeley: New Riders Press.
31. Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological bulletin, 124(3), 371-422. https://doi.org/10.1037/0033-2909.124.3.372
32. Renshaw, J. A., & Webb, N. (2007). Eye tracking in practice. In 21st British HCI Group Annual Conference University of Lancaster, UK (pp. 239-241). http://doi.org/10.14236/ewic/HCI2007.41
33. Scheiter, K., & Van Gog, T. (2009). Using eye tracking in applied research to study and stimulate the processing of information from multi-representational sources. Applied Cognitive Psychology, 23(9), 1209–1214. http://doi.org/10.1002/acp.1524
34. Shackel, B. (2009). Usability–Context, framework, definition, design and evaluation. Interacting with Computers, 21(5-6), 339-346.
35. Tashakkori, A., & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative approaches. Sage.
36. Tobii Studio (2008) Tobii Studio 1.2 User Manual. https://manualzz.com/doc/4199338/tobii-studio-1.2-user-manual. Accessed 16 September 2018
37. Virzi, R. A. (1992). Refining the test phase of usability evaluation: How many subjects is enough?. Human Factors: The Journal of the Human Factors and Ergonomics Society, 34(4), 457-468. https://doi.org/10.1177/001872089203400407
38. White, R. W. (2016). Interactions with Search Systems. New York: Cambridge University Press.
39. Yavuz, I., Cinar, N. O., & Cagiltay, K. (2016). Kamu internet sitelerinde yer alan arama alanlarının kullanılabilirliği ve buna yönelik kullanıcı davranışlarının belirlenmesi. International Journal of Informatics Technologies, 9(1), 41. http://doi.org/ 10.17671/btd.81715