Investigation of the Relationship between Visual Measurements and Test and Item Statistics

Year 2025, Issue: 65, 45 - 64, 19.09.2025

Tolga Coşguner , Burcu Atar

https://doi.org/10.9779/pauefd.1453691

Abstract

The purpose of this study was to investigate the correlations between time-and count-oriented visual data collected from a wearable eye tracker and test and item statistics. To accomplish this, a wearable eye tracker was used with an achievement test that included 30 multiple-choice questions assessing eighth-grade achievements, administered to 60 high school students. The acquired data were then used to compute t-values, Pearson correlation coefficients, and regression coefficients. The results of the study demonstrated a substantial inverse relationship between time and count-oriented visual assessments and achievement scores. Furthermore, time and count-oriented measures were shown to be significant predictors of achievement test results. Additionally, substantial disparities in times to first fixation and total durations of visit were identified in favor of females, whereas significant differences in average durations of visit were discovered in favor of males. The results of this study show that there is a substantial association between time-oriented measures and test and item statistics.

Keywords

Eye-tracking method , Test statistics , Item statistics , Cognitive processes

Ethical Statement

This research was conducted with the permission obtained by the Pamukkale University Scientific Research and Publication Ethics Social and Human Sciences Board's decision dated 25/02/2021 and numbered 68282350/2018/G04.

References

• Ackerman, T. A. (1992). Assessing construct validity using multidimensional item response theory. Paper Presented at the Annual Meeting of American Educational Research Association. San Fransisco, CA, USA.
• Akçay, A., & Altun, A. (2019). Farklı kısa süreli bellek uzamlarına sahip öğrencilerin farklı dikkat tasarımına sahip öğrenme ortamlarındaki göz hareketlerinin incelenmesi. Eğitim Teknolojisi Kuram ve Uygulama, 9(2), 588-614.
• Antle, A. N. (2013). Research opportunities: Embodied child-computer interaction. International Journal of Child-Computer Interaction, 1(1), 30-36.
• Ariasi, N., & Mason, L. (2011). Uncovering the effect of text structure in learning from a science text: An eye-tracking study. Instructional Science, 39(5), 581-601.
• Bayazıt, A. (2013). Farklı soru biçimlerinin göz hareketleri, başarım ve cevaplama süresine olan etkilerinin incelenmesi [Doktora tezi, Hacettepe Üniversitesi]. Ulusal Tez Merkezi.
• Becker, W. J., & Menges, J. I. (2013). Biological implicit measures in HRM and OB: A question of how not if. Human Resource Management Review, 23, 219-228.
• Berzak, Y., Katz, B., & Levy, R. (2018). Assessing language proficiency from eye movements in reading. In M. A. Walker, H. Ji, & A. Stent (Ed.), Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 1986-1996). Association for Computational Linguistics.
• Bolt, D. M. (2002). A Monte Carlo comparison of parametric and nonparametric polytomous DIF detection methods. Applied Measurement in Education, 15(2), 113-141.
• Büyüköztürk, Ş. (2020). Sosyal bilimler için veri analizi el kitabı. Pegem Akademi.
• Clinton, V., Cooper, J. L., Michaelis, J. E., Alibali, M. W., & Nathan, M. J. (2017). How revisions to mathematical visuals affect cognition: Evidence from eye tracking. In C. Was, F. Sansosti, & B. Morris (Ed.), Eye-tracking technology applications in educational research (pp. 195-218). IGI Global.
• Conati, C., & Merten, C. (2007). Eye-tracking for user modeling in exploratory learning environments: An empirical evaluation. Knowledge-Based Systems, 20(6), 557-574.
• Demirus, K. B., & Gelbal, S. (2016). Ortak maddelerin değişen madde fonksiyonu gösterip göstermemesi durumunda test eşitlemeye etkisinin farklı yöntemlerle incelenmesi. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 7(1),182-201.
• Duchowski, A. T. (2002). A breadth-first survey of eye-tracking applications. Behavior Research Methods, Instruments, & Computers, 34(4), 455-470.
• Duchowski, A. (2007). Eye tracking methodology: Theory and practice. Springer-Verlag.
• Durna, Y., & Arı, F. (2016b). Polinom fonksiyonları ile göz B-bakış yeri tespiti geliştirilmesi ve uygulaması. Savunma Bilimleri Dergisi, 15 (2), 24-45.
• Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates, Publishers.
• Erkuş, A. (2014). Psikolojide ölçme ve ölçek geliştirme-I: Temel kavramlar ve işlemler. Pegem Akademi.
• Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education. McGraw-Hill.
• Galván, A. (2010). Neural plasticity of development and learning. Human Brain Mapping, 31, 879-890.
• Glaholt, M. G., Wu, M. C., & Reingold, E. M. (2010). Evidence for top-down control of eye movements during visual decision making. Journal of Vision, 10(5), 1-10.
• Goldberg, J., & Helfman, J. (2011). Eye tracking for visualization evaluation: Reading values on linear versus radial graphs. Journal of Information Visualization, 10(3), 182-195.
• Goldberg, J. H., & Kotval, X. P. (1999). Computer interface evaluation using eye movements: methods and constructs. International Journal of Industrial Ergonomics, 24(6), 631-645.
• Jacob, R. J., & Karn, K. S. (2003). Eye tracking in human-computer interaction and usability research: Ready to deliver the promises. In Radach, R., Hyona, J., & Deube, H. (Eds.), The mind’s eye (pp.573-605). Elsevier.
• Jarodzka, H., Janssen, N., Kirschner, P. A., & Erkens, G. (2015). Avoiding split attention in computer-based testing: Is neglecting additional information facilitative? British Journal of Educational Technology, 46(4), 803-817.
• Just, M. A., & Carpenter, P. A. (1976a). Eye fixations and cognitive processes. Cognitive Psychology, 8(4), 441-480.
• Just, M. A., Carpenter, P. A. (1976b). The role of eye-fixation research in cognitive psychology. Behavior Research Methods & Instrumentation, 8(2), 139-143.
• Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87(4), 329-354.
• Khamis, H., & Kepler, M. (2010). Sample size in multiple regression: 20 + 5K. Journal of Applied Statistical Science, 17(4), 505-517. Retrieved from https://www.researchgate.net/publication/285744052_Sample_size_in_multiple_regression_20_5k
• Kirsh, D. (2013). Embodied cognition and the magical future of interaction design. ACM Transactions on Computer-Human Interaction, 20(1), 1-30.
• Klein, P., Lichtenberger, A., Küchemann, S., Becker, S., Kekule, M., Viiri, J., Baadte, C., Vaterlaus, A., & Kuhn, J. (2020). Visual attention while solving the test of understanding graphs in kinematics: An eyetracking analysis. European Journal of Physics, 41, 1-16.
• Korkmaz, L. (2017). Tutumlarımızın ne kadar farkındayız? Örtük tutumlar ve örtük ölçüm yöntemleri. Türk Psikoloji Yazıları, 20(40), 109-127.
• Lai, M. L., Tsai, M. J., Yang, F. Y., Hsu, C. Y., Liu, T. C., Lee, S. W. Y., Lee, M. H., Chiou, G. L., Liang, J. C., & Tsai, C. C. (2013). A review of using eye-tracking technology in exploring learning from 2000 to 2012. Educational Research Review, 10, 90-115.
• Lindner, M. A., Eitel, A., Strobel, B., & Köller, O. (2017). Identifying processes underlying the multimedia effect in testing: An eye-movement analysis. Learning and Instruction, 47, 91-102.
• Lindner, M. A., Eitel, A., Thoma, G. B., Dalehefte, I. M., Ihme, J. M. & Köller, O. (2014). Tracking the decision-making process in multiple-choice assessment: Evidence from eye movements. Applied Cognitive Psychology, 28(5), 738-752.
• Liu, H. C., Lai, M. L., & Chuang, H. H. (2011). Using eye-tracking technology to investigate the redundant effect of multimedia web pages on viewers' cognitive processes. Computers in Human Behavior, 27, 2410-2417.
• Liu, P. L. (2014). Using eye tracking to understand learners’ reading process through the concept mapping learning strategy. Computers & Education, 78, 237-249.
• Liu, W., Yu, M., Fan, Z., Xu, J., & Tian, Y. (2017, Oct.). Visual attention-based evaluation for multiple-choice tests in e-learning applications. 2017 IEEE Frontiers in Education Conference (FIE). Indianapolis, IN, USA.
• Liversedge, S. P., Paterson, K. B., & Pickering, M. J. (1998). Eye movements and measures of reading time. In Underwood, G. (Eds.), Eye guidance in reading and scene perception (pp. 55-75). Elsevier. Liversedge, S. P., & Findlay, J. M. (2000). Saccadic eye movements and cognition. Trends in Cognitive Sciences, 4(1), 6-14.
• Majooni, A., Masood, M., & Akhavan, A. (2016). An eye-tracking experiment on strategies to minimize the redundancy and split attention effects in scientific graphs and diagrams. In Di Bucchianico, G., & Kercher, P. (Eds.), Advances in Design for Inclusion: Proceedings of the AHFE 2016 (pp. 529-540). Springer International Publishing.
• Mele, M. L., & Federici, S. (2012). Gaze and eye-tracking solutions for psychological research. Cognitive Processing, 13(1), 261-265.
• Mitra, R., McNeal, K. S., & Bondell, H. D. (2017). Pupillary response to complex interdependent tasks: A cognitive-load theory perspective. Behavior Research Methods, 49, 1905-1919.
• Negi, S., & Mitra, R. (2020). Fixation duration and the learning process: An eye tracking study with subtitled videos. Journal of Eye Movement Research, 13(6), 1-15.
• Nugrahaningsih, N., Porta, M., & Ricotti, S. (2013, Oct.). Gaze behavior analysis in multiple-answer tests: An eye tracking investigation. 12th International Conference on Information Technology Based Higher Education and Training (ITHET). Antalya, Turkey.
• Ömur, S., & Görgülü Aydoğdu, A. (2017). Göz izleme araştırmaları ve iletişim alanında yeni yönelimler. International Journal of Social Sciences and Education Research, 3(4), 1296-1307.
• Peterson, S. J., Reina, C. S., Waldman, D. A., & Becker, W. J. (2015). Using physiological methods to study emotions in organizations. In Härtel, C. E. J., Zerbe, W. J., & Ashkanasy, N. M. (Eds.), New Ways of Studying Emotions in Organizations (pp. 1-27). Emerald Group Publishing Limited.
• Poole, A., Ball, L. J., & Phillips, P. (2005). In search of salience: A response‐time and eye‐movement analysis of bookmark recognition. In Fincher, S., Markopoulos, P., Moore, D., & Ruddle, R. (Eds.), People and Computers XVIII—Design for Life: Proceedings of HCI 2004 (pp. 363-378). Springer-Verlag.
• Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124(3), 372-422.
• Rayner, K. (2009). The 35th Sir Frederick Bartlett Lecture: Eye movements and attention in reading, scene perception, and visual search. The Quarterly Journal of Experimental Psychology, 62(8), 1457-1506.
• Rayner, K., Chace, K. H., Slattery, T. J., & Ashby, J. (2006). Eye movements as reflections of comprehension processes in reading. Scientific Studies of Reading, 10(3), 241-255.
• Rodrigues, R., & Rosa, P. (2017). Eye-tracking as a research methodology in educational context. In Was, C., Sansosti, F., & Morris, B. (Eds.), Eye-Tracking Technology Applications in Educational Research (pp. 1-26). IGI Global.
• Rosa, P. J. (2015). What do your eyes really say? Bridging eye movements to consumer behavior. International Journal of Psychological Research, 8(2), 91-104.
• Saleem, M. R., Straus, A., & Napolitano, R. (2021). Interpretation of historic structure for non-invasive assessment using eye tracking. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences-ISPRS Archives, 46(1), 653-660.
• Saß, S., Schütte, K., & Lindner, M. A. (2017). Test-takers’ eye movements: Effects of integration aids and types of graphical representations. Computers & Education, 109, 85-97.
• Scheiter, K., & Eitel, A. (2015). Signals foster multimedia learning by supporting integration of highlighted text and diagram elements. Learning and Instruction, 36, 11-26.
• Scheiter, K., & Eitel, A. (2017). The use of eye tracking as a research and instructional tool in multimedia learning. In Was, C., Sansosti, F., & Morris, B. (Eds.), Eye-Tracking Technology Applications in Educational Research (pp. 143-164). IGI Global.
• Schwonke, R., Berthold, K., & Renkl, A. (2009). How multiple external representations are used and how they can be made more useful. Applied Cognitive Psychology, 23, 1227-1243.
• Shayan, S., Abrahamson, D., Bakker, A., Duijzer, A. C. G., & van der Schaaf, M. F. (2017). Eye-tracking the emergence of attentional anchors in a mathematics learning tablet activity. In Was, C., Sansosti, F., & Morris, B. (Eds.), Eye-tracking technology applications in educational research (pp. 166-194). IGI Global.
• Solheim, O., & Uppstad, P. (2011). Eye-tracking as a tool in process-oriented reading test validation. International Electronic Journal of Elementary Education, 4(1), 153-168.
• Tabachnick, B. G., & Fidell, L. (2015). Çok değişkenli istatistiklerin kullanımı. Baloğlu, M. (Çev. ed.). Nobel Akademik Yayıncılık.
• Tabbers, H. K., Paas, F., Lankford, C., Martens, R. L., & van Merriënboer, J. J. G. (2008). Studying eye movements in multimedia learning. In Rouet, J. F., Lowe, R., & Schnotz, W. (Eds.), Understanding Multimedia Documents (pp. 169-184). Springer.
• Tai, R. H., Loehr, F. J., & Brigham, F. J. (2006). An exploration of the use of eye-gaze tracking to study problem-solving on standardized science assessments. International Journal of Research & Method in Education, 29(2), 185-208.
• Tobii Pro. (2019, March). Learn & support. https://www.tobiipro.com/learn-and-support/
• Tsai, M. J., Hou, H. T., Lai, M. L., Liu, W. Y., & Yang, F. Y. (2011). Visual attention for solving multiple-choice science problem: An eye-tracking analysis. Computers & Education, 58, 375-385.
• Underwood, G., Hubbard, A., & Wilkinson, H. (1990). Eye fixations predict reading comprehension: The relationships between reading skill, reading speed, and visual inspection. Language and Speech, 33(1), 69-81.
• van Meeuwen, L. W., van Merriënboer, J. J. G., Jarodzka, H., Brand-Gruwel, S., Kirschner, P. A., & Bock, J. J. P. R. (2014). Identification of effective visual problem solving strategies in a complex visual domain. Learning and Instruction, 32, 10-21.
• Zentall, S. R., & Junglen, A. G. (2017). Investigating mindsets and motivation through eye tracking and other physiological measures. In C. Was, F. Sansosti, & B. Morris (Eds.), Eye-tracking technology applications in educational research (pp. 48-64). IGI Global.