Exploring the Transitions of Online Engagement Through Learning Analytics with Markov Modelling with Elementary Education Pre-Service Teachers

Abstract

This paper provides a learning analysis based on system navigation to expand on the changes in their interactions with online learning systems over time. By providing an analysis of their participation and communication logs, this research aims to shed light on how they communicate with global learning systems and how these patterns are illuminated over time. The study was conducted with students in the elementary education department at a state university in the fall semester of the 2023-2024 academic year. Fifty-seven undergraduate students who actively participated in the Fundamentals of Computer Science course participated in the study. This study used Moodle data from the Fundamentals of Computer Science course for 14 weeks. Weekly learning content and materials in various formats have been uploaded to the system by the instructor. For each course, students were required to review different learning materials uploaded by the instructor and completely different learning task. Student's behavioral patterns will be examined in terms of the time they spend in the course, the time they spend on the content and the frequency with which they access the course content. Markov Chains have been applied to model online browsing behavior with time-varying variables. The findings show that the Markov chain for time spent, revealing the transition probabilities between engagement states at T1 and T2. The analysis indicates that students in the Low engagement group at T1 have a 50% probability of remaining in the Low cluster at T2, while also demonstrating a 50% chance of transitioning to the High cluster. Conversely, students in the High engagement group at T1 exhibit an 83% probability of staying in the High cluster at T2, with a 17% likelihood of moving to the Low cluster. Furthermore, the Markov chain for visit course, emphasizing the transition probabilities for students between engagement clusters at T1 and T2. It reveals that students initially in the High engagement group have an 83% probability of remaining in the High cluster at T2, while also indicating a 17% likelihood of transitioning to the Low cluster. On the other hand, students in the Low engagement group at T1 display a 50% probability of moving to the High cluster at T2, alongside a 65.5% chance of remaining in the Low cluster.

Keywords

• Elementary Education
• Pre-service Teachers
• Behavioral pattern
• Markov chain
• Learning analytics

References

1. Abebe, B. T., Weiss, M., Modess, C., Roustom, T., Tadken, T., Wegner, D., Schwantes, U., Neumeister, C., Schulz, H., Scheuch, E., al., et, Abu-Saad, K., Murad, H., Barid, R., Olmer, L., Ziv, A., Younis-Zeidan, N., Kaufman-Shriqui, V., Gillon-Keren, M., … Masha’al, D. (2019). Mindfulness virtual community. Trials, 17(1).
2. Ben-Eliyahu, A., Moore, D., Dorph, R., & Schunn, C. D. (2018). Investigating the multidimensionality of engagement: Affective, behavioral, and cognitive engagement across science activities and contexts. Contemporary Educational Psychology, 53. https://doi.org/10.1016/j.cedpsych.2018.01.002
3. Bergdahl, N., Nouri, J., & Fors, U. (2020). Disengagement, engagement and digital skills in technology-enhanced learning. Education and Information Technologies, 25(2). https://doi.org/10.1007/s10639-019-09998-w
4. Besenczi, R., Bátfai, N., Jeszenszky, P., Major, R., Monori, F., & Ispány, M. (2021). Large-scale simulation of traffic flow using Markov model. PLoS ONE, 16(2 February). https://doi.org/10.1371/journal.pone.0246062
5. Foster, E., & Siddle, R. (2020). The effectiveness of learning analytics for identifying at-risk students in higher education. Assessment and Evaluation in Higher Education, 45(6). https://doi.org/10.1080/02602938.2019.1682118
6. Gutiérrez, F., Seipp, K., Ochoa, X., Chiluiza, K., De Laet, T., & Verbert, K. (2020). LADA: A learning analytics dashboard for academic advising. Computers in Human Behavior, 107. https://doi.org/10.1016/j.chb.2018.12.004
7. Hasan, R., Palaniappan, S., Mahmood, S., Abbas, A., Sarker, K. U., & Sattar, M. U. (2020). Predicting student performance in higher educational institutions using video learning analytics and data mining techniques. Applied Sciences (Switzerland), 10(11). https://doi.org/10.3390/app10113894
8. He, Y., & Dong, X. (2020). Real time speech recognition algorithm on embedded system based on continuous Markov model. Microprocessors and Microsystems, 75. https://doi.org/10.1016/j.micpro.2020.103058

9. Henrie, C. R., Bodily, R., Larsen, R., & Graham, C. R. (2018). Exploring the potential of LMS log data as a proxy measure of student engagement. Journal of Computing in Higher Education, 30(2). https://doi.org/10.1007/s12528-017-9161-1
10. Hilpert, J. C., Greene, J. A., & Bernacki, M. (2023). Leveraging complexity frameworks to refine theories of engagement: Advancing self-regulated learning in the age of artificial intelligence. British Journal of Educational Technology, 54(5). https://doi.org/10.1111/bjet.13340
11. Huang, A. Y. Q., Lu, O. H. T., Huang, J. C. H., Yin, C. J., & Yang, S. J. H. (2020). Predicting students’ academic performance by using educational big data and learning analytics: evaluation of classification methods and learning logs. Interactive Learning Environments, 28(2). https://doi.org/10.1080/10494820.2019.1636086.
12. Khlaif, Z. N., Salha, S., & Kouraichi, B. (2021). Emergency remote learning during COVID-19 crisis: Students’ engagement. Education and information technologies, 26(6), 7033-7055.
13. Khodaei, A., Feizi-Derakhshi, M. R., & Mozaffari-Tazehkand, B. (2021). A Markov chain-based feature extraction method for classification and identification of cancerous DNA sequences. BioImpacts, 11(2). https://doi.org/10.34172/BI.2021.16
14. Kokoç, M., Akçapınar, G., & Hasnine, M. N. (2021). Unfolding Students’ Online Assignment Submission Behavioral Patterns Using Temporal Learning Analytics. Educational Technology and Society, 24(1).
15. Lam, S. F., Jimerson, S., Wong, B. P. H., Kikas, E., Shin, H., Veiga, F. H., Hatzichristou, C., Polychroni, F., Cefai, C., Negovan, V., Stanculescu, E., Yang, H., Liu, Y., Basnett, J., Duck, R., Farrell, P., Nelson, B., & Zollneritsch, J. (2014). Understanding and measuring student engagement in School: The results of an international study from 12 countries. School Psychology Quarterly, 29(2). https://doi.org/10.1037/spq0000057
16. Lei, H., Cui, Y., & Zhou, W. (2018). Relationships between student engagement and academic achievement: A meta-analysis. Social Behavior and Personality, 46(3). https://doi.org/10.2224/sbp.7054
17. Martin, A. J. (2007). Examining a multidimensional model of student motivation and engagement using a construct validation approach. British Journal of Educational Psychology, 77(2). https://doi.org/10.1348/000709906X118036
18. Martin, F., & Borup, J. (2022). Online learner engagement: Conceptual definitions, research themes, and supportive practices. Educational Psychologist, 57(3). https://doi.org/10.1080/00461520.2022.2089147
19. Mubarak, A. A., Cao, H., & Zhang, W. (2020). Prediction of students’ early dropout based on their interaction logs in online learning environment. Interactive Learning Environments. https://doi.org/10.1080/10494820.2020.1727529
20. Papadopoulos, C. T., Li, J., & O’Kelly, M. E. J. (2019). A classification and review of timed Markov models of manufacturing systems. Computers and Industrial Engineering, 128. https://doi.org/10.1016/j.cie.2018.12.019
21. Pentaraki, A., & Burkholder, G. J. (2017). Emerging Evidence Regarding the Roles of Emotional, Behavioural, and Cognitive Aspects of Student Engagement in the Online Classroom. European Journal of Open, Distance and E-Learning, 20(1). https://doi.org/10.1515/eurodl-2017-0001
22. Perifanou, M., Economides, A. A., & Tzafilkou, K. (2022). Greek teachers’ difficulties & opportunities in emergency distance teaching. E-Learning and Digital Media, 19(4), 361-379.
23. Polyzou, A., Nikolakopoulos, A. N., & Karypis, G. (2019). Scholars walk: A Markov chain framework for course recommendation. EDM 2019 - Proceedings of the 12th International Conference on Educational Data Mining.
24. Queiroga, E. M., Batista Machado, M. F., Paragarino, V. R., Primo, T. T., & Cechinel, C. (2022). Early Prediction of At-Risk Students in Secondary Education: A Countrywide K-12 Learning Analytics Initiative in Uruguay. Information (Switzerland), 13(9). https://doi.org/10.3390/info13090401
25. Redmond, P., Abawi, L. A., Brown, A., Henderson, R., & Heffernan, A. (2018). An online engagement framework for higher education. Online Learning Journal, 22(1). https://doi.org/10.24059/olj.v22i1.1175
26. Reeve, J., Cheon, S. H., & Jang, H. (2020). How and why students make academic progress: Reconceptualizing the student engagement construct to increase its explanatory power. Contemporary Educational Psychology, 62. https://doi.org/10.1016/j.cedpsych.2020.101899
27. Riestra-González, M., Paule-Ruíz, M. del P., & Ortin, F. (2021). Massive LMS log data analysis for the early prediction of course-agnostic student performance. Computers and Education, 163. https://doi.org/10.1016/j.compedu.2020.104108
28. Schnitzler, K., Holzberger, D., & Seidel, T. (2021). All better than being disengaged: Student engagement patterns and their relations to academic self-concept and achievement. European Journal of Psychology of Education, 36(3). https://doi.org/10.1007/s10212-020-00500-6
29. Sedrakyan, G., Malmberg, J., Verbert, K., Järvelä, S., & Kirschner, P. A. (2020). Linking learning behavior analytics and learning science concepts: Designing a learning analytics dashboard for feedback to support learning regulation. Computers in Human Behavior, 107. https://doi.org/10.1016/j.chb.2018.05.004
30. Skinner, E. A., Kindermann, T. A., & Furrer, C. J. (2009). A motivational perspective on engagement and disaffection: Conceptualization and assessment of children’s behavioral and emotional participation in academic activities in the classroom. Educational and Psychological Measurement, 69(3). https://doi.org/10.1177/0013164408323233
31. Sun, J. C. Y., Lin, C. T., & Chou, C. (2018). Applying learning analytics to explore the effects of motivation on online students’ reading behavioral patterns. International Review of Research in Open and Distance Learning, 19(2). https://doi.org/10.19173/irrodl.v19i2.2853
32. Tada, T., Toyoda, H., Yasuda, S., Miyake, N., Kumada, T., Kurisu, A., Ohisa, M., Akita, T., & Tanaka, J. (2019). Natural history of liver-related disease in patients with chronic hepatitis C virus infection: An analysis using a Markov chain model. Journal of Medical Virology, 91(10). https://doi.org/10.1002/jmv.25533
33. Teugels, J. L. (2008). Markov Chains: Models, Algorithms and Applications. Journal of the American Statistical Association, 103(483). https://doi.org/10.1198/jasa.2008.s254
34. Trichilli, Y., Boujelbène Abbes, M., & Masmoudi, A. (2020). Predicting the effect of Googling investor sentiment on Islamic stock market returns: A five-state hidden Markov model. International Journal of Islamic and Middle Eastern Finance and Management, 13(2). https://doi.org/10.1108/IMEFM-07-2018-0218
35. Uslu, N. A., & Durak, H. Y. (2022). Understanding self-regulation, achievement emotions, and mindset of undergraduates in emergency remote teaching: a latent profile analysis. Interactive Learning Environments, 1-20.
36. Vatsalan, D., Rakotoarivelo, T., Bhaskar, R., Tyler, P., & Ladjal, D. (2022). Privacy risk quantification in education data using Markov model. British Journal of Educational Technology, 53(4). https://doi.org/10.1111/bjet.13223
37. Vezne, R., Yildiz Durak, H., & Atman Uslu, N. (2023). Online learning in higher education: Examining the predictors of students’ online engagement. Education and information technologies, 28(2), 1865-1889.
38. Wang, H. Y., & Chih-Yuan, J. (2022). Influences of Online Synchronous VR Co-Creation on Behavioral Patterns and Motivation in Knowledge Co-Construction. Educational Technology and Society, 25(2).
39. Wong, Z. Y., & Liem, G. A. D. (2022). Student Engagement: Current State of the Construct, Conceptual Refinement, and Future Research Directions. In Educational Psychology Review (Vol. 34, Issue 1). https://doi.org/10.1007/s10648-021-09628-3
40. Ye, Y., Grossmann, I. E., Pinto, J. M., & Ramaswamy, S. (2020). Integrated Redundancy and Storage Design Optimization for Reliable Air Separation Units Based on Markov Chain-A Game Theoretic Solution. Industrial and Engineering Chemistry Research, 59(6). https://doi.org/10.1021/acs.iecr.9b04609
41. Yildiz Durak, H. (2019). Examining the acceptance and use of online social networks by preservice teachers within the context of unified theory of acceptance and use of technology model. Journal of Computing in Higher Education, 31(1), 173-209.
42. Yildiz Durak, H. (2023). Examining various variables related to authentic learning self-efficacy of university students in educational online social networks: creative self-efficacy, rational experiential thinking, and cognitive flexibility. Current Psychology, 42(25), 22093-22102.
43. Yong, K., Zaid, N. M., Wahid, N. A., Ashari, Z. M., Suhairom, N., & Said, M. M. (2021). Challenges in emergency remote teaching among Malaysian public elementary school teachers. International Journal of Emerging Technologies in Learning (iJET), 16(24), 74-90.
44. Yutong, X. (2021). Applications of Markov Chain in Forecast. Journal of Physics: Conference Series, 1848(1). https://doi.org/10.1088/1742-6596/1848/1/012061
45. Zhou, J., & Bhat, S. (2021). Modeling consistency using engagement patterns in online courses. ACM International Conference Proceeding Series. https://doi.org/10.1145/3448139.3448161