E-learning experience: Modeling students’ e-learning interactions using log data

Abstract

This study aims to examine e-learning experiences of the learners by using learner system interaction metrics. In this context, an e-learning environment has been structured within the scope of a course. Learners interacted with learning activities and leave various traces when they interact with others, contents, and assessment tasks. Log data were formed on these e-learning interactions. In the data analysis phase, firstly, a data pre-processing was performed, and then confirmatory factor analysis (CFA) was used to test how well the measured learning activity variables represent the latent system component variables. Then it was tested whether these components compose a latent e-learning experience variable (second-order CFA). The results showed that the learners interacted with five different system components: hypertext, the content package, video, discussion, and e-assessment. In conclusion, there is a factorial relationship between the system components and learning activities. These components taken together constitute an e-learning experience variable. When the factor loadings between the e-learning experience structure and subcomponents were examined, the discussion interactions in which the learner structured knowledge highlighted. In summary, the discussions, formative assessments, and content activities formed the learners’ e-learning experience together. In order to form a well-structured e-learning environment, these activities together should be experienced by the learners.

Keywords

• e-learning
• learner-system interaction
• log data
• interaction pattern
• e-learning experience
• confirmatory factor analysis

References

1. Baker, R. S., & Yacef, K. (2009). The State of Educational Data Mining in 2009: A Review and Future Visions. Journal of Educational Data Mining, 1(1), 3–17. https://doi.org/10.5281/zenodo.3554657
2. Baker, R. S., & Inventado, P. S. (2014). Educational data mining and learning analytics. In J. A. Larusson, & B. White, (Eds.), Learning analytics (pp. 61-75). Springer.
3. Baker, R. S., Lindrum, D., Lindrum, M. J., & Perkowski, D. (2015). Analyzing early at-risk factors in higher education E-learning courses. International Educational Data Mining Society. https://files.eric.ed.gov/fulltext/ED560553.pdf Accessed 20 August 2020.
4. Bra, P. D. (1998) Adaptive Hypermedia on the Web: Methods, techniques and applications. In Proceedings of the AACE WebNet'98 (pp. 220-225), AACE, Orlando.
5. Brusilovsky, P. (1998) Methods and techniques of adaptive hypermedia. In P. Brusilovsky, A. Kobsa, J. Vassileva (Eds.), Adaptive Hypertext and Hypermedia (pp. 1-44). Kluwer Academic Publishers.
6. Cela, K. L., Sicilia, M. Á., & Sánchez, S. (2015). Social network analysis in e-learning environments: A preliminary systematic review. Educational Psychology Review, 27(1), 219-246.
7. Cheng, L. C., & Chu, H. C. (2019). An innovative consensus map-embedded collaborative learning system for ER diagram learning: sequential analysis of students’ learning achievements. Interactive Learning Environments, 27(3), 410-425. https://doi.org/10.1080/10494820.2018.1482357
8. Christenson, S. L., Reschly, A. L., & Wylie, C. (2012). Handbook of research on student engagement. Springer US. https://doi.org/10.1007/978-1-4614-2018-7

9. Conole, G. G. (2013). MOOCs as disruptive technologies: strategies for enhancing the learner experience and quality of MOOCs. Distance Education Journal, (39). https://revistas.um.es/red/article/view/234221 Accessed 13 May 2020.
10. Conole, G. (2015). Designing effective MOOCs. Educational Media International, 52(4), 239-252. https://doi.org/10.1080/09523987.2015.1125989
11. Eryılmaz, M. (2019). The analysis of student behaviors in virtual learning environments by clustering method. Journal of Van Yüzüncü Yıl University Faculty of Education, 16(1), 725-743. http://doi.org/10.23891/efdyyu.2019.139
12. Ferguson, R. (2012). Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5/6), 304-317. https://doi.org/10.1504/IJTEL.2012.051816
13. Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of marketing research, 18(1), 39-50.
14. Govindasamy, T. (2001). Successful implementation of e-learning: Pedagogical considerations. The Internet and Higher Education, 4(3-4), 287-299.
15. Harrington, D. (2009). Confirmatory factor analysis. Oxford University Press.
16. Hrastinski, S. (2008). Asynchronous and synchronous e-learning. Educause Quarterly, 31(4), 51-55.
17. Huang, J., Dasgupta, A., Ghosh, A., Manning, J., & Sanders, M. (2014, March). Superposter behavior in MOOC forums. In Proceedings of the first ACM conference on Learning@ scale conference (pp. 117-126).
18. Huang, C. Q., Han, Z. M., Li, M. X., Jong, M. S. Y., & Tsai, C. C. (2019). Investigating students' interaction patterns and dynamic learning sentiments in online discussions. Computers & Education, 140, 103589. https://doi.org/10.1016/j.compedu.2019.05.015
19. Hwang, G. J., & Chang, H. F. (2011). A formative assessment-based mobile learning approach to improving the learning attitudes and achievements of students. Computers & Education, 56(4), 1023-1031. https://doi.org/10.1016/j.compedu.2010.12.002
20. Ichimura, Y., & Suzuki, K. (2017). Dimensions of MOOCs for quality design: analysis and synthesis of the literature. International Journal for Educational Media and Technology, 11(1), 42-49. https://jaems.jp/contents/icomej/vol11/05_Ichimura.pdf
21. Kent, C., Laslo, E., & Rafaeli, S. (2016). Interactivity in online discussions and learning outcomes. Computers & Education, 97, 116-128. https://doi.org/10.1016/j.compedu.2016.03.002
22. Keskin, S., Aydın, F., & Yurdugül, H. (2019). The determining of outliers on e-learning data in the context of educational data mining and learning analytics. Educational Technology Theory and Practice, 9(1), 292-309. https://doi.org/10.17943/etku.475149
23. Keskin, S., & Yurdugül, H. (2019). Factors Affecting Students’ Preferences for Online and Blended Learning: Motivational vs. Cognitive. European Journal of Open, Distance and E-learning, 22(2). https://doi.org/10.2478/eurodl-2019-0011
24. Kuh, G. D. (2009). The national survey of student engagement: Conceptual and empirical foundations. In R. M. Gonyea & G. D. Kuh (Eds.), New Directions for Institutional Research: No. 141. Using NSSE in institutional research (pp. 5-20). Jossey-Bass.
25. Lee, H. J., & Rha, I. (2009). Influence of structure and interaction on student achievement and satisfaction in web-based distance learning. Journal of Educational Technology & Society, 12(4), 372-382.
26. Liang, K., Zhang, Y., He, Y., Zhou, Y., Tan, W., & Li, X. (2017). Online behavior analysis-based student profile for intelligent E-learning. Journal of Electrical and Computer Engineering. https://doi.org/10.1155/2017/9720396
27. Martin, T., & Sherin, B. (2013). Learning analytics and computational techniques for detecting and evaluating patterns in learning: An introduction to the special issue. Journal of the Learning Sciences, 22(4), 511-520.
28. Menzi Çetin, N. & Altun, A. (2014). Uyarlanabilir öğrenme ortamları ve bir model önerisi. Eğitim Teknolojileri Araştırmaları Dergisi, 5(3).
29. Moore, J. L., Dickson-Deane, C., & Galyen, K. (2011). e-Learning, online learning, and distance learning environments: Are they the same? The Internet and Higher Education, 14(2), 129-135. https://doi.org/10.1016/j.iheduc.2010.10.001
30. Mubarak, A. A., Cao, H., & Zhang, W. (2020). Prediction of students’ early dropout based on their interaction logs in online learning environment. Interactive Learning Environments, 1-20. https://doi.org/10.1080/10494820.2020.1727529
31. Nguyen, Q., Huptych, M., & Rienties, B. (2018, March). Linking students’ timing of engagement to learning design and academic performance. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge (pp. 141–150). ACM.
32. Nortvig, A. M., Petersen, A. K., & Balle, S. H. (2018). A literature review of the factors influencing e-learning and blended learning in relation to learning outcome, student satisfaction and engagement. Electronic Journal of E-learning, 16(1), 46-55. https://files.eric.ed.gov/fulltext/EJ1175336.pdf Accessed 10 May 2021.
33. Osmanoğlu, U. Ö., Atak, O. N., Çağlar, K., Kayhan, H., & Can, T. C. (2020). Sentiment analysis for distance education course materials: A machine learning approach. Journal of Educational Technology and Online Learning, 3(1), 31-48. https://doi.org/10.31681/jetol.663733
34. Rodgers, T. (2008). Student engagement in the e-learning process and the impact on their grades. International Journal of Cyber Society and Education, 1(2), 143-156.
35. Saa, A. A. (2016). Educational data mining & students’ performance prediction. International Journal of Advanced Computer Science and Applications, 7(5), 212-220.
36. Sampson, D. (2016). Educational Data Analytics Technologies for Data-Driven Decision Making in Schools. eLearning Industry. https://elearningindustry.com/educational-data-analytics-technologies Accessed 20 May 2021.
37. Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8(2), 23-74
38. Schindler, L. A., Burkholder, G. J., Morad, O. A., & Marsh, C. (2017). Computer-based technology and student engagement: a critical review of the literature. International Journal of Educational Technology in Higher Education, 14(25), 1-28. https://doi.org/10.1186/s41239-017-0063-0
39. Shahiri, A. M., & Husain, W. (2015). A review on predicting student's performance using data mining techniques. Procedia Computer Science, 72, 414-422. https://doi.org/10.1016/j.procs.2015.12.157
40. Shuell, T J. (1988). The role of the student in learning from instruction. Contemporary Educational Psychology, 13, 276-295.
41. Siemens, G., & Baker, R. S. D. (2012, April). Learning analytics and educational data mining: towards communication and collaboration. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 252-254).
42. Shukla, N., Sharma, A., & Saggu, A. K. (2019, September). E-assessments and feedback mechanisms in Moocs. In 2019 International Conference on Issues and Challenges in Intelligent Computing Techniques (ICICT) (Vol. 1, pp. 1-6). IEEE.
43. Southwell, B. G., Anghelcev, G., Himelboim, I., & Jones, J. (2007). Translating user control availability into perception: The moderating role of prior experience. Computers in Human Behavior, 23(1), pp. 554–563. https://doi.org/10.1016/j.chb.2004.10.025
44. Şahin, M., Keskin, S., & Yurdugül, H. (2020). Sequential analysis of online learning behaviors according to e-learning readiness. In Isaias, P., Sampson, D., Ifenthaler, D. (Ed.), Online Teaching and Learning in Higher Education. Springer.
45. Wong, J. S., Pursel, B., Divinsky, A., & Jansen, B. J. (2015, March). An analysis of MOOC discussion forum interactions from the most active users. In International Conference on Social Computing, Behavioral-Cultural Modeling, and Prediction (pp. 452-457). Springer.
46. Yang, D., Lavonen, M. J., & Niemi, H. (2018). Online learning engagement: Factors and results-evidence from literature. Themes in eLearning, 11(1), 1-22. https://files.eric.ed.gov/fulltext/EJ1204753.pdf
47. Yıldırım, D., (2018). Interrelated analysis of academic achievement, interaction and navigation patterns of distance education students [Unpublished doctoral dissertation]. Hacettepe University.
48. Yousef, A. M. F., Chatti, M. A., Schroeder, U., Wosnitza, M., & Jacobs, H. (2014). MOOCs: A review of the state-of the-art. In Proceedings of CSEDU2014, 6th International Conference on Computer Supported Education, 9- 20. Barcelona, Spain.