IDENTIFYING BEHAVIORAL PATTERNS IN MOOC VIDEO ENGAGEMENT USING CLUSTERING APPROACH

Abstract

Videos are the core components of MOOCs for delivering course content and teaching the core concepts effectively. While the literature provided strong and consistent evidence regarding the link between video engagement and the success in MOOCs, the research on video engagement behavior is still emerging and in demand of further research. This research aims to contribute to the literature by identifying behavioral patterns of video engagement in a MOOC and reveal the association of these patterns with success and failure. In particular, we employed simple video engagement metrics with an attempt to identify clusters of behavioral patterns that can be applied to different contexts. Acknowledging that students may exhibit varied engagement behaviors across study sessions, a session-level clustering analysis was performed, differently from previous research. After applying K-Means clustering algorithm, three clusters of behavioral patterns were identified: static viewing (the most predominant behavior), in which students viewed videos with minimal interactions; engaged viewing, involving high frequency of play and pause events; and focused viewing (the least frequent pattern), which involved mainly seeking the video for specific information. While video sessions with static viewing were very common among both high and low achieving students, most engaged-viewing sessions or focused-viewing sessions consistently belonged to the successful students. In addition, successful students were found to demonstrate multiple viewing behaviors, suggesting their effort in using multiple strategies while watching videos. Based on the findings, the paper discusses implications for the design of MOOCs and other online learning platforms that support video-based learning.

Keywords

• massive open online course
• video analytics
• clustering
• video-based learning

MOOC VİDEO ETKİLEŞİMİNDEKİ DAVRANIŞ ÖRÜNTÜLERİNİN KÜMELEME YAKLAŞIMI İLE BELİRLENMESİ

Öz

Videolar, ders içeriğini iletmek ve temel kavramları etkili bir şekilde öğretmek için kitlesel açık çevrimiçi derslerin temel bileşenleridir. Literatür, video izleme ile bu kitlesel derslerdeki öğrencilerin başarısı arasındaki bağlantı konusunda güçlü ve tutarlı bulgular sunsa da video izleme davranışı üzerine yapılan araştırmalar hala sınırlı ve yeni gelişmekte olan bir alandır. Bu araştırma makalesi, bir kitlesel açık çevrimiçi dersindeki video izleme aktivitelerindeki davranışsal desenleri tanımlayarak bu desenlerin başarı ve başarısızlıkla ilişkisini ortaya çıkarmayı amaçlamaktadır. Özellikle farklı bağlamlarda kullanılabilmesi ve uygulanabilmesi amacıyla temel ve yaygınlaştırılabilir video izleme metrikleri kullanılmıştır. Öğrencilerin bir öğrenme oturumu süresince farklı video görüntüleme davranışları gösterebileceği kabul edilerek, önceki araştırmalardan farklı olarak kümeleme analizi öğrenci özelinde değil oturum düzeyinde gerçekleştirilmiştir. Kümeleme analizi sonucunda üç davranışsal desen kümesi ortaya çıkmıştır: statik görüntüleme (en yaygın davranış), öğrencilerin minimum etkileşimle videoları izlediği durum; katılımlı görüntüleme, oynatma ve duraklama olaylarının sık olduğu durum; ve odaklı görüntüleme (en az rastlanan desen), özellikle belirli bir bilgiyi arama durumu. Statik görüntülemenin hâkim olduğu video oturumları hem başarılı hem de başarısız öğrenciler arasında yaygın olarak gözlemlenmiştir. Ancak katılımlı görüntüleme oturumları veya odaklı görüntüleme oturumları ise en çok başarılı öğrenciler tarafından sergilenmiştir. Ayrıca başarılı öğrencilerin birden fazla görüntüleme davranışı sergilediği saptanmıştır. Bu bulgu, öğrencilerin videoları izlerken çeşitli sayıda strateji uygulama çabalarını göstermektedir. Bulgulara dayalı olarak, video tabanlı öğrenmeyi destekleyen diğer çevrimiçi öğrenme platformlarının tasarımı için pratik öneriler paylaşılmıştır.

Anahtar Kelimeler

• kümeleme
• video tabanlı öğrenme
• video etkileşimleri
• öğrenme analitiği
• Kitlesel açık çevrimiçi ders

References

1. Aggarwal, D., & Sharma, D. (2019). Application of clustering for student result analysis. International Journal of Recent Technology and Engineering, 7(6C). https://www.researchgate.net/publication/333115249.
2. Akcapinar, G., & Bayazit, A. (2018). Investigating Video Viewing Behaviors of Students with Different Learning Approaches Using Video Analytics. Turkish Online Journal of Distance Education, 19(4), 116–125. https://doi.org/10.17718/tojde.471907.
3. Antonenko, P. D., Toy, S., & Niederhauser, D. S. (2012). Using cluster analysis for data mining in educational technology research. Educational Technology Research and Development, 60(3), 383–398. https://doi.org/10.1007/s11423-012-9235-8.
4. Armstrong, A. W., Idriss, N. Z., & Kim, R. H. (2011). Effects of video-based, online education on behavioral and knowledge outcomes in sunscreen use: A randomized controlled trial. Patient Education and Counseling, 83(2), 273–277. https://doi.org/10.1016/j.pec.2010.04.033.
5. Benson, R., & Samarawickrema, G. (2009). Addressing the context of e‐learning: using transactional distance theory to inform design. Distance Education, 30(1), 5–21. https://doi.org/10.1080/01587910902845972.
6. Boroujeni, M. S., & Dillenbourg, P. (2019). Discovery and temporal analysis of MOOC study patterns. Journal of Learning Analytics, 6(1), 16–33. https://doi.org/10.18608/jla.2019.61.2.
7. Brinton, C. G., Buccapatnam, S., Chiang, M., & Poor, H. V. (2016). Mining MOOC Clickstreams: Video-Watching Behavior vs. In-Video Quiz Performance. IEEE Transactions on Signal Processing, 64(14), 3677–3692. https://doi.org/10.1109/TSP.2016.2546228.
8. Chatti, M. A., Marinov, M., Sabov, O., Laksono, R., Sofyan, Z., Fahmy Yousef, A. M., & Schroeder, U. (2016). Video annotation and analytics in CourseMapper. Smart Learning Environments, 3(1). https://doi.org/10.1186/s40561-016-0035-1.

9. Colasante, M. (2022). Not drowning, waving: The role of video in a renewed digital learning world. Australasian Journal of Educational Technology, 38(4), 176–189. https://doi.org/10.14742/ajet.7915.
10. Darmayanti, P. and Nova, M. (2022). Evaluating interactive video utilization in english for tourism business class. Premise Journal of English Education, 11(3), 646-662. https://doi.org/10.24127/pj.v11i3.5661.
11. Desai, T. and Kulkarni, D. (2022). Assessment of interactive video to enhance learning experience: a case study. Journal of Engineering Education Transformations, 35(S1), 74-80. https://doi.org/10.16920/jeet/2022/v35is1/22011.
12. Eisenberg, M., & Fischer, G. (2014). MOOCs: a Perspective from the Learning Sciences. Proceedings of ICLS 2014.
13. Er, E., Gómez-Sánchez, E., Dimitriadis, Y., Bote-Lorenzo, M. L., Asensio-Pérez, J. I., & Álvarez-Álvarez, S. (2019). Aligning learning design and learning analytics through instructor involvement: A MOOC case study. Interactive Learning Environments, 27(5–6), 685–698. https://doi.org/10.1080/10494820.2019.1610455.
14. Giannakos, M. N., Chorianopoulos, K. & Chrisochoides, N. (2014), Collecting and making sense of video learning analytics, in ‘Frontiers in Education Conference (FIE), 2014 IEEE’, IEEE, pp. 1–7.
15. Giannakos, M. N., Chorianopoulos, K., & Chrisochoides, N. (2015). Making sense of video analytics: Lessons learned from clickstream interactions, attitudes, and learning outcome in a video-assisted course. International Review of Research in Open and Distributed Learning, 16(1).
16. Glance, D. G., Forsey, M., & Riley, M. (2013). The pedagogical foundations of massive open online courses. First Monday, 18(5). https://doi.org/10.5210/fm.v18i5.4350.
17. Guo, P. J., Kim, J., & Rubin, R. (2014). How video production affects student engagement. Proceedings of the First ACM Conference on Learning @ Scale Conference, 41–50. https://doi.org/10.1145/2556325.2566239.
18. Hasan, R., Palaniappan, S., Mahmood, S., Abbas, A., Sarker, K. U., & Sattar, M. U. (2020). Predicting student performance in higher educational institutions using video learning analytics and data mining techniques. Applied Sciences (Switzerland), 10(11). https://doi.org/10.3390/app10113894.
19. Hew, K. F. (2015). Towards a model of engaging online students: Lessons from MOOCs and four policy documents. International Journal of Information and Education Technology, 5(6), 425–431. https://doi.org/10.7763/IJIET.2015.V5.543.
20. Kay, J., Reimann, P., Diebold, E., & Kummerfeld, B. (2013). MOOCs: So many learners , so much potential. IEEE Intelligent Systems, 28(3), 70–77.
21. Khalil, M., Topali, P., Ortega-Arranz, A. et al. (2023) Video Analytics in Digital Learning Environments: Exploring Student Behaviour Across Different Learning Contexts. Technology, Knowledge, and Learning. https://doi.org/10.1007/s10758-023-09680-8.
22. Kim, J., Guo, P. J., Seaton, D. T., Mitros, P., Gajos, K. Z., & Miller, R. C. (2014). Understanding in-video dropouts and interaction peaks inonline lecture videos. Proceedings of the First ACM Conference on Learning @ Scale Conference, 31–40. https://doi.org/10.1145/2556325.2566237.
23. Kizilcec, R. F., Piech, C., & Schneider, E. (2013). Deconstructing disengagement: Analyzing learner subpopulations in massive open online courses. Proceedings of Third International Conference on Learning Analytics and Knowledge, 170–179. https://doi.org/10.1145/2460296.2460330.
24. Lan, A. S., Brinton, C. G., Yang, T.-Y., & Chiang, M. (2017). Behavior-based latent variable model for learner engagement. Proceedings of the 10th International Conference on Educational Data Mining (EDM 2017).
25. Lee, J. K., & Lee, W. K. (2008). The relationship of e-Learner’s self-regulatory efficacy and perception of e-Learning environmental quality. Computers in Human Behavior, 24(1), 32–47.
26. Lemay, D. J., & Doleck, T. (2020). Grade prediction of weekly assignments in MOOCS: mining video-viewing behavior. Education and Information Technologies, 25(2), 1333–1342. https://doi.org/10.1007/s10639-019-10022-4.
27. Li, N., Kidzinski, L., Jermann, P., & Dillenbourg, P. (2015). MOOC video interaction patterns: What do they tell us? In G. Conole, T. Klobučar, C. Rensing, J. Konert, & E. Lavoué (Eds.), Design for Teaching and Learning in a Networked World (Vol. 9307). Springer, Cham. https://doi.org/10.1007/978-3-319-24258-3.
28. Littlejohn, A., Hood, N., Milligan, C., & Mustain, P. (2016). Learning in MOOCs : Motivations and self-regulated learning in MOOCs. The Internet and Higher Education, 29, 40–48. https://doi.org/10.1016/j.iheduc.2015.12.003.
29. Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action: aligning learning analytics with learning design. American Behavioral Scientist, 57(10), 1439–1459. https://doi.org/10.1177/0002764213479367.
30. Matcha, W., Gasevic, D., Uzir, N. A., Jovanovic, J., Pardo, A., Lim, L., & Maldonado-Mahauad, J. (2020). Analytics of learning strategies: Role of course design and delivery modality. Journal of Learning Analytics, 7(2), 45–71.
31. Mbouzao, B., Desmarais, M. C., & Shrier, I. (2020). Early prediction of success in MOOC from video interaction features. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12164 LNAI, 191–196. https://doi.org/10.1007/978-3-030-52240-7_35.
32. Milligan, C., Littlejohn, A., & Margaryan, A. (2013). Patterns of engagement in connectivist MOOCs. MERLOT Journal of Online Learning and Teaching, 9(2), 149–159.
33. Mirriahi, N., Liaqat, D., Dawson, S., & Gašević, D. (2016). Uncovering student learning profiles with a video annotation tool: Reflective learning with and without instructional norms. Educational Technology Research and Development, 64(6), 1083–1106. https://doi.org/10.1007/s11423-016-9449-2.
34. Mubarak, A. A., Cao, H., Zhang, W., & Zhang, W. (2021). Visual analytics of video-clickstream data and prediction of learners’ performance using deep learning models in MOOCs’ courses. Computer Applications in Engineering Education, 29(4), 710–732. https://doi.org/10.1002/cae.22328.
35. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Louppe, G., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, É. (2012). Scikit-learn: Machine learning in Python. 12, 2825–2830. https://doi.org/10.1007/s13398-014-0173-7.2.
36. Rousseeuw, P. J. (1987). Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. In Journal of Computational and Applied Mathematics (Vol. 20).
37. Safitri, D., Lestari, I., Maksum, A., Ibrahim, N., Marini, A., Zahari, M., … & Iskandar, R. (2021). Web-based animation video for student environmental education at elementary schools. International Journal of Interactive Mobile Technologies, 15(11), 66-80. https://doi.org/10.3991/ijim.v15i11.22023.
38. Shen, W. (2014). Using Video Recording System to Improve Student Performance in High-Fidelity Simulation. In S. Li, Q. Jin, X. Jiang, & J. H. Par (Eds.), Frontier and Future Development of Information Technology in Medicine and Education (pp. 1753–1757).
39. Stöhr, C., Stathakarou, N., Mueller, F., Nifakos, S., & McGrath, C. (2019). Videos as learning objects in MOOCs: A study of specialist and non-specialist participants’ video activity in MOOCs. British Journal of Educational Technology, 50(1), 166–176. https://doi.org/10.1111/bjet.12623.
40. Su, Y. S., & Wu, S. Y. (2021). Applying data mining techniques to explore user behaviors and watching video patterns in converged IT environments. Journal of Ambient Intelligence and Humanized Computing. https://doi.org/10.1007/s12652-020-02712-6.
41. Walji, S., Deacon, A., Small, J., & Czerniewicz, L. (2016). Learning through engagement: MOOCs as an emergent form of provision. Distance Education, 37(2), 208–223. https://doi.org/10.1080/01587919.2016.1184400.
42. Yoon, M., Lee, J., & Jo, I. H. (2021). Video learning analytics: Investigating behavioral patterns and learner clusters in video-based online learning. Internet and Higher Education, 50. https://doi.org/10.1016/j.iheduc.2021.100806.
43. Yürüm, O., Temizel, T., & Yildirim, S. (2022). The use of video clickstream data to predict university students’ test performance: a comprehensive educational data mining approach. Education and Information Technologies, 28(5), 5209-5240. https://doi.org/10.1007/s10639-022-11403-y.
44. Vioskha, Y., Roza, Y., & Maimunah, M. (2021). Improving mathematics cognitive learning outcomes through the application of bandicam video to class x senior high school students in kampar regency. Journal of Educational Sciences, 5(4), 665-667. https://doi.org/10.31258/jes.5.4.p.665-677.
45. Zhang, J., Huang, Y., & Gao, M. (2022). Video Features, Engagement, and Patterns of Collective Attention Allocation: An Open Flow Network Perspective. Journal of Learning Analytics, 9(1), 32–52. https://doi.org/10.18608/jla.2022.7421