Learner engagement within MOOCs: Scale adaptation and an exploration regarding gender

Abstract

With the COVID-19 outbreak, now more than ever, minimizing human interaction and social distancing are of great importance. Since anyone is obligated to study remotely, attention to distance learning systems, specifically to massive online open courses (MOOCs), has exponentially risen. The present study aimed to adapt and validate the MOOC Engagement Scale (MES), developed by Deng et al. (2020a), into Turkish based on data collected from 258 participants in Türkiye enrolled in various MOOC courses to enhance their skills for future career development. The results indicated that the MES has adequate evidence of internal consistency reliability (ranging between .68 and .90) and factorial, discriminant, and convergent validity. Confirmatory factor analysis results supported the four-factor structure of the MES (i.e., behavioral, emotional, cognitive, and social). The MANOVA results indicated a statistically significant difference in MOOC engagement between genders, where males scored higher than females. The findings provided evidence that the Turkish MES has promising validity and reliability properties; therefore, it can be used to measure learner engagement in MOOCs.

Keywords

• Distance learning
• Massive open online course
• Learner engagement
• MOOC engagement scale
• Scale adaptation
• Scale validation

Ethical Statement

The Social and Humanity Sciences Ethical Committee at Akdeniz University granted approval for the study on March 26, 2024, under the reference number 886065.

KAÇD' lerde öğrenci katılımı: Ölçek uyarlaması ve cinsiyete ilişkin bir araştırma

Abstract

COVID-19 salgını ile birlikte, insan etkileşiminin minimize edilmesi ve sosyal mesafenin korunması her zamankinden daha büyük bir önem taşımaktadır. Herkesin uzaktan eğitime yönlendirilmesi, özellikle kitlesel açık çevrimiçi derslere (KAÇD) olan ilgi katlanarak artmıştır. Bu çalışma, gelecekteki kariyer gelişimleri için çeşitli KAÇD kurslarına kaydolan Türkiye'den 258 katılımcıdan toplanan verilere dayanarak Deng ve arkadaşları (2020a) tarafından geliştirilen KAÇD Katılım Ölçeği'ni (KKÖ) Türkçeye uyarlamayı ve geçerliliğini doğrulamayı amaçlamıştır. Sonuçlar, KKÖ'nün iç tutarlılık güvenilirliğine (.68 ile .90 arasında) ve faktöriyel, ayırıcı ve yakınsak geçerliliğe yeterli kanıt sağladığını göstermiştir. Doğrulayıcı faktör analizi sonuçları, MES'in dört faktörlü yapısını (davranışsal, duygusal, bilişsel ve sosyal katılım) desteklemiştir. MANOVA sonuçları, cinsiyetler arasında KAÇD katılımında istatistiksel olarak anlamlı bir fark olduğunu, erkeklerin kadınlardan daha yüksek puan aldığını göstermiştir. Elde edilen bulgular, KAÇD'nin Türkçe versiyonunun umut verici geçerlilik ve güvenilirlik özelliklerine sahip olduğunu ve bu nedenle KAÇD'de öğrenenlerin katılımını ölçmek için kullanılabileceğini ortaya koymuştur.

Keywords

• Uzaktan eğitim
• Kitlesel açık çevrimiçi dersler
• öğrenen katılımı
• KAÇD katılım ölçeği
• ölçek uyarlama
• ölçek doğrulama

References

1. Ağır, A. (2023). Kitlesel Açık Çevrimiçi Dersler (KAÇD) Bağlılık Ölçeği’nin Türkçeye uyarlanması, geçerlik ve güvenirlik çalışması. Erzincan Üniversitesi Eğitim Fakültesi Dergisi, 25(2), 325-339. https://doi.org/10.17556/erziefd.1097405
2. Alemayehu, L., & Chen, H. L. (2021). Learner and instructor-related challenges for learners’ engagement in MOOCs: A review of 2014–2020 publications in selected SSCI indexed journals. Interactive Learning Environments, 1-23. https://doi.org/10.1080/10494820.2021.1920430
3. Antonio, A., & Tuffley, D. (2014). The gender digital divide in developing countries. Future Internet, 6(4), 673-687. https://doi.org/10.3390/fi6040673
4. Appleton, J. J., Christenson, S. L., & Furlong, M. J. (2008). Student engagement with school: Critical conceptual and methodological issues of the construct. Psychology in the Schools, 45(5), 369–386. https://doi.org/10.1002/pits.20303
5. Appleton, J. J., Christenson, S. L., Kim, D., & Reschly, A. L. (2006). Measuring cognitive and psychological engagement: Validation of the student engagement instrument. Journal of School Psychology, 44(2006), 427–445. https://doi.org/10.1016/j.jsp.2006.04.002
6. Barba, P. D., Kennedy, G. E., & Ainley, M. D. (2016). The role of students’ motivation and participation in predicting performance in a MOOC motivation and participation in MOOCs. Journal of Computer Assisted Learning, 32, 218–231. https://doi.org/10.1111/jcal.12130
7. Barghaus, K. M., Henderson, C. M., Fantuzzo, J. W., Brumley, B., Coe, K., LeBoeuf, W. A., & Weiss, E. M. (2021). Classroom engagement scale: Validation of a teacher-report assessment used to scale in the kindergarten report card of a large urban school district. Early Education and Development, 1-23. https://doi.org/10.1080/10409289.2021.1985047
8. Baturay, M. H. (2015). An overview of the world of MOOCs. Procedia - Social and Behavioral Sciences, 174, 427–433. https://doi.org/10.1016/j.sbspro.2015.01.685

9. Bezerra, L. N. M., & da Silva, M. T. (2017). A review of literature on the reasons that cause the high dropout rates in the MOOCS. Register Espacios, 38(5), 11. http://www.revistaesacios.com/a17v38n05/a17v38n05p11.pdf
10. Carmines, E.G. and Zeller, R.A., (1979). Reliability and Validity assessment. Sage. Beverly Hills, CA.
11. Cohoon, J. M., & Aspray, W. (Eds.). (2006). Women and information technology: Research on underrepresentation. MIT Press.
12. Deng, R., Benckendorff, P., & Gannaway, D. (2020a). Learner engagement in MOOCs: Scale development and validation. British Journal of Educational Technology, 51(1), 245-262. https://doi.org/10.1111/bjet.12810
13. Deng, R., Benckendorff, P., & Gannaway, D. (2020b). Linking learner factors, teaching context, and engagement patterns with MOOC learning outcomes. Journal of Computer Assisted Learning, 36(5), 688-708. https://doi.org/10.1111/jcal.12437
14. Dixson, M. D. (2015). Measuring student engagement in the online course: The Online Student Engagement Scale (OSE). Online Learning, 19(4), 1–15. https://doi.org/10.24059/olj.v19i4.561
15. Eriksson, T., Adawi, T., & Stöhr, C. (2017). “Time is the bottleneck”: A qualitative study exploring why learners drop out of MOOCs. Journal of Computing in Higher Education, 29(1), 133–146. https://doi.org/10.1007/s12528-016-9127-8
16. EUGAIN. (2023). European Network for Gender Balance in Informatics (CA19122). Retrieved from https://eugain.eu Fredericks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59–109. https://doi.org/10.3102/00346543074001059
17. Goopio, J., & Cheung, C. (2021). The MOOC dropout phenomenon and retention strategies. Journal of Teaching in Travel & Tourism, 21(2), 177–197. https://doi.org/10.1080/15313220.2020.1809050
18. Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2014). Multivariate data analysis: Pearson new international edition. Harlow, Essex: Pearson.
19. Henrie, C. R., Halverson, L. R., & Graham, C. R. (2015). Measuring student engagement in technology-mediated learning: A review. Computers & Education, 90, 36–53. https://doi.org/10.1016/j.compedu.2015.09.005
20. Kala, D., & Chaubey, D. S. (2022). Examination of relationships among technology acceptance, student engagement, and perceived learning on tourism-related MOOCs. Journal of Teaching in Travel & Tourism, 1-18. https://doi.org/10.1080/15313220.2022.2038342
21. Karaguven, M. (2012). The Adaptation of Academic Motivation Scale to Turkish. Educational Sciences: Theory and Practice, 12(4), 2611-2618. https://eric.ed.gov/?id=EJ1002866
22. Kennedy, T., Wellman, B., & Klement, K. (2003). Gendering the digital divide. IT & society, 1(5), 72-96. http://www.ifs.tuwien.ac.at/~dieter/teaching/GmA/Kennedy2003.pdf
23. Kim, R., & Song, H.-D. (2023). Developing an agentic engagement scale in a self-paced MOOC. Distance Education, 44(1), 120–136. https://doi.org/10.1080/01587919.2022.2155619
24. Kuh, G.D (2001). Assessing what really matters to student learning: Inside the National Survey of Student Engagement. Change, 33(3), 10-17,66. https://doi.org/10.1080/00091380109601795
25. Kuo, T. M., Tsai, C. C., & Wang, J. C. (2021). Linking web-based learning self-efficacy and learning engagement in MOOCs: The role of online academic hardiness. The Internet and Higher Education, 51, 100819. https://doi.org/10.1016/j.iheduc.2021.100819
26. Lam, L. W. (2012). Impact of competitiveness on salespeople's commitment and performance. Journal of Business Research, 65(9), 1328-1334. https://doi.org/10.1016/j.jbusres.2011.10.026
27. Lam, S.-F., Wong, B. P. H., Yang, H., & Liu, Y. (2012). Understanding student engagement with a contextual model. In S.L. Christenson, A.L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 149–172). Springer. https://doi.org/10.1007/978-1-4614-2018-7_19
28. Lan, M., & Hew, K. F. (2018). The validation of the MOOC learner engagement and motivation scale. In T. Bastiaens, J. Van Braak, M. Brown, L. Cantoni, M. Castro, R. Christensen, ... O. Zawacki-Richter (Eds.), Proceedings of EdMedia: World Conference on Educational Media and Technology (pp. 1625-1636). Association for the Advancement of Computing in Education (AACE). Retrieved April 15, 2024, from https://www.learntechlib.org/primary/p/184389/
29. Lan, M., & Hew, K. F. (2020). Examining learning engagement in MOOCs: A self-determination theoretical perspective using mixed method. International Journal of Educational Technology in Higher Education, 17(1), 1-24. https://doi.org/10.1186/s41239-020-0179-5
30. Lee, J.-S. (2014). The relationship between student engagement and academic performance: Is it a myth or reality? The Journal of Educational Research, 107(3), 177–185. https://doi.org/10.1080/00220671.2013.807491
31. Lee, Y., & Choi, J. (2011). A review of online course dropout research: Implications for practice and future research. Educational Technology Research and Development, 59(5), 593–618. https://doi.org/10.1007/s11423-010-9177-y
32. Li, S., Du, J., & Sun, J. (2022). Unfolding the learning behaviour patterns of MOOC learners with different levels of achievement. International Journal of Educational Technology in Higher Education, 19(1), 1-20. https://doi.org/10.1186/s41239-022-00328-8
33. Lietaert, S., Roorda, D., Laevers, F., Verschueren, K., & De Fraine, B. (2015). The gender gap in student engagement: The role of teachers’ autonomy support, structure, and involvement. British Journal of Educational Psychology, 85(4), 498-518. https://doi.org/10.1111/bjep.12095
34. Lin, S. H., & Huang, Y. C. (2018). Assessing college student engagement: Development and validation of the student course engagement scale. Journal of Psychoeducational Assessment, 36(7), 694-708. https://doi.org/10.1177/0734282917697618
35. Liu, Y., Zhang, M., Qi, D., & Zhang, Y. (2022). Understanding the role of learner engagement in determining MOOCs satisfaction: A self-determination theory perspective. Interactive Learning Environments, 1-15. https://doi.org/10.1080/10494820.2022.2028853
36. Merenda, P. F. (2006). An overview of adapting educational and psychological assessment instruments: Past and present. Psychological Reports, 99, 307–314. https://doi.org/10.2466/pr0.99.2.307-314
37. Ong, C. S., & Lai, J. Y. (2006). Gender differences in perceptions and relationships among dominants of e-learning acceptance. Computers in Human Behavior, 22(5), 816–829. https://doi.org/10.1016/j.chb.2004.03.006
38. Öz, Y., & Boyacı, A. (2021). The role of student engagement in student outcomes in higher education: Implications from a developing country. International Journal of Educational Research, 110, 101880. https://doi.org/10.1016/j.ijer.2021.101880
39. Reschly, A. L., & Christenson, S. L. (2012). Jingle, jangle, and conceptual haziness: evolution and future directions of the engagement construct. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 3e20). New York, NY: Springer. http://dx.doi.org/10.1007/978-1-4614-2018-7_1
40. Reyes, M. R., Brackett, M. A., Rivers, S. E., White, M., & Salovey, P. (2012). Classroom emotional climate, student engagement, and academic achievement. Journal of Educational Psychology, 104, 700–712. https://psycnet.apa.org/doi/10.1037/a0027268
41. Shah, D. (2021). By the numbers: MOOCs in 2021. https://www.classcentral.com/report/mooc-stats-2021/
42. Siddiqi, A.F., Shabbir, M.S., Abbas, M., Mahmood, A. and Salman, R. (2022). Developing and testing student engagement scale for higher educational students. Journal of Applied Research in Higher Education. 14(1), 424-439. https://doi.org/10.1108/JARHE-11-2020-0388
43. Skinner, E.A., & Pitzer, J.R. (2012). Developmental Dynamics of Student Engagement, Coping, and Everyday Resilience. In: Christenson, S., Reschly, A., Wylie, C. (Eds) Handbook of Research on Student Engagement (pp. 21–44). Springer, Boston, MA. https://doi.org/10.1007/978-1-4614-2018-7_2
44. Solorzano-García, M., Navio-Marco, J., & Laguia, A. (2020). The influence of intrinsic motivation and contextual factors on mooc students’ social entrepreneurial intentions. Interactive Learning Environments, 29, 1–13. https://doi.org/10.1080/10494820.2020.1769680
45. Stoet, G., & Geary, D. C. (2018). The gender-equality paradox in STEM education. Psychological Science, 29(4), 581–593. https://doi.org/10.1177/0956797617741719
46. Sun, J. C. Y., & Rueda, R. (2012). Situational interest, computer self-efficacy and self-regulation: Their impact on student engagement in distance education. British Journal of Educational Technology, 43(2), 191–204. https://doi.org/10.1111/j.1467-8535.2010.01157.x
47. Tabachnick, B. G., & Fidell, L. S., (2007). Using multivariate statistics (6th ed.). Pearson.
48. Trowler, V., & Trowler, P. (2010). Student engagement evidence summary. York, England: Higher Education Academy.
49. UNESCO. (2020). I’d blush if I could: Closing gender divides in digital skills through education. https://unesdoc.unesco.org/ark:/48223/pf0000367416
50. Venkatesh, V., & Morris, M. G. (2000). Why don't men ever stop to ask for directions? Gender and technology acceptance. MIS Quarterly, 24(1), 115–139. https://doi.org/10.2307/3250981
51. Veletsianos, G., Collier, A., & Schneider, E. (2015). Digging deeper into learners’ experiences in MOOCs: Participation in social networks outside of MOOCs, notetaking and contexts surrounding content consumption. British Journal of Educational Technology, 46(3), 570–587. https://doi.org/10.1111/bjet.12297
52. Wan, L., Xie, S., & Shu, A. (2020). Toward an understanding of university students’ continued intention to use MOOCS: When UTAUT model meets TTF model. SAGE Open, 10(3). https://doi.org/10.1177/2158244020941858
53. Wang, M. T., Fredricks, J. A., Ye, F., Hofkens, T. L., & Linn, J. S. (2016). The Math and Science Engagement Scales: Scale Development, Validation, and Psychometric Properties. Learning and Instruction, 43, 16-26. https://doi.org/10.1016/j.learninstruc.2016.01.008
54. Wang, Y., Chen, H., & Li, K. C. (2022). A systematic review of empirical research on learning engagement in MOOCs: 2015–2022. Frontiers in Education, 7, 1074435. https://doi.org/10.3389/feduc.2022.1074435
55. Weber, E. Weippl & O. Zawacki-Richter (Eds.), Proceedings of EdMedia: World Conference on Educational Media and Technology (pp. 1625-1636). Amsterdam, Netherlands: Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/primary/p/184389/.
56. Williams, K. M., Stafford, R. E., Corliss, S. B., & Reilly, E. D. (2018). Examining student characteristics, goals, and engagement in Massive Open Online Courses. Computers & Education, 126, 433-442. https://doi.org/10.1016/j.compedu.2018.08.014
57. Xing, W., & Du, D. (2019). Dropout prediction in MOOCs: Using deep learning for personalized intervention. Journal of Educational Computing Research, 57(3), 547–570. https://doi.org/10.1177/0735633118757015
58. Yang, M., Shao, Z., Liu, Q., & Liu, C. (2017). Understanding the quality factors that influence the continuance intention of students toward participation in MOOCs. Educational Technology Research and Development, 65(5), 1195–1214. https://doi.org/10.1007/s11423-017-9513-6
59. York, T. T., Gibson, C., & Rankin, S. (2015). Defining and measuring academic success. Practical Assessment, Research, and Evaluation. 20, 1-20. https://doi.org/10.7275/hz5x-tx03
60. Zhang, Y., Wang, L., & Zhang, L. (2020). Gender differences in self-regulated online learning during the COVID-19 lockdown. Frontiers in Psychology, 11, 752131. https://doi.org/10.3389/fpsyg.2021.752131
61. Zhoc, K. C., Webster, B. J., King, R. B., Li, J. C., & Chung, T. S. (2019). Higher education student engagement scale (HESES): Development and psychometric evidence. Research in Higher Education, 60(2), 219-244. https://doi.org/10.1007/s11162-018-9510-6