Developing and implementing a student satisfaction scale for the emergency remote teaching in higher education

Abstract

With the COVID-19 pandemic that started in Turkey in 2020, a compulsory and rapid transition to distance education has been made at all levels of education, and online systems have become indispensable environments of the education system. In this process, students' satisfaction with the distance education process has emerged as a variable that educational institutions evaluate primarily. This study aims to develop a valid and reliable measurement tool to determine the satisfaction levels of students in higher education institutions in the process of emergency remote teaching (ERT). The scale items were created by the researchers through a literature review, and the scope and face validities were ensured by means of expert opinion. The scale was applied online to 6540 students at a state university in the spring term of the 2020–2021 academic year, via the learning management system. Because of the factor analysis, a scale consisting of 29 items and 4 factors was obtained. The factors are "The role of the Instructor" with 11 items, "Attitude" with 7 items, "ICT Infrastructure" with 7 items and "Usability and Access" with 4 items. It is seen that scale items explain 71.35% of the total variance. The structure of the scale was evaluated with confirmatory factor analysis and it was seen that the model had an acceptable fit. The values obtained in the reliability analyses show that the scale has high reliability. The Cronbach alpha coefficient for the scale was calculated as 0.975. Additionally, the satisfaction scores from the scale were examined in terms of the gender, field and degree level. While a significant relationship was found between satisfaction and gender, there was no significant relationship between satisfaction and field as well as between satisfaction and degree level.

Keywords

• distance education
• emergency remote teaching
• satisfaction scale
• college students

References

1. Absah, Y., Karina, B., & Harahap, R.H. (2021). Strategies to Improve Student Satisfaction Through the Quality of Online Learning Facilities and Infrastructure, Characteristics of Academic Staff, Lecturers Competence, and Good University Governance in Medan. Systematic Reviews in Pharmacy, 12(3), 671-675.
2. Alqurashi, E. (2018). Predicting student satisfaction and perceived learning within online learning environments. Distance Education, 40(1), 133–148. https://doi.org/10.1080/01587919.2018.1553562
3. Appleton-Knapp, S., & Krentler, K., (2006). Measuring student expectations and their effects on satisfaction: the importance of managing student expectations. Journal of Marketing Education, pp. 254-264
4. Avcı, İ., & Yıldız, E. (2021). COVID-19 Pandemi Sürecinde Uzaktan Eğitimi Kullanan Öğrencilerin Memnuniyet ve Davranışlarının Teknoloji Kabul Modeli Çerçevesinde İncelenmesi. Gümüşhane Üniversitesi Sosyal Bilimler Dergisi, 12 (3), 814-830. Retrieved from https://dergipark.org.tr/en/pub/gumus/issue/65088/886553
5. Azorín, C. (2020). Beyond COVID-19 supernova. Is another education coming? Journal of Professional Capital and Community, Vol. 5 No. 3/4, pp. 381-390. https://doi.org/10.1108/JPCC-05-2020-0019
6. Baber, H. (2020). Determinants of students’ perceived learning outcome and satisfaction in online learning during the pandemic of COVID-19. Journal of Education and e-learning Research, 7(3), 285-292.
7. Baumgartner, H., & Homburg, C. (1996). Applications of structural equation modeling in marketing and consumer research: A review. International journal of Research in Marketing, 13(2), 139-161.
8. Bawa'aneh, M. S., (2021). Distance Learning during COVID-19 Pandemic in UAE Public Schools: Student Satisfaction, Attitudes and Challenges. Contemporary Educational Technology, 13(3).

9. Bayrak, F., & Altun, A. (2020). Development of online course satisfaction scale. Turkish Online Journal of Distance Education, 21(4), 110-123.
10. Beavers, A. S., Lounsbury, J. W., Richards, J. K., Huck, S. W., Skolits, G. J., & Esquivel, S. L. (2013). Practical considerations for using exploratory factor analysis in educational research. Practical Assessment, Research, and Evaluation, 18(1), 6.
11. Bond, M. (2021). Schools and emergency remote education during the COVID-19 pandemic: A living rapid systematic review. Asian Journal of Distance Education, 15(2), 191-247.
12. Bozkurt, A. (2020). Koronavirüs (COVID-19) pandemi süreci ve pandemi sonrası dünyada eğitime yönelik değerlendirmeler: Yeni normal ve yeni eğitim paradigması. Açıköğretim Uygulamaları ve Araştırmaları Dergisi, 6(3), 112-142. Retrieved from https://dergipark.org.tr/en/pub/auad/issue/56247/773769
13. Cakir, O. (2017). The factors that affect online learners’ satisfaction. Anthropologist, 17(3): 895-902.
14. Carpenter, S. (2018). Ten steps in scale development and reporting: A guide for researchers. Communication methods and measures, 12(1), 25-44.
15. Cohen, L. Manion. L., & Morrison, K. (2007). Research methods in education (Sixth edition). Routledge
16. Costello, A. B., & Osborne, J. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical assessment, research, and evaluation, 10(1), 7.
17. DeVellis, R. F. (2014). Ölçek Geliştirme Kuram ve Uygulamalar (Scale Development Theory and Applications) (Üçüncü Basım) (T. Totan, Çev.). Ankara: Nobel Yayıncılık. (Orijinal çalışma basım tarihi 2012).
18. Dhaqane, M. K., & Afrah, N. A. (2016). Satisfaction of Students and Academic Performance in Benadir University. Journal of Education and Practice, 7(24), 59-63.
19. Elshami, W., Taha, M. H., Abuzaid, M., Saravanan, C., Al Kawas, S., & Abdalla, M. E. (2021). Satisfaction with online learning in the new normal: perspective of students and faculty at medical and health sciences colleges. Medical Education Online, 26(1), 1920090.
20. Fairchild, A. J., Horst, S. J., Finney, S. J., & Barron, K. E. (2005). Evaluating existing and new validity evidence for the Academic Motivation Scale. Contemporary Educational Psychology, 30(3), 331-358.
21. Farooq, R. (2016). Role of structural equation modeling in scale development. Journal of Advances in Management Research. 13(1), 75-91.
22. Fırat, M., Kılınç, H., & Yüzer, T. V. (2018). Level of intrinsic motivation of distance education students in e‐learning environments. Journal of Computer Assisted Learning, 34(1), 63-70.
23. Gibson, A. (2010). Measuring business student satisfaction: A review and summary of the major predictors. Journal of Higher Education Policy and Management, 32(3), 251–259. doi: 10.1080/13600801003743349
24. Glaesmer, H., Rief, W., Martin, A., Mewes, R., Brähler, E., Zenger, M., & Hinz, A. (2012). Psychometric properties and population‐based norms of the Life Orientation Test Revised (LOT‐R). British journal of health psychology, 17(2), 432-445.
25. Gray, J. A., & DiLoreto, M. (2016). The effects of student engagement, student satisfaction, and perceived learning in online learning environments. International Journal of Educational Leadership Preparation, 11(1), n1.
26. Harrington, D. (2009). Confirmatory factor analysis. Oxford University press.
27. Harsasi, M., & Sutawijaya, A. (2018). Determinants of Student Satisfaction in Online Tutorial: A Study of A Distance Education Institution. Turkish Online Journal of Distance Education, 19 (1), 89-99. DOI: 10.17718/tojde.382732
28. Hettiarachchi, S., Damayanthi, B. W. R., Heenkenda, S., Dissanayake, D. M. S. L. B., Ranagalage, M., & Ananda, L. (2021). Student satisfaction with online learning during the COVID-19 pandemic: a study at state universities in Sri Lanka. Sustainability, 13(21), 11749.
29. Hodges, C., Moore, S., Lockee, B., Trust, T., & Bond, A. (2020). The difference between emergency remote teaching and online learning. EDUCAUSE Review. https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teachingand-online-learning
30. Hwang, S., & Kim, H. K. (2022). Development and validation of the e-learning satisfaction scale (eLSS). Teaching and Learning in Nursing. https://doi.org/10.1016/j.teln.2022.02.004
31. Jiang, H., Islam, A. Y. M., Gu, X., & Spector, J. M. (2021). Online learning satisfaction in higher education during the COVID-19 pandemic: A regional comparison between Eastern and Western Chinese universities. Education and information technologies, 26(6), 6747-6769.
32. Kafes, A. Y., & Yıldırım, A. (2021). Üniversite Öğrencilerinin Uzaktan Eğitim Memnuniyeti Ölçeği’nin Geliştirilmesi: Geçerlik ve Güvenirlik Çalışması. Electronic Turkish Studies, 16(6).
33. Kanwar, A., & Sanjeeva, M. (2022). Student satisfaction survey: A key for quality improvement in the higher education institution. Journal of Innovation and Entrepreneurship, 11(1), 1-10.
34. Kim, S., & Kim, D. J. (2021). Structural relationship of key factors for student satisfaction and achievement in asynchronous online learning. Sustainability, 13(12), 6734. https://doi.org/10.3390/su13126734
35. Landrum, B., Bannister, J., Garza, G., & Rhame, S. (2021). A class of one: Students’ satisfaction with online learning. Journal of Education for Business, 96(2), 82-88.
36. Langan, A. M., & Harris, W. E. (2019). National student survey metrics: where is the room for improvement?. Higher Education, 78(6), 1075-1089.
37. Liaw, S. S., & Huang, H. M. (2013). Perceived satisfaction, perceived usefulness and interactive learning environments as predictors to self-regulation in e-learning environments. Computers & Education, 60(1), 14-24.
38. Lodico, M. G., Spaulding, D. T., & Voegtle, K. H. (2006). Methods in Educational Resarch From theory to Practice. John Wiley & Sons.
39. Martín-Rodríguez, Ó., Fernández-Molina, J. C., Montero-Alonso, M. Á., & González-Gómez, F. (2015). The main components of satisfaction with e-learning. Technology, Pedagogy and Education, 24(2), 267-277.
40. Meyers, L. S., Gamst, G., & Guarino, A. J. (2016). Applied multivariate research: Design and interpretation. Sage publications.
41. Miller, A. N., Sellnow, D. D., & Strawser, M. G. (2021). Pandemic pedagogy challenges and opportunities: Instruction communication in remote, HyFlex, and BlendFlex courses. Communication Education, 70(2), 202-204.
42. Mitchell, B., & Geva-May, I. (2009). Attitudes affecting online learning implementation in higher education. International Journal of E-Learning & Distance Education/Revue internationale du e-learning et la formation à distance, 23(1), 71-88.
43. Morgan, G. A., & Griego, O. V. (1998). Easy use and interpretation of SPSS for Windows: Answering research questions with statistics (Vol. 1). Psychology Press.
44. Ozturk, D. S., Ozturk, F., & Ozen , R. (2018). The relationship between prospective teachers’ readiness and satisfactions about internet-based distance education. Turkish Online Journal of Distance Education, 19(1), 147-162.
45. Resch, K., Alnahdi, G., & Schwab, S. (2022). Exploring the effects of the COVID-19 emergency remote education on students’ social and academic integration in higher education in Austria. Higher Education Research & Development, 1-15.
46. Richardson, J. T. (2005). Instruments for obtaining student feedback: A review of the literature. Assessment & evaluation in higher education, 30(4), 387-415.
47. Rothman, T., Romeo, L., Brennan, M., & Mitchell, D. (2011). Criteria for assessing student satisfaction with online courses. International Journal for e-Learning Security, 1(1-2), 27-32.
48. Schwarz, N. (2007). Attitude construction: Evaluation in context. Social Cognition, 25(5), 638–656.
49. Sharma, K., Deo, G., Timalsina, S., Joshi, A., Shrestha, N., & Neupane, H. C. (2020). Online learning in the face of COVID-19 pandemic: Assessment of students’ satisfaction at Chitwan medical college of Nepal. Kathmandu University Medical Journal, 18(2), 40-47.
50. Sun, P. C., Tsai, R. J., Finger, G., Chen, Y. Y., & Yeh, D. (2008). What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers & education, 50(4), 1183-1202.
51. Tabachnick, B. G., & Fidell, L.S. (2013). Using Multivariate Statistics (sixth ed.). Pearson
52. Toraman, Ç., Karadağ, E., & Polat, M. (2022). Validity and reliability evidence for the scale of distance education satisfaction of medical students based on item response theory (IRT). BMC medical education, 22(1), 1-13.
53. Tran, Q. H., & Nguyen, T. M. (2022). Determinants in Student Satisfaction with Online Learning: A Survey Study of Second-Year Students at Private Universities in HCMC. International Journal of TESOL & Education, 2(1), 63-80.
54. Tsai, C. C., & Lin, C. C. (2004). Taiwanese adolescents' perceptions and attitudes regarding the Internet: Exploring gender differences. Adolescence, 39(156), 725-734.
55. Tucker, L. R., & MacCallum, R. C. (1997). Exploratory factor analysis. Unpublished manuscript, Ohio State University, Columbus.
56. Tzafilkou, K., Perifanou, M., & Economides, A. A. (2021). Development and validation of a students’ remote learning attitude scale (RLAS) in higher education. Education and Information Technologies, 26(6), 7279-7305.
57. UNESCO (United Nations Educational Scientific and Cultural Organization) (2020). COVID-19 impact on education. https://en.unesco.org/covid19/educationresponse
58. Venkatesh, S., Rao, Y. K., Nagaraja, H., Woolley, T., Alele, F. O., & Malau-Aduli, B. S. (2020). Factors influencing medical students’ experiences and satisfaction with blended integrated E-learning. Medical Principles and Practice, 29(4), 396-402.
59. Weerasinghe, I. S., & Fernando, R. L. (2017). Students' satisfaction in higher education. American journal of educational research, 5(5), 533-539.
60. World Health Organization. (2020). WHO Director-General's opening remarks at the media briefing on COVID-19. https://www.who.int/dg/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19—11-march-2020 (March 11, 2020), Accessed 27 July 2022.
61. Williams, J., and Kane, D. (2009). Assessment and Feedback: Institutional Experiences of Student Feedback, 1996 to 2007. Higher Education Quarterly 63 (3), 264–86. https://doi.org/10.1111/j.1468-2273.2009.00430.x.
62. Winstone, N. E., Ajjawi, R., Dirkx, K., & Boud, D. (2021). Measuring what matters: the positioning of students in feedback processes within national student satisfaction surveys. Studies in Higher Education, 1-13. https://doi.org/10.1080/03075079.2021.1916909
63. Yıldız, E. P., Çengel, M., & Alkan, A. (2021). Pandemi Sürecinde Uzaktan Eğitim Ortamlarının Kullanımına İlişkin Tutum Ölçeği. OPUS Uluslararası Toplum Araştırmaları Dergisi, 17(33), 132-153. https://doi.org/10.26466/opus.811510
64. Yılmaz, E. O. (2020). Pandemi sürecinde google classroom uzaktan eğitim sisteminin entegrasyonu ve öğrenci memnuniyet düzeylerinin incelenmesi. Gelecek Vizyonlar Dergisi, 4(3), 42-55.
65. Yunusa, A. A., & Umar, I. N. (2021). A scoping review of critical predictive factors (CPFs) of satisfaction and perceived learning outcomes in E-learning environments. Education and Information Technologies, 26(1), 1223-1270. https://doi.org/10.1007/s10639-020-10286-1
66. YÖK (Yükseköğretim Kurulu). (2020). Küresel Salgın Bağlamında Yükseköğretim Kurumlarında Sağlıklı ve Temiz Ortamların Geliştirilmesi Kılavuzu. Accessed 27 July 2022. https://www.yok.gov.tr/Documents/Yayinlar/Yayinlarimiz/2020/yuksekogretim-kurumlarinda-saglikli-ve-temiz-ortamlarin-gelistirilmesi-kilavuzu/index.html