Assessment Practices in and Beyond Emergency Remote Teaching: Exploring Faculty Experiences and Levels of Assessment Literacy

Öz

This study aims to investigate and compare faculty perceptions and practices regarding assessment during and after emergency remote teaching. The research focuses on understanding faculty experiences, challenges, and needs in these two distinct time frames. In the initial phase, 19 semi-structured focus group meetings were conducted with 54 faculty members from various disciplines during July and August 2020, when emergency remote teaching was implemented due to the pandemic. In the subsequent phase, a separate group of faculty members (4 persons) were interviewed after the resolution of pandemic conditions. The study specifically examines faculty perspectives and practices related to assessment. The collected data were analyzed using a qualitative thematic analysis approach with the assistance of MaxQDA. The findings of the study indicate that assessment practices during emergency remote teaching were strongly influenced by faculty professional development, student motivation, and the functionality of Learning Management Systems. Moreover, the study reveals that faculty members' assessment habits and practices were modified after experiencing emergency remote teaching. Based on these findings, recommendations are provided for higher education institutions to enhance professional development opportunities and support faculty in adapting to changing assessment practices.

Anahtar Kelimeler

• student assessment
• emergency remote teaching
• faculty development

Proje Numarası

120K190

Acil Uzaktan Öğretimde ve Sonrasında Değerlendirme Uygulamaları: Akademisyenlerin Deneyimlerinin ve Değerlendirme Okuryazarlığı Düzeylerinin Araştırılması

Öz

Bu çalışma, acil uzaktan eğitim sırasında ve sonrasında değerlendirmeye ilişkin akademisyenlerin algılarını ve uygulamalarını araştırmayı ve karşılaştırmayı amaçlamaktadır. Araştırma kapsamında, iki farklı zaman dilimindeki akademisyen deneyimlerini, zorluklarını ve ihtiyaçlarını anlamaya odaklanılmıştır. İlk aşamada, pandemi nedeniyle acil uzaktan eğitimin uygulandığı Temmuz ve Ağustos 2020'de çeşitli disiplinlerden 54 öğretim üyesiyle 19 yarı yapılandırılmış odak grup görüşmesi gerçekleştirilmiştir. Sonraki aşamada, pandemi koşullarının ortadan kalkmasının ardından ayrı bir grup öğretim üyesiyle (4 kişi) görüşülmüştür. Görüşmelerden elde veriler MaxQDA yardımıyla nitel tematik analiz yaklaşımı kullanılarak analiz edilmiştir. Çalışmanın bulguları, acil durum uzaktan öğretimi sırasında değerlendirme uygulamalarının akademisyenleri mesleki gelişimi, öğrenci motivasyonu ve Öğrenme Yönetim Sistemlerinin işlevselliğinden güçlü bir şekilde etkilendiğini göstermektedir. Ayrıca, çalışma, akademisyenlerin değerlendirme alışkanlıklarının ve uygulamalarının acil uzaktan öğretim sürecinden sonra değiştiğini ortaya koymaktadır. Bu bulgulara dayanarak, yükseköğretim kurumlarına mesleki gelişim fırsatlarını artırmaları ve değişen değerlendirme uygulamalarına uyum sağlamada akademisyenleri desteklemeleri için önerilerde bulunulmuştur.

Anahtar Kelimeler

• öğrenci değerlendirmesi
• acil uzaktan eğitim
• akademik personel gelişimi

Destekleyen Kurum

TÜBİTAK

Proje Numarası

120K190

Kaynakça

1. Astiandani, F. R., & Anam, S. (2021). EFL teachers’ perceptions towards the implementation of online formative assessment amidst the COVID- 19 pandemic. ELT Worldwide: Journal of English Language Teaching, 8(2), 269–277.
2. Bozkurt, A. (2020). Koronavirüs (COVID-19) pandemi süreci ve pandemi sonrası dünyada eğitime yönelik değerlendirmeler: Yeni normal ve yeni eğitim paradigması [Evaluations on education in the world during and after Coronavirus (COVID-19) pandemic: New normal and new education paradigm]. Açıköğretim Uygulamaları ve Araştırmaları Dergisi, 6(3), 112-142. https://dergipark.org.tr/en/pub/auad/issue/56247/773769
3. Bozkurt, A., Jung, I., Xiao, J., Vladimirschi, V., Schuwer, R., Egorov, G., ... Paskevicius, M. (2020). A global outlook to the interruption of education due to COVID-19 Pandemic: Navigating in a time of uncertainty and crisis. Asian Journal of Distance Education, 15(1), 1-–126. https://doi.org/10.5281/zenodo.3878572
4. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa
5. Chan, C. K. Y. (2023). A review of the changes in higher education assessment and grading policy during covid-19. Assessment & Evaluation in Higher Education, 48(6), 874-887. https://doi.org/10.1080/02602938.2022.2140780
6. Chapman, C., & Bell, I. (2020). Building back better education systems: Equity and COVID-19. Journal of Professional Capital and Community, 5(3/4), 227-236. https://doi.org/10.1108/JPCC-07-2020-0055
7. Condon, W., Iverson, E. R., Manduca, C. A., Rutz, C., & Willett, G. (2016). Faculty development and student learning: Assessing the connections. Indiana University Press.
8. Daniel, S. J. (2020). Education and the COVID-19 pandemic. Prospects, 1-6. https://doi.org/10.1007/s11125-020-09464-3

9. Doodle (2019). Doodle AG. Available from https://doodle.com/en/
10. Farrington, R. (2020, March 30). Colleges go to pass-fail due to coronavirus concerns: what does this mean to students? Forbes. https://www.forbes.com/sites/robertfarrington/2020/03/30/colleges-go-to-pass-fail-due-to-coronavirus-concerns-what-does-this-mean-for-students/?sh=37fb4d4e7eaa
11. Flaherty, C. (2020, April 23). Grading for pandemic: How lenient, or not, should professors be with students right now? Inside Higher Ed. https://www.insidehighered.com/news/2020/04/23/how-lenient-or-not-should-professors-be-students-right-now
12. Hodges, C., Moore, S., Lockee, B., Trust, T., & Bond, A. (2020, March 27). The difference between emergency remote teaching and online learning. Educause Review. https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-and-online-learning
13. Jensen, T., Marinoni, G., & Van’t Land, H. (2022). Higher education one year into the COVID-19 pandemic, second IAU global survey report. International Association of Universities (IAU).
14. Knapper, C. (2010). Changing Teaching Practice: Barriers and Strategies. In J. Christensen Hughes & J. Mighty (Eds.), Taking Stock: Research on Teaching and Learning in Higher Education (pp. 229-242). McGill-Queen’s University Press.
15. Kruger, R. A., & Casey, M. A. (2009). Focus groups: A practical guide for applied research (4th ed.). Sage.
16. Küppers, B., & Schroeder, U. (2020). Teacher’s perspective on e-assessment. In Proceedings of the 12th International Conference on Computer Supported Education—Vol. 1: CSEDU, CSEDU 2020—12th International Conference on Computer Supported Education, Online Streaming, 02 May 2020–04 May 2020 (pp. 495–502). SciTe Press. ISBN 978-989-758-417-6. https://doi.org/10.5220/0009578004950502.
17. Ladyshewsky, R. K. (2015). Post-graduate student performance in ‘supervised in-class’ vs. ‘unsupervised online’ multiple choice tests: implications for cheating and test security. Assessment & Evaluation in Higher Education, 40(7), 883-897. http://dx.doi.org/10.1080/02602938.2014.956683
18. MaxQDA 24-Teamwork (2018). VERBI Software. Berlin, Germany: VERBI Software. Available from https://www.maxqda.com/help.php?version=max18&pdf=1&id=34886
19. Mellar, H., Peytcheva-Forsyth, R., Kocdar, S., Karadeniz, A., & Yovkova, B. (2018). Addressing cheating in e-assessment using student authentication and authorship checking systems: teachers’ perspectives. International Journal for Educational Integrity, 14(1), 1-21. https://doi.org/10.1007/s40979-018-0025-x
20. Mirza, H. S. (2021). University teachers’ perceptions of online assessment during the COVID-19 pandemic in Lebanon. American Academic & Scholarly Research Journal, 11(10).
21. Munoz, A., & Mackay, J. (2019). An online testing design choice typology towards cheating threat minimisation. Journal of University Teaching & Learning Practice, 16(3), 5. https://ro.uow.edu.au/jutlp/vol16/iss3/5
22. National Research Council (2001). Knowing what students know: The science and design of educational assessment. National Academies Press. https://doi.org/10.17226/10019
23. O'Keefe, L., Rafferty, J., Gunder, A., & Vignare, K. (2020). Delivering high-quality instruction online in response to COVID-19: Faculty playbook. Online Learning Consortium.
24. Rossiter, S. (2020, Jun 21). Cheating becoming an unexpected Covid-19 side effect for universities. CBC News. https://www.cbc.ca/news/canada/edmonton/cheating-becoming-an-unexpected-covid-19-side-effect-for-universities-1.5620442
25. Parthasarathy, S., & Murugesan, S. (2020). Overnight Transformation to online education due to the COVID-19 pandemic: Lessons learned. eLearn, 2020(9), 1-9.
26. Şenel, S., & Şenel, H. C. (2021). Remote assessment in higher education during COVID-19 pandemic. International Journal of Assessment Tools in Education, 8(2), 181-199. https://doi.org/10.21449/ijate.820140
27. Taylor K. L., & Rege Colet N. (2010). Making the shift from faculty development to educational development: A conceptual framework grounded in practice. In A. Saroyan & M. Frenay (Eds.), Building teaching capacities in higher education: A comprehensive international model (pp. 139-167). Stylus.
28. UNESCO (2002). Open and distance learning: Trends, policy and strategy considerations. https://unesdoc.unesco.org/ark:/48223/pf0000128463
29. UNESCO (2020). School closures caused by Coronavirus (COVID-19). https://en.unesco.org/covid19/educationresponse
30. Unger, S., & Meiran, W. (2020). Student attitudes towards online education during the COVID-19 viral outbreak of 2020: Distance learning in a time of social distance. International Journal of Technology in Education and Science (IJTES), 4(4), 256-266. https://doi.org/10.46328/ijtes.v4i4.107
31. Vaismoradi, M., Jones, J., Turunen, H., & Snelgrove, S. (2016). Theme development in qualitative content analysis and thematic analysis. Journal of Nursing Education and Practice, 6(5), 100-110. https://doi.org/10.5430/jnep.v6n5p100
32. VERBI Software. (2020). MAXQDA 2020 [computer software]. Berlin, Germany: VERBI Software. Available from https://www.maxqda.com
33. Williamson, B., Eynon, R., & Potter, J. (2020). Pandemic politics, pedagogies and practices: digital technologies and distance education during the coronavirus emergency. Learning, Media and Technology, 45(2), 107-114. https://doi.org/10.1080/17439884.2020.1761641
34. WHO (2020). Coronavirus disease (COVID-19) pandemic. World Health Organization. https://www.who.int/emergencies/diseases/novel-coronavirus-2019
35. Yorke, M. (2013). Surveys of the student experience and the politics of feedback. In S. Merry, M. Price, D. Carless, & M. Taras (Eds.), Reconceptualising feedback higher education: Developing dialogue with students (pp 6-18). Routledge.
36. Zhang, C., Yan, X., & Wang, J. (2021). EFL teachers’ online assessment practices during the COVID-19 pandemic: Changes and mediating factors. The Asia-Pacific Education Researcher, 30, 499-507. https://doi.org/10.1007/s40299-021-00589-3