An In-Depth Examination of Writing Assessment Literacy Among University Preparatory Students in Turkish Higher Education

Year 2024, Volume: 26 Issue: 2, 291 - 304, 30.06.2024

Doğukan Hazar Özçubuk Ali Merç

https://doi.org/10.17556/erziefd.1427430

Abstract

As a nascent research domain, there is a scarcity of research studies that examine the levels of assessment literacy, perceptions, beliefs, and practices among students concerning their Student Writing Assessment Literacy (SWAL). To address this research gap, the present study adopts a mixed-methods design, incorporating the Scale of SWAL (Xu et al., 2023) alongside semi-structured interviews. 134 students answered the scale, while an additional twelve students volunteered to participate in the interviews. Student responses from the scale were subjected to analysis through descriptive statistics, revealing a notable correlation between students’ comprehension of assessment objectives and the manifestation of positive attitudes towards learning. Nevertheless, a misalignment was noticed between their theoretical understanding and beliefs and their pragmatic implementation regarding written assignments. Interview data underwent analysis through thematic analysis, elucidating four main themes: (1) feedback valuation, (2) scoring criteria awareness, (3) feedback delivery preferences, and (4) alternative assessment avoidance. Implications emphasise the importance of involving students in the assessment process and advocate for teacher training initiatives to enhance SWAL. Future research could broaden their scope to reach diverse academic levels and integrate grades as a variable. Overall, the present study provides valuable insights into SWAL, shedding light on assessment practices at the tertiary level.

Keywords

assessment knowledge, assessment literacy, student writing assessment literacy, writing assessment, writing assessment literacy

Türk Yükseköğretiminde Üniversite Hazırlık Öğrencilerinin Yazma Değerlendirme Okuryazarlığının Derinlemesine İncelenmesi

Abstract

Yeni gelişmekte olan bir araştırma alanı olarak, öğrencilerin Öğrenci Yazma Değerlendirme Okuryazarlığı (SWAL) ile ilgili dil değerlendirme okuryazarlık düzeylerini, algılarını, inançlarını ve uygulamalarını inceleyen araştırma sayısı oldukça azdır. Bu araştırma boşluğunu gidermek için, bu çalışma, yarı yapılandırılmış görüşmelerin yanı sıra SWAL Ölçeğini (Xu vd., 2023) içeren karma bir yöntem tasarımını benimsemiştir. 134 üniversite hazırlık sınıfı öğrencisi ölçeği yanıtlarken, on iki öğrenci de görüşmelere katılmak için gönüllü olmuştur. Öğrencilerin ölçekten aldıkları yanıtlar tanımlayıcı istatistikler aracılığıyla analize tabi tutulmuş ve öğrencilerin değerlendirme hedeflerini anlamaları ile öğrenmeye yönelik olumlu tutum sergilemeleri arasında kayda değer bir korelasyon olduğu ortaya çıkmıştır. Bununla birlikte, öğrencilerin teorik anlayışları ve inançları ile yazılı ödevlere ilişkin pragmatik uygulamaları arasında bir uyumsuzluk olduğu fark edilmiştir. Yarı-yapılandırılmış görüşme verileri tematik analiz yoluyla analiz edilmiş ve dört ana tema ortaya çıkmıştır: (1) geribildirim değerlemesi, (2) puanlama kriterleri farkındalığı, (3) geribildirim verme tercihleri ve (4) alternatif değerlendirmeden kaçınma. Bulgular, öğrencileri değerlendirme sürecine dahil etmenin önemini vurgulamakta ve SWAL’ı geliştirmek için öğretmen eğitimi girişimlerini savunmaktadır. Gelecekteki araştırmalar, farklı akademik seviyelere ulaşmak ve notları bir değişken olarak entegre etmek için kapsamlarını genişletebilir. Genel olarak, mevcut çalışma SWAL konusunda değerli bilgiler sunmakta ve yükseköğretim düzeyindeki değerlendirme uygulamalarına ışık tutmaktadır.

Keywords

değerlendirme bilgisi, değerlendirme okuryazarlığı, öğrenci yazma değerlendirme okuryazarlığı, yazma değerlendirmesi, yazma değerlendirme okuryazarlığı

References

• Absolum, M., Flockton, L., Hattie, J., Hipkins, R., & Reid, I. (2009). Directions for assessment in New Zealand: Developing students’ assessment capabilities. Unpublished paper prepared for the Ministry of Education.
• Aitken, N. (2011). Student Voice in Fair Assessment Practice. In Leading Student Assessment (pp. 175-200). Springer. https://doi.org/10.1007/978-94-007-1727-5_9
• Alonzo, D. (2016). Development and Application of a Teacher Assessment for Learning (AfL) Literacy Tool. unsworks.unsw.edu.au. https://doi.org/10.26190/unsworks/18809
• Ataie-Tabar, M., Zareian, G., Amirian, S. M. R., & Adel, S. M. R. (2019). A Study of Socio-Cultural Conception of Writing Assessment Literacy: Iranian EFL Teachers’ and Students’ Perspectives. English Teaching & Learning, 43(4), 389-409. https://doi.org/10.1007/s42321-019-00035-0
• Baird, J.-A., Andrich, D., Hopfenbeck, T. N., & Stobart, G. (2017). Assessment and learning: fields apart? Assessment in Education: Principles, Policy & Practice, 24(3), 317-350. https://doi.org/10.1080/0969594x.2017.1319337
• Bernard, H. R. (2018). Research methods in anthropology: qualitative and quantitative approaches (Sixth edition. ed.). Rowman & Littlefield.
• Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa
• Braun, V., & Clarke, V. (2012). Thematic analysis. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research methods in psychology, Vol. 2. Research designs: Quantitative, qualitative, neuropsychological, and biological (pp. 57–71). American Psychological Association. https://doi.org/10.1037/13620-004
• Brown, G. T. L., & Wang, Z. (2016). Understanding Chinese university student conceptions of assessment: cultural similarities and jurisdictional differences between Hong Kong and China. Social Psychology of Education, 19(1), 151-173. https://doi.org/10.1007/s11218-015-9322-x
• Butler, Y. G., Peng, X., & Lee, J. (2021). Young learners’ voices: Towards a learner-centered approach to understanding language assessment literacy. Language Testing, 38(3), 429-455. https://doi.org/10.1177/0265532221992274
• Carless, D., Salter, D., Yang, M., & Lam, J. (2010). Developing sustainable feedback practices. Studies in Higher Education, 36(4), 395-407. https://doi.org/10.1080/03075071003642449
• Chan, C. K. Y., & Luo, J. (2021). A four-dimensional conceptual framework for student assessment literacy in holistic competency development. Assessment & Evaluation in Higher Education, 46(3), 451-466. https://doi.org/10.1080/02602938.2020.1777388
• Chen, G., Wang, X., & Wang, L. (2023). Developing assessment literacy among trainee translators: scaffolding self and peer assessment as an intervention. Assessment & Evaluation in Higher Education, 48(6), 888-902. https://doi.org/10.1080/02602938.2022.2142515
• Charteris, J., & Thomas, E. (2016). Uncovering ‘unwelcome truths’ through student voice: teacher inquiry into agency and student assessment literacy. Teaching Education, 28(2), 162-177. https://doi.org/10.1080/10476210.2016.1229291
• Colby-Kelly, C., & Turner, C. E. (2007). AFL Research in the L2 Classroom and Evidence of Usefulness: Taking Formative Assessment to the Next Level. The Canadian Modern Language Review, 64(1), 9-37. https://doi.org/10.3138/cmlr.64.1.009
• Corbin, J. M., & Strauss, A. (1990). Grounded theory research: Procedures, canons, and evaluative criteria. Qualitative Sociology, 13(1), 3-21. https://doi.org/10.1007/bf00988593
• Creswell, J. W., & Creswell, J. D. (2018). Research design: qualitative, quantitative, and mixed methods approaches (Fifth edition. ed.). SAGE.
• Crusan, D. (2010). Assessment in the second language writing classroom. University of Michigan Press.
• Crusan, D., Plakans, L., & Gebril, A. (2016). Writing assessment literacy: Surveying second language teachers’ knowledge, beliefs, and practices. Assessing Writing, 28, 43-56. https://doi.org/10.1016/j.asw.2016.03.001
• Crusan, D. (2022). Writing Assessment Literacy. In Research Questions in Language Education and Applied Linguistics (pp. 431-435). https://doi.org/10.1007/978-3-030-79143-8_77
• Davari Torshizi, M., & Bahraman, M. (2019). I explain, therefore I learn: Improving students’ assessment literacy and deep learning by teaching. Studies in Educational Evaluation, 61, 66-73. https://doi.org/10.1016/j.stueduc.2019.03.002
• Davies, A. (2008). Textbook trends in teaching language testing. Language Testing, 25(3), 327-347. https://doi.org/10.1177/0265532208090156
• Deeley, S. J., & Bovill, C. (2017). Staff student partnership in assessment: enhancing assessment literacy through democratic practices. Assessment & Evaluation in Higher Education, 42(3), 463-477. https://doi.org/10.1080/02602938.2015.1126551
• DeLuca, C., & Klinger, D. A. (2010). Assessment literacy development: identifying gaps in teacher candidates’ learning. Assessment in Education: Principles, Policy & Practice, 17(4), 419-438. https://doi.org/10.1080/0969594x.2010.516643
• Denton, P., & McIlroy, D. (2017). Response of students to statement bank feedback: the impact of assessment literacy on performances in summative tasks. Assessment & Evaluation in Higher Education, 43(2), 197-206. https://doi.org/10.1080/02602938.2017.1324017
• Dörnyei, Z. (2007). Research methods in applied linguistics. Oxford University Press.
• Dörnyei, Z., & Dewaele, J.-M. (2022). Questionnaires in Second Language Research. Routledge.
• Flores, M. A., Brown, G., Pereira, D., Coutinho, C., Santos, P., & Pinheiro, C. (2019). Portuguese university students’ conceptions of assessment: taking responsibility for achievement. Higher Education, 79(3), 377-394. https://doi.org/10.1007/s10734-019-00415-2
• Francis, R. A. (2008). An investigation into the receptivity of undergraduate students to assessment empowerment. Assessment & Evaluation in Higher Education, 33(5), 547-557. https://doi.org/10.1080/02602930701698991
• Fulcher, G. (2010). Practical language testing. London: Hodder Education.
• Fulcher, G. (2012). Assessment Literacy for the Language Classroom. Language Assessment Quarterly, 9(2), 113-132. https://doi.org/10.1080/15434303.2011.642041
• Gipps, C. (1999). Socio-Cultural Aspects of Assessment. Review of Research in Education, 24. https://doi.org/10.2307/1167274
• Greene, J. C. (2007). Mixed methods in social inquiry. San Francisco, CA: Jossey-Bass.
• Hannigan, C., Alonzo, D., & Oo, C. Z. (2022). Student assessment literacy: indicators and domains from the literature. Assessment in Education: Principles, Policy & Practice, 29(4), 482-504. https://doi.org/10.1080/0969594x.2022.2121911
• Hattie, J., & Timperley, H. (2016). The Power of Feedback. Review of Educational Research, 77(1), 81-112. https://doi.org/10.3102/003465430298487
• Hawe, E. M., & Dixon, H. R. (2014). Building students’ evaluative and productive expertise in the writing classroom. Assessing Writing, 19, 66-79. https://doi.org/10.1016/j.asw.2013.11.004
• Hirvela, A., & Belcher, D. (2007). Writing scholars as teacher educators: Exploring writing teacher education. Journal of Second Language Writing, 16(3), 125-128. https://doi.org/10.1016/j.jslw.2007.08.001
• Homayounzadeh, Z., & Razmjoo, S. A. (2021). Examining Assessment Literacy in Practice in an Iranian Context: Does it Differ for Instructors and Learners? Teaching English as a Second Language Quarterly (Formerly Journal of Teaching Language Skills), 40(2), 1-45.
• Inbar-Lourie, O. (2008). Constructing a language assessment knowledge base: A focus on language assessment courses. Language Testing, 25(3), 385-402. https://doi.org/10.1177/0265532208090158
• Inbar-Lourie, O. (2013). Guest Editorial to the special issue on language assessment literacy. Language Testing, 30(3), 301-307. https://doi.org/10.1177/0265532213480126
• Jeong, H. (2013). Defining assessment literacy: Is it different for language testers and non-language testers? Language Testing, 30(3), 345-362. https://doi.org/10.1177/0265532213480334
• Lam, R. (2019). Teacher assessment literacy: Surveying knowledge, conceptions, and practices of classroom-based writing assessment in Hong Kong. System, 81, 78-89. https://doi.org/10.1016/j.system.2019.01.006
• Lee, I. (2017). Classroom Writing Assessment and Feedback in L2 School Contexts. Springer. https://doi.org/10.1007/978-981-10-3924-9
• Lee, I., & Mao, Z. (2024). Writing teacher feedback literacy: Surveying second language teachers’ knowledge, values, and abilities. Journal of Second Language Writing, 63. https://doi.org/10.1016/j.jslw.2024.101094
• Yu, S., & Lee, I. (2024). Writing assessment and feedback literacy: Where do we stand and where can we go? Assessing Writing, 60. https://doi.org/10.1016/j.asw.2024.100829
• Lodge, C. (2008). Engaging student voice to improve pedagogy and learning: An exploration of examples of innovative pedagogical approaches for school improvement. International Journal of Pedagogies and Learning, 4(5), 4-19. https://doi.org/10.5172/ijpl.4.5.4
• Mackey, A., & Gass, S. M. (2022). Second language research: methodology and design (2nd ed.). Routledge.
• Mellati, M., & Khademi, M. (2018). Exploring Teachers’ Assessment Literacy: Impact on Learners’ Writing Achievements and Implications for Teacher Development. Australian Journal of Teacher Education, 43(6), 1-18. https://doi.org/10.14221/ajte.2018v43n6.1
• O’Donovan, B., Price, M., & Rust, C. (2001). The Student Experience of Criterion-Referenced Assessment (Through the Introduction of a Common Criteria Assessment Grid). Innovations in Education and Teaching International, 38(1), 74-85. https://doi.org/10.1080/147032901300002873
• O’Donovan, B., Price, M., & Rust, C. (2004). Know what I mean? Enhancing student understanding of assessment standards and criteria. Teaching in Higher Education, 9(3), 325-335. https://doi.org/10.1080/1356251042000216642
• O’Donovan, B., Price, M., & Rust, C. (2008). Developing student understanding of assessment standards: a nested hierarchy of approaches. Teaching in Higher Education, 13(2), 205-217. https://doi.org/10.1080/13562510801923344
• Ölmezer-Öztürk, E., & Aydin, B. (2018). Toward measuring language teachers’ assessment knowledge: development and validation of Language Assessment Knowledge Scale (LAKS). Language Testing in Asia, 8(1). https://doi.org/10.1186/s40468-018-0075-2
• Panadero, E., Jonsson, A., & Strijbos, J.-W. (2016). Scaffolding Self-Regulated Learning Through Self-Assessment and Peer Assessment: Guidelines for Classroom Implementation. In C. Wyatt-Smith (Ed.), Assessment for Learning: Meeting the Challenge of Implementation (pp. 311-326). https://doi.org/10.1007/978-3-319-39211-0_18
• Pereira, D., Niklasson, L., & Flores, M. A. (2016). Students’ perceptions of assessment: a comparative analysis between Portugal and Sweden. Higher Education, 73(1), 153-173. https://doi.org/10.1007/s10734-016-0005-0
• Peterson, E. R., Brown, G. T. L., & Jun, M. C. (2015). Achievement emotions in higher education: A diary study exploring emotions across an assessment event. Contemporary Educational Psychology, 42, 82-96. https://doi.org/10.1016/j.cedpsych.2015.05.002
• Popham, W. J. (2009). Assessment Literacy for Teachers: Faddish or Fundamental? Theory Into Practice, 48(1), 4-11. https://doi.org/10.1080/00405840802577536
• Rezai, A., Alibakhshi, G., Farokhipour, S., & Miri, M. (2021). A phenomenographic study on language assessment literacy: hearing from Iranian university teachers. Language Testing in Asia, 11(1). https://doi.org/10.1186/s40468-021-00142-5
• Roslan, R., Nishio, Y., & Jawawi, R. (2022). Analysing English language teacher candidates’ assessment literacy: A case of Bruneian and Japanese universities. System, 111. https://doi.org/10.1016/j.system.2022.102934
• Rust, C., Price, M., & O’Donovan, B. (2003). Improving Students’ Learning by Developing their Understanding of Assessment Criteria and Processes. Assessment & Evaluation in Higher Education, 28(2), 147-164. https://doi.org/10.1080/02602930301671
• Sadler, D. R. (2010). Beyond feedback: developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35(5), 535-550. https://doi.org/10.1080/02602930903541015
• Scarino, A. (2013). Language assessment literacy as self-awareness: Understanding the role of interpretation in assessment and in teacher learning. Language Testing, 30(3), 309-327. https://doi.org/10.1177/0265532213480128
• Shepard, L. A. (2000). The role of assessment in a learning culture. Teaching and Learning, 229–253.
• Smith, C. D., Worsfold, K., Davies, L., Fisher, R., & McPhail, R. (2013). Assessment literacy and student learning: the case for explicitly developing students ‘assessment literacy.’ Assessment & Evaluation in Higher Education, 38(1), 44-60. https://doi.org/10.1080/02602938.2011.598636
• Stiggins, R. J. (1991). Assessment literacy. Phi Delta Kappan, 72(7), 534-539.
• Stiggins, R. J. (2007). Assessment through the student’s eyes. Educational Leadership, 64(8), 22-26.
• Stiggins, R. J. (2008). Student-involved assessment FOR learning (5th ed.). Upper Saddle River, NJ: Pearson Merrill Prentice Hall.
• Struyven, K., Dochy, F., & Janssens, S. (2005). Students’ perceptions about evaluation and assessment in higher education: a review. Assessment & Evaluation in Higher Education, 30(4), 325-341. https://doi.org/10.1080/02602930500099102
• Su, Y. (2014). Self-directed, genuine graduate attributes: the person-based approach. Higher Education Research & Development, 33(6), 1208-1220. https://doi.org/10.1080/07294360.2014.911255
• Tai, J., Ajjawi, R., Boud, D., Dawson, P., & Panadero, E. (2017). Developing evaluative judgement: enabling students to make decisions about the quality of work. Higher Education, 76(3), 467-481. https://doi.org/10.1007/s10734-017-0220-3
• Taylor, L. (2009). Developing Assessment Literacy. Annual Review of Applied Linguistics, 29, 21-36. https://doi.org/10.1017/s0267190509090035
• Taylor, L. (2013). Communicating the theory, practice, and principles of language testing to test stakeholders: Some reflections. Language Testing, 30(3), 403-412. https://doi.org/10.1177/0265532213480338
• Tian, W., Louw, S., & Khan, M. K. (2021). Covid-19 as a critical incident: Reflection on language assessment literacy and the need for radical changes. System, 103. https://doi.org/10.1016/j.system.2021.102682
• Vogt, K., & Tsagari, D. (2014). Assessment Literacy of Foreign Language Teachers: Findings of a European Study. Language Assessment Quarterly, 11(4), 374-402. https://doi.org/10.1080/15434303.2014.960046
• Volante, L., & Fazio, X. (2007). Exploring Teacher Candidates' Assessment Literacy: Implications for Teacher Education Reform and Professional Development [Abstract]. Canadian Journal of Education / Revue canadienne de l'éducation, 30(3). https://doi.org/10.2307/20466661
• Watanabe, Y. (2011). Teaching a course in assessment literacy to test takers: Its rationale, procedure, content, and effectiveness. Cambridge Research Notes, 46(Nov. 2011), 29-34.
• Weigle, S. C. (2007). Teaching writing teachers about assessment. Journal of Second Language Writing, 16(3), 194-209. https://doi.org/10.1016/j.jslw.2007.07.004
• White, E. (2009). Are you assessment literate? Some fundamental questions regarding effective classroom-based assessment. OnCUE Journal, 3(1), 3-25.
• Wiliam, D. (2018). Embedded formative assessment. Second edition. In Solution Tree Press.
• Xu, J., Zheng, Y., & Braund, H. (2023). Voices from L2 learners across different languages: Development and validation of a student writing assessment literacy scale. Journal of Second Language Writing, 60. https://doi.org/10.1016/j.jslw.2023.100993
• Yan, X., & Fan, J. (2020). “Am I qualified to be a language tester?”: Understanding the development of language assessment literacy across three stakeholder groups. Language Testing, 38(2), 219-246. https://doi.org/10.1177/0265532220929924
• Yorke, M. (2003). Formative assessment in higher education: Moves towards theory and enhancement of pedagogic practice, Higher Education, 45, 477–501.