Türk Yükseköğretiminde Üniversite Hazırlık Öğrencilerinin Yazma Değerlendirme Okuryazarlığının Derinlemesine İncelenmesi

Öz

Yeni gelişmekte olan bir araştırma alanı olarak, öğrencilerin Öğrenci Yazma Değerlendirme Okuryazarlığı (SWAL) ile ilgili dil değerlendirme okuryazarlık düzeylerini, algılarını, inançlarını ve uygulamalarını inceleyen araştırma sayısı oldukça azdır. Bu araştırma boşluğunu gidermek için, bu çalışma, yarı yapılandırılmış görüşmelerin yanı sıra SWAL Ölçeğini (Xu vd., 2023) içeren karma bir yöntem tasarımını benimsemiştir. 134 üniversite hazırlık sınıfı öğrencisi ölçeği yanıtlarken, on iki öğrenci de görüşmelere katılmak için gönüllü olmuştur. Öğrencilerin ölçekten aldıkları yanıtlar tanımlayıcı istatistikler aracılığıyla analize tabi tutulmuş ve öğrencilerin değerlendirme hedeflerini anlamaları ile öğrenmeye yönelik olumlu tutum sergilemeleri arasında kayda değer bir korelasyon olduğu ortaya çıkmıştır. Bununla birlikte, öğrencilerin teorik anlayışları ve inançları ile yazılı ödevlere ilişkin pragmatik uygulamaları arasında bir uyumsuzluk olduğu fark edilmiştir. Yarı-yapılandırılmış görüşme verileri tematik analiz yoluyla analiz edilmiş ve dört ana tema ortaya çıkmıştır: (1) geribildirim değerlemesi, (2) puanlama kriterleri farkındalığı, (3) geribildirim verme tercihleri ve (4) alternatif değerlendirmeden kaçınma. Bulgular, öğrencileri değerlendirme sürecine dahil etmenin önemini vurgulamakta ve SWAL’ı geliştirmek için öğretmen eğitimi girişimlerini savunmaktadır. Gelecekteki araştırmalar, farklı akademik seviyelere ulaşmak ve notları bir değişken olarak entegre etmek için kapsamlarını genişletebilir. Genel olarak, mevcut çalışma SWAL konusunda değerli bilgiler sunmakta ve yükseköğretim düzeyindeki değerlendirme uygulamalarına ışık tutmaktadır.

Anahtar Kelimeler

• değerlendirme bilgisi
• değerlendirme okuryazarlığı
• öğrenci yazma değerlendirme okuryazarlığı
• yazma değerlendirmesi
• yazma değerlendirme okuryazarlığı

An In-Depth Examination of Writing Assessment Literacy Among University Preparatory Students in Turkish Higher Education

Öz

As a nascent research domain, there is a scarcity of research studies that examine the levels of assessment literacy, perceptions, beliefs, and practices among students concerning their Student Writing Assessment Literacy (SWAL). To address this research gap, the present study adopts a mixed-methods design, incorporating the Scale of SWAL (Xu et al., 2023) alongside semi-structured interviews. 134 students answered the scale, while an additional twelve students volunteered to participate in the interviews. Student responses from the scale were subjected to analysis through descriptive statistics, revealing a notable correlation between students’ comprehension of assessment objectives and the manifestation of positive attitudes towards learning. Nevertheless, a misalignment was noticed between their theoretical understanding and beliefs and their pragmatic implementation regarding written assignments. Interview data underwent analysis through thematic analysis, elucidating four main themes: (1) feedback valuation, (2) scoring criteria awareness, (3) feedback delivery preferences, and (4) alternative assessment avoidance. Implications emphasise the importance of involving students in the assessment process and advocate for teacher training initiatives to enhance SWAL. Future research could broaden their scope to reach diverse academic levels and integrate grades as a variable. Overall, the present study provides valuable insights into SWAL, shedding light on assessment practices at the tertiary level.

Anahtar Kelimeler

• assessment knowledge
• assessment literacy
• student writing assessment literacy
• writing assessment
• writing assessment literacy

Kaynakça

1. Absolum, M., Flockton, L., Hattie, J., Hipkins, R., & Reid, I. (2009). Directions for assessment in New Zealand: Developing students’ assessment capabilities. Unpublished paper prepared for the Ministry of Education.
2. Aitken, N. (2011). Student Voice in Fair Assessment Practice. In Leading Student Assessment (pp. 175-200). Springer. https://doi.org/10.1007/978-94-007-1727-5_9
3. Alonzo, D. (2016). Development and Application of a Teacher Assessment for Learning (AfL) Literacy Tool. unsworks.unsw.edu.au. https://doi.org/10.26190/unsworks/18809
4. Ataie-Tabar, M., Zareian, G., Amirian, S. M. R., & Adel, S. M. R. (2019). A Study of Socio-Cultural Conception of Writing Assessment Literacy: Iranian EFL Teachers’ and Students’ Perspectives. English Teaching & Learning, 43(4), 389-409. https://doi.org/10.1007/s42321-019-00035-0
5. Baird, J.-A., Andrich, D., Hopfenbeck, T. N., & Stobart, G. (2017). Assessment and learning: fields apart? Assessment in Education: Principles, Policy & Practice, 24(3), 317-350. https://doi.org/10.1080/0969594x.2017.1319337
6. Bernard, H. R. (2018). Research methods in anthropology: qualitative and quantitative approaches (Sixth edition. ed.). Rowman & Littlefield.
7. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa
8. Braun, V., & Clarke, V. (2012). Thematic analysis. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research methods in psychology, Vol. 2. Research designs: Quantitative, qualitative, neuropsychological, and biological (pp. 57–71). American Psychological Association. https://doi.org/10.1037/13620-004

9. Brown, G. T. L., & Wang, Z. (2016). Understanding Chinese university student conceptions of assessment: cultural similarities and jurisdictional differences between Hong Kong and China. Social Psychology of Education, 19(1), 151-173. https://doi.org/10.1007/s11218-015-9322-x
10. Butler, Y. G., Peng, X., & Lee, J. (2021). Young learners’ voices: Towards a learner-centered approach to understanding language assessment literacy. Language Testing, 38(3), 429-455. https://doi.org/10.1177/0265532221992274
11. Carless, D., Salter, D., Yang, M., & Lam, J. (2010). Developing sustainable feedback practices. Studies in Higher Education, 36(4), 395-407. https://doi.org/10.1080/03075071003642449
12. Chan, C. K. Y., & Luo, J. (2021). A four-dimensional conceptual framework for student assessment literacy in holistic competency development. Assessment & Evaluation in Higher Education, 46(3), 451-466. https://doi.org/10.1080/02602938.2020.1777388
13. Chen, G., Wang, X., & Wang, L. (2023). Developing assessment literacy among trainee translators: scaffolding self and peer assessment as an intervention. Assessment & Evaluation in Higher Education, 48(6), 888-902. https://doi.org/10.1080/02602938.2022.2142515
14. Charteris, J., & Thomas, E. (2016). Uncovering ‘unwelcome truths’ through student voice: teacher inquiry into agency and student assessment literacy. Teaching Education, 28(2), 162-177. https://doi.org/10.1080/10476210.2016.1229291
15. Colby-Kelly, C., & Turner, C. E. (2007). AFL Research in the L2 Classroom and Evidence of Usefulness: Taking Formative Assessment to the Next Level. The Canadian Modern Language Review, 64(1), 9-37. https://doi.org/10.3138/cmlr.64.1.009
16. Corbin, J. M., & Strauss, A. (1990). Grounded theory research: Procedures, canons, and evaluative criteria. Qualitative Sociology, 13(1), 3-21. https://doi.org/10.1007/bf00988593
17. Creswell, J. W., & Creswell, J. D. (2018). Research design: qualitative, quantitative, and mixed methods approaches (Fifth edition. ed.). SAGE.
18. Crusan, D. (2010). Assessment in the second language writing classroom. University of Michigan Press.
19. Crusan, D., Plakans, L., & Gebril, A. (2016). Writing assessment literacy: Surveying second language teachers’ knowledge, beliefs, and practices. Assessing Writing, 28, 43-56. https://doi.org/10.1016/j.asw.2016.03.001
20. Crusan, D. (2022). Writing Assessment Literacy. In Research Questions in Language Education and Applied Linguistics (pp. 431-435). https://doi.org/10.1007/978-3-030-79143-8_77
21. Davari Torshizi, M., & Bahraman, M. (2019). I explain, therefore I learn: Improving students’ assessment literacy and deep learning by teaching. Studies in Educational Evaluation, 61, 66-73. https://doi.org/10.1016/j.stueduc.2019.03.002
22. Davies, A. (2008). Textbook trends in teaching language testing. Language Testing, 25(3), 327-347. https://doi.org/10.1177/0265532208090156
23. Deeley, S. J., & Bovill, C. (2017). Staff student partnership in assessment: enhancing assessment literacy through democratic practices. Assessment & Evaluation in Higher Education, 42(3), 463-477. https://doi.org/10.1080/02602938.2015.1126551
24. DeLuca, C., & Klinger, D. A. (2010). Assessment literacy development: identifying gaps in teacher candidates’ learning. Assessment in Education: Principles, Policy & Practice, 17(4), 419-438. https://doi.org/10.1080/0969594x.2010.516643
25. Denton, P., & McIlroy, D. (2017). Response of students to statement bank feedback: the impact of assessment literacy on performances in summative tasks. Assessment & Evaluation in Higher Education, 43(2), 197-206. https://doi.org/10.1080/02602938.2017.1324017
26. Dörnyei, Z. (2007). Research methods in applied linguistics. Oxford University Press.
27. Dörnyei, Z., & Dewaele, J.-M. (2022). Questionnaires in Second Language Research. Routledge.
28. Flores, M. A., Brown, G., Pereira, D., Coutinho, C., Santos, P., & Pinheiro, C. (2019). Portuguese university students’ conceptions of assessment: taking responsibility for achievement. Higher Education, 79(3), 377-394. https://doi.org/10.1007/s10734-019-00415-2
29. Francis, R. A. (2008). An investigation into the receptivity of undergraduate students to assessment empowerment. Assessment & Evaluation in Higher Education, 33(5), 547-557. https://doi.org/10.1080/02602930701698991
30. Fulcher, G. (2010). Practical language testing. London: Hodder Education.
31. Fulcher, G. (2012). Assessment Literacy for the Language Classroom. Language Assessment Quarterly, 9(2), 113-132. https://doi.org/10.1080/15434303.2011.642041
32. Gipps, C. (1999). Socio-Cultural Aspects of Assessment. Review of Research in Education, 24. https://doi.org/10.2307/1167274
33. Greene, J. C. (2007). Mixed methods in social inquiry. San Francisco, CA: Jossey-Bass.
34. Hannigan, C., Alonzo, D., & Oo, C. Z. (2022). Student assessment literacy: indicators and domains from the literature. Assessment in Education: Principles, Policy & Practice, 29(4), 482-504. https://doi.org/10.1080/0969594x.2022.2121911
35. Hattie, J., & Timperley, H. (2016). The Power of Feedback. Review of Educational Research, 77(1), 81-112. https://doi.org/10.3102/003465430298487
36. Hawe, E. M., & Dixon, H. R. (2014). Building students’ evaluative and productive expertise in the writing classroom. Assessing Writing, 19, 66-79. https://doi.org/10.1016/j.asw.2013.11.004
37. Hirvela, A., & Belcher, D. (2007). Writing scholars as teacher educators: Exploring writing teacher education. Journal of Second Language Writing, 16(3), 125-128. https://doi.org/10.1016/j.jslw.2007.08.001
38. Homayounzadeh, Z., & Razmjoo, S. A. (2021). Examining Assessment Literacy in Practice in an Iranian Context: Does it Differ for Instructors and Learners? Teaching English as a Second Language Quarterly (Formerly Journal of Teaching Language Skills), 40(2), 1-45.
39. Inbar-Lourie, O. (2008). Constructing a language assessment knowledge base: A focus on language assessment courses. Language Testing, 25(3), 385-402. https://doi.org/10.1177/0265532208090158
40. Inbar-Lourie, O. (2013). Guest Editorial to the special issue on language assessment literacy. Language Testing, 30(3), 301-307. https://doi.org/10.1177/0265532213480126
41. Jeong, H. (2013). Defining assessment literacy: Is it different for language testers and non-language testers? Language Testing, 30(3), 345-362. https://doi.org/10.1177/0265532213480334
42. Lam, R. (2019). Teacher assessment literacy: Surveying knowledge, conceptions, and practices of classroom-based writing assessment in Hong Kong. System, 81, 78-89. https://doi.org/10.1016/j.system.2019.01.006
43. Lee, I. (2017). Classroom Writing Assessment and Feedback in L2 School Contexts. Springer. https://doi.org/10.1007/978-981-10-3924-9
44. Lee, I., & Mao, Z. (2024). Writing teacher feedback literacy: Surveying second language teachers’ knowledge, values, and abilities. Journal of Second Language Writing, 63. https://doi.org/10.1016/j.jslw.2024.101094
45. Yu, S., & Lee, I. (2024). Writing assessment and feedback literacy: Where do we stand and where can we go? Assessing Writing, 60. https://doi.org/10.1016/j.asw.2024.100829
46. Lodge, C. (2008). Engaging student voice to improve pedagogy and learning: An exploration of examples of innovative pedagogical approaches for school improvement. International Journal of Pedagogies and Learning, 4(5), 4-19. https://doi.org/10.5172/ijpl.4.5.4
47. Mackey, A., & Gass, S. M. (2022). Second language research: methodology and design (2nd ed.). Routledge.
48. Mellati, M., & Khademi, M. (2018). Exploring Teachers’ Assessment Literacy: Impact on Learners’ Writing Achievements and Implications for Teacher Development. Australian Journal of Teacher Education, 43(6), 1-18. https://doi.org/10.14221/ajte.2018v43n6.1
49. O’Donovan, B., Price, M., & Rust, C. (2001). The Student Experience of Criterion-Referenced Assessment (Through the Introduction of a Common Criteria Assessment Grid). Innovations in Education and Teaching International, 38(1), 74-85. https://doi.org/10.1080/147032901300002873
50. O’Donovan, B., Price, M., & Rust, C. (2004). Know what I mean? Enhancing student understanding of assessment standards and criteria. Teaching in Higher Education, 9(3), 325-335. https://doi.org/10.1080/1356251042000216642
51. O’Donovan, B., Price, M., & Rust, C. (2008). Developing student understanding of assessment standards: a nested hierarchy of approaches. Teaching in Higher Education, 13(2), 205-217. https://doi.org/10.1080/13562510801923344
52. Ölmezer-Öztürk, E., & Aydin, B. (2018). Toward measuring language teachers’ assessment knowledge: development and validation of Language Assessment Knowledge Scale (LAKS). Language Testing in Asia, 8(1). https://doi.org/10.1186/s40468-018-0075-2
53. Panadero, E., Jonsson, A., & Strijbos, J.-W. (2016). Scaffolding Self-Regulated Learning Through Self-Assessment and Peer Assessment: Guidelines for Classroom Implementation. In C. Wyatt-Smith (Ed.), Assessment for Learning: Meeting the Challenge of Implementation (pp. 311-326). https://doi.org/10.1007/978-3-319-39211-0_18
54. Pereira, D., Niklasson, L., & Flores, M. A. (2016). Students’ perceptions of assessment: a comparative analysis between Portugal and Sweden. Higher Education, 73(1), 153-173. https://doi.org/10.1007/s10734-016-0005-0
55. Peterson, E. R., Brown, G. T. L., & Jun, M. C. (2015). Achievement emotions in higher education: A diary study exploring emotions across an assessment event. Contemporary Educational Psychology, 42, 82-96. https://doi.org/10.1016/j.cedpsych.2015.05.002
56. Popham, W. J. (2009). Assessment Literacy for Teachers: Faddish or Fundamental? Theory Into Practice, 48(1), 4-11. https://doi.org/10.1080/00405840802577536
57. Rezai, A., Alibakhshi, G., Farokhipour, S., & Miri, M. (2021). A phenomenographic study on language assessment literacy: hearing from Iranian university teachers. Language Testing in Asia, 11(1). https://doi.org/10.1186/s40468-021-00142-5
58. Roslan, R., Nishio, Y., & Jawawi, R. (2022). Analysing English language teacher candidates’ assessment literacy: A case of Bruneian and Japanese universities. System, 111. https://doi.org/10.1016/j.system.2022.102934
59. Rust, C., Price, M., & O’Donovan, B. (2003). Improving Students’ Learning by Developing their Understanding of Assessment Criteria and Processes. Assessment & Evaluation in Higher Education, 28(2), 147-164. https://doi.org/10.1080/02602930301671
60. Sadler, D. R. (2010). Beyond feedback: developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35(5), 535-550. https://doi.org/10.1080/02602930903541015
61. Scarino, A. (2013). Language assessment literacy as self-awareness: Understanding the role of interpretation in assessment and in teacher learning. Language Testing, 30(3), 309-327. https://doi.org/10.1177/0265532213480128
62. Shepard, L. A. (2000). The role of assessment in a learning culture. Teaching and Learning, 229–253.
63. Smith, C. D., Worsfold, K., Davies, L., Fisher, R., & McPhail, R. (2013). Assessment literacy and student learning: the case for explicitly developing students ‘assessment literacy.’ Assessment & Evaluation in Higher Education, 38(1), 44-60. https://doi.org/10.1080/02602938.2011.598636
64. Stiggins, R. J. (1991). Assessment literacy. Phi Delta Kappan, 72(7), 534-539.
65. Stiggins, R. J. (2007). Assessment through the student’s eyes. Educational Leadership, 64(8), 22-26.
66. Stiggins, R. J. (2008). Student-involved assessment FOR learning (5th ed.). Upper Saddle River, NJ: Pearson Merrill Prentice Hall.
67. Struyven, K., Dochy, F., & Janssens, S. (2005). Students’ perceptions about evaluation and assessment in higher education: a review. Assessment & Evaluation in Higher Education, 30(4), 325-341. https://doi.org/10.1080/02602930500099102
68. Su, Y. (2014). Self-directed, genuine graduate attributes: the person-based approach. Higher Education Research & Development, 33(6), 1208-1220. https://doi.org/10.1080/07294360.2014.911255
69. Tai, J., Ajjawi, R., Boud, D., Dawson, P., & Panadero, E. (2017). Developing evaluative judgement: enabling students to make decisions about the quality of work. Higher Education, 76(3), 467-481. https://doi.org/10.1007/s10734-017-0220-3
70. Taylor, L. (2009). Developing Assessment Literacy. Annual Review of Applied Linguistics, 29, 21-36. https://doi.org/10.1017/s0267190509090035
71. Taylor, L. (2013). Communicating the theory, practice, and principles of language testing to test stakeholders: Some reflections. Language Testing, 30(3), 403-412. https://doi.org/10.1177/0265532213480338
72. Tian, W., Louw, S., & Khan, M. K. (2021). Covid-19 as a critical incident: Reflection on language assessment literacy and the need for radical changes. System, 103. https://doi.org/10.1016/j.system.2021.102682
73. Vogt, K., & Tsagari, D. (2014). Assessment Literacy of Foreign Language Teachers: Findings of a European Study. Language Assessment Quarterly, 11(4), 374-402. https://doi.org/10.1080/15434303.2014.960046
74. Volante, L., & Fazio, X. (2007). Exploring Teacher Candidates' Assessment Literacy: Implications for Teacher Education Reform and Professional Development [Abstract]. Canadian Journal of Education / Revue canadienne de l'éducation, 30(3). https://doi.org/10.2307/20466661
75. Watanabe, Y. (2011). Teaching a course in assessment literacy to test takers: Its rationale, procedure, content, and effectiveness. Cambridge Research Notes, 46(Nov. 2011), 29-34.
76. Weigle, S. C. (2007). Teaching writing teachers about assessment. Journal of Second Language Writing, 16(3), 194-209. https://doi.org/10.1016/j.jslw.2007.07.004
77. White, E. (2009). Are you assessment literate? Some fundamental questions regarding effective classroom-based assessment. OnCUE Journal, 3(1), 3-25.
78. Wiliam, D. (2018). Embedded formative assessment. Second edition. In Solution Tree Press.
79. Xu, J., Zheng, Y., & Braund, H. (2023). Voices from L2 learners across different languages: Development and validation of a student writing assessment literacy scale. Journal of Second Language Writing, 60. https://doi.org/10.1016/j.jslw.2023.100993
80. Yan, X., & Fan, J. (2020). “Am I qualified to be a language tester?”: Understanding the development of language assessment literacy across three stakeholder groups. Language Testing, 38(2), 219-246. https://doi.org/10.1177/0265532220929924
81. Yorke, M. (2003). Formative assessment in higher education: Moves towards theory and enhancement of pedagogic practice, Higher Education, 45, 477–501.