Employing G-theory and rater interviews, the study investigated how a high-stakes writing assessment procedure (i.e., a single-task, single-rater, and holistic scoring procedure) impacted the variability and reliability of its scores within the Turkish higher education context. Thirty-two essays written on two different writing tasks (i.e., narrative and opinion) by 16 EFL students studying at a Turkish state university were scored by 10 instructor raters both holistically and analytically. After the raters completed the scoring procedure, semi-structured individual interviews were held with them to gain insight into their views regarding the quality of the current scoring procedure. The G-theory results showed that the reliability coefficients obtained from the current scoring procedure would not be sufficient to draw sound conclusions. The quantitative results were partly supported by the qualitative data. Important implications were discussed to improve the quality of the current high-stakes EFL writing assessment policy.
EFL writing assessment Writing assessment in higher education Scoring variability Scoring reliability Generalizability (G-) theory
The study has not been supported by any institutions.
Employing G-theory and rater interviews, the study investigated how a high-stakes writing assessment procedure (i.e., a single-task, single-rater, and holistic scoring procedure) impacted the variability and reliability of its scores within the Turkish higher education context. Thirty-two essays written on two different writing tasks (i.e., narrative and opinion) by 16 EFL students studying at a Turkish state university were scored by 10 instructor raters both holistically and analytically. After the raters completed the scoring procedure, semi-structured individual interviews were held with them to gain insight into their views regarding the quality of the current scoring procedure. The G-theory results showed that the reliability coefficients obtained from the current scoring procedure would not be sufficient to draw sound conclusions. The quantitative results were partly supported by the qualitative data. Important implications were discussed to improve the quality of the current high-stakes EFL writing assessment policy.
EFL writing assessment Writing assessment in higher education Scoring variability Scoring reliability Generalizability (G-) theory
Birincil Dil | İngilizce |
---|---|
Konular | Eğitimde Ölçme ve Değerlendirme (Diğer) |
Bölüm | Makaleler |
Yazarlar | |
Erken Görünüm Tarihi | 21 Ekim 2024 |
Yayımlanma Tarihi | |
Gönderilme Tarihi | 1 Kasım 2023 |
Kabul Tarihi | 26 Ağustos 2024 |
Yayımlandığı Sayı | Yıl 2024 Cilt: 11 Sayı: 4 |