Training EFL writers in self-assessment of source use: Why it fell short

Abstract

Self-assessment has attracted growing attention in language testing for its many benefits, such as fostering reflection, enhancing autonomy, and boosting confidence. This study investigated the effectiveness of training L2 writers to become proficient self-assessors of their source use ability. Ten secondary school EFL writers in China, ranging from beginner to high-intermediate levels, completed three integrated writing tasks, followed by self-assessment for each essay. Afterward, they received instructor assessment and feedback before submitting revisions for each task. A post-task interview was conducted to gather their perceptions of the training and self-assessment. Findings revealed that while some participants improved in source use, many struggled to benefit from self-assessment because they were confused about the evaluation criteria. For classroom practice, the study suggests that careful design of assessment tools, explicit modeling, scaffolding, and sustained teacher involvement are crucial for enabling students to engage meaningfully in self-assessment, thus promoting learning. For future research, this highlights the need to involve trained students, incorporate face-to-face interactions, use longitudinal designs, and explore how self-assessment can be adapted for different proficiency levels and how bilingual or first-language support may enhance its effectiveness.

Keywords

• Integrated skills assessment
• Source use
• Self-assessment
• EFL learners
• Secondary school

References

1. AlFallay, I. (2004). The role of some selected psychological and personality traits of the rater in the accuracy of self- and peer assessment. System, 32(3), 407 425. https://doi.org/10.1016/j.system.2004.04.006
2. Alverno College Faculty. (2000). Self assessment at Alverno College. Alverno College Institute.
3. Andrade, M.S., & Evans, N.W. (2012). Principles and practices for response in second language writing: Developing self-regulated learners. Routledge.
4. Bachman, L.F., & Palmer, A.S. (1989). The construct validation of self-ratings of communicative language ability. Language Testing, 6(1), 14-29. https://doi.org/10.1177/026553228900600104
5. Biggs, J. (1996). Enhancing teaching through constructive alignment. Higher Education, 32(3), 347 364. https://doi.org/10.1007/BF00138871
6. Boud, D. (2000). Sustainable assessment: rethinking assessment for the learning society. Studies in Continuing Education, 22(2), 151-167. https://doi.org/10.1080/713695728
7. Boud, D. (2003). Enhancing learning through self assessment. Routledge/Falmer. https://doi.org/10.1080/713695728
8. Brantmeier, C., Vanderplank, R., & Strube, M. (2012). What about me?: Individual self-assessment by skill and level of language instruction. System, 40(1), 144 160. https://doi.org/10.1016/j.system.2012.01.003

9. Butler, Y.G., & Lee, J. (2006). On-task versus off-task self-assessments among Korean elementary school students studying English. The Modern Language Journal, 90(4), 506 518. https://doi.org/10.1111/j.1540-4781.2006.00463.x
10. Carless, D. (2007). Learning‐oriented assessment: conceptual bases and practical implications. Innovations in education and teaching international, 44(1), 57 66. https://doi.org/10.1080/14703290601081332
11. Carless, D. (2015). Exploring learning oriented assessment processes. Higher Education, 69, 963 976. https://doi.org/10.1007/s10734-014-9816-z
12. Carless, D., Joughin, G., & Mok, M. (2006a). Learning-oriented assessment: Principles and practice. Assessment and evaluation in Higher Education, 31(4), 395 398. https://doi.org/10.1080/02602930600679043
13. Carless, D., Joughin, G., & Liu, N.F. (2006b). How assessment supports learning: Learning-oriented assessment in action (Vol. 1). Hong Kong University Press.
14. Cumming, A., Kantor, R., Baba, K., Erdosy, U., Eouanzoui, K., & James, M. (2005). Differences in written discourse in independent and integrated prototype tasks for next generation TOEFL. Assessing Writing, 10(1), 5 43. https://doi.org/10.1016/j.asw.2005.02.001
15. Cumming, A., Lai, C., & Cho, K. (2016). Students’ writing from sources for academic purposes: A synthesis of recent research. Journal of Second Language Writing, 31, 1 17. https://doi.org/10.1016/j.jslw.2016.06.001
16. De Saint-Leger, D. (2009). Self-assessment of speaking skills and participation in a foreign language class. Foreign Language Annals, 42, 158-178. https://doi.org/10.1111/j.1944-9720.2009.01013.x
17. De Saint-Leger, D., & Storch, N. (2009). Learners’ perceptions and attitudes: Implications for willingness to communicate in an L2 classroom. System, 37(2), 269 285. https://doi.org/10.1016/j.system.2009.01.001
18. Dochy, F., Segers, M., & Sluijsmans, D. (1999). The use of self-, peer and co-assessment in higher education: A review. Studies in Higher Education, 24(3), 331 350. https://doi.org/10.1080/03075079912331379935
19. Falchikov, N. (2006). Improving assessment through student involvement: Practical solutions for learning in higher and further education. Routledge/Falmer.
20. Frederiksen, J.R., & Collins, A. (1989). A systems approach to educational testing. Educational Researcher, 18(9), 27-32. https://doi.org/10.3102/0013189X018009027
21. Gebril, A., & Plakans, L. (2009). Investigating source use, discourse features, and process in integrated writing tests. Spaan Fellow Working Papers in Second or Foreign Language Assessment, 7(1), 47-84.
22. Gebril, A., & Plakans, L. (2016). Source-based tasks in academic writing assessment: Lexical diversity, textual borrowing and proficiency. Journal of English for academic purposes, 24, 78 88. https://doi.org/10.1016/j.jeap.2016.10.001
23. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81 112. https://doi.org/10.3102/003465430298487
24. Hauser, G.M., Koutouzos, D.W., & Olson, G.E. (2005). The standards-based school leader digital portfolios. International Journal of Learning, 12(7), 311-316.
25. Heilenman, L.K. (1990). Self-assessment of second language ability: The role of response effects. Language Testing, 7(2), 174-201. https://doi.org/10.1177/026553229000700204
26. Hirvela, A., & Du, Q. (2013). “Why am I paraphrasing?”: Undergraduate ESL writers’ engagement with source based academic writing and reading. Journal of English for Academic Purposes, 12(2), 87 98. https://doi.org/10.1016/j.jeap.2012.11.005
27. Jamieson, J.M., Eignor, D., Grabe, W., & Kunnan, A.J. (2008). Frameworks for a New TOEFL. In C. Chapelle, M. Enright, & J. Jamieson (Eds.), Building a validity argument for the Test of English as a Foreign Language (pp. 55-96). Routledge.
28. Jones, N., & Saville, N. (2016). Learning oriented assessment: A systemic approach. Cambridge University Press..
29. Keck, C. (2006). The use of paraphrase in summary writing: A comparison of L1 and L2 writers. Journal of Second Language Writing, 15(4), 261-278. https://doi.org/10.1016/j.jslw.2006.09.006
30. Knoch, U., Read, J., & von Randow, J. (2007). Re-training writing raters online: How does it compare with face to face training? Assessing Writing, 12(1), 26 43. https://doi.org/10.1016/j.asw.2007.04.001
31. Lee, I. (2007). Assessment for learning: Integrating assessment, teaching, and learning in the ESL/EFL writing classroom. The Canadian Modern Language Review, 64(1), 199 213. https://doi.org/10.3138/cmlr.64.1.199
32. Liu, X., & Li, L. (2014). Assessment training effects on student assessment skills and task performance in a technology facilitated peer assessment. Assessment & Evaluation in Higher Education, 39(3), 275 292. https://doi.org/10.1080/02602938.2013.823540
33. Lowry, C. (2016). Choosing & using sources: A guide to academic research. Ohio State University.
34. MacIntyre, P.D., Noels, K.A., & Clement, R. (1997). Biases in self-ratings of second language proficiency: The role of language anxiety. Language Learning, 47(2), 265 287. https://doi.org/10.1111/0023-8333.81997008
35. Miles, M.B., Huberman, A.M., & Saldaña, J. (2014). Qualitative data analysis: A methods sourcebook (3rd ed.). SAGE Publications.
36. Min, H.T. (2005). Training students to become successful peer reviewers. System, 33(2), 293 308. https://doi.org/10.1016/j.system.2004.11.003
37. Office of the Vice President for Research. (2020, March). Temporary policy on human subjects research related visits during COVID 19 outbreak effective March 16. The University of Iowa. https://research.uiowa.edu/news/2020/03/temporary-policy-human-subjects-research-related-visits-during-covid-19-outbreak
38. Oscarson, M. (1997). Self-assessment of foreign and second language proficiency. In C. Clapham & D. Corson (Eds.), The encyclopedia of language and education. Vol. 7: Language testing and assessment (pp. 175-87). Kluwer.
39. Oscarson, M. (2013). Self-assessment in the classroom. In A. J. Kunnan (Ed.), The companion to language assessment: Approaches and development (pp. 712 729). John Wiley & Sons, Inc. https://doi.org/10.1002/9781118411360.wbcla046
40. Oskarsson, M. (1978). Approaches to self-assessment in foreign language learning. Council of Europe, Council for Cultural Cooperation.
41. Patri, M. (2002). The influence of peer feedback on self-assessment and peer assessment of oral skills. Language Testing, 19(2), 109-131. https://doi.org/10.1191/0265532202lt224oa
42. Plakans, L. (2008). Comparing composing processes in writing-only and reading-to-write test tasks. Assessing Writing, 13(2), 111-129. https://doi.org/10.1016/j.asw.2008.07.001
43. Plakans, L. (2009a). Discourse synthesis in integrated second language writing assessment. Language Testing, 26(4), 561-587. https://doi.org/10.1177/0265532209340192
44. Plakans, L. (2009b). The role of reading strategies in integrated L2 writing tasks. Journal of English for Academic Purposes, 8(4), 252-266. https://doi.org/10.1016/j.jeap.2009.09.002
45. Plakans, L., & Gebril, A. (2013). Using multiple texts in an integrated writing assessment: Source text use as a predictor of score. Journal of Second Language Writing, 22(3), 217 230. https://doi.org/10.1016/j.jslw.2013.02.003
46. Plakans, L., & Gebril, A. (2015). Assessment myths: Applying second language research to classroom teaching. University of Michigan Press. https://doi.org/10.3998/mpub.5056216
47. Plakans, L., Liao, J.T., & Wang, F. (2019). “I should summarize this whole paragraph”: Shared processes of reading and writing in iterative integrated assessment tasks. Assessing Writing, 40, 14 26. https://doi.org/10.1016/j.asw.2019.03.003
48. Purpura, J.E. (2016). Second and foreign language assessment. Modern Language Journal, 100(Supplement), 190-208. https://doi.org/10.1111/modl.12308
49. Rahimi, M. (2013). Is training student reviewers worth its while? A study of how training influences the quality of students’ feedback and writing. Language Teaching Research, 17(1), 67 89. https://doi.org/10.1177/1362168812459151
50. Rolheiser, C. (Ed.). (1996). Self-evaluation: Helping students get better at it. Visutronx.
51. Rolheiser, C., & Ross, J.A. (2001). Student self-evaluation: What research says and what practice shows. In R.D. Small & A.P. Thomas (Eds.), PlainTalk about education: Transforming crisis into success (pp. 43-57). Center for Development & Learning.
52. Ross, J.A. (2006). The reliability, validity, and utility of self-assessment. Practical Assessment, Research, and Evaluation, 11(1), Article 10. https://doi.org/10.7275/9wph-vv65
53. Ross, S. (1998). Self-assessment in second language testing: A meta-analysis and analysis of experiential factors. Language Testing, 15(1), 1-20. https://doi.org/10.1177/026553229801500101
54. Saldaña, J. (2015). The coding manual for qualitative researchers. Sage Publication.
55. Sadler, R. (2002) Ah! … So that’s ‘quality’, In P. Schwartz & G. Webb (Eds), Assessment: case studies, experience and practice from higher education. Kogan Page.
56. Schuessler, J.N. (2010). Self assessment as learning: Finding the motivations and barriers for adopting the learning-oriented instructional design of student self assessment [Unpublished doctoral dissertation]. Capella University.
57. Shi, L. (2010). Plagiarism in second language writing. Language Teaching, 43(3), 269 296. https://doi.org/10.1017/S0261444814000056
58. Suzuki, Y. (2015). Self-assessment of Japanese as a second language: The role of experiences in the naturalistic acquisition. Language Testing, 32(1), 63 81. https://doi.org/10.1177/0265532214541885
59. Weigle, S.C. (2004). Integrating reading and writing in a competency test for non-native speakers of English. Assessing Writing, 9(1), 27-55. https://doi.org/10.1016/j.asw.2004.01.002
60. Weir, C.J. (2005). Language testing and validation: An evidence based approach. Palgrave Macmillan. https://doi.org/10.1057/9780230514577
61. Wright, W.E. (2019). Foundations for teaching English language learners: Research, theory, policy, and practice (3rd ed.). Caslon Publishing.
62. Zhang, P., Fan, J., & Jia, W. (2022). Assessing L2 integrated writing self-efficacy: Development and validation of a scale. Assessing Writing, 54, Article 100665. https://doi.org/10.1016/j.asw.2022.100665