The effects of learning oriented assessment on academic writing

Abstract

The purpose of this study is to investigate the impact of learning-oriented assessment (LOA) on the academic writing ability of EFL students (N:40) during a 12-week in the semester of 2019-2020 academic year in the context of a higher education. Within a pretest-posttest intact group design, the experimental group received instruction following the principles of LOA, and the comparison group received routine procedures for academic writing. This is a quantitative experimental design. The test of normality, Kolmogorov-Smirnov, Mann-Whitney Tests and Wilcoxon Signed Ranks Test were administered in order to see the significance of the intervention the data for this study included scores of a pretest, several assessments, and a posttest at the beginning, during, and at the end of instruction, respectively. The prompt for both pre and post-tests required participants to write argumentative essays. To rate the writing tasks, we followed the rubrics developed by the testing office of the institution. The findings revealed that the experimental group outscored the comparison group indicating the effectiveness of LOA procedures in student learning. Further, the findings indicated that implementing LOA could have significant implications and applications for EFL writing education.

Keywords

• Learning Oriented Assessment
• Academic Writing Ability
• Argumentative Essay

References

1. Alghasab, M., & Handley, Z. (2017). Capturing (non-) collaboration in wiki-mediated collaborative writing activities: The need to examine discussion posts and editing acts in tandem. Computer Assisted Language Learning, 30(7), 664 691. https://doi.org/10.1080/09588221.2017.1341928
2. Arslan, R. (2014). Integrating feedback into prospective English language teachers’ writing process via blogs and portfolios. The Turkish Online Journal of Educational Technology, 13(1), 131-150.
3. Ashton, K., Salamoura, A., & Diaz, E. (2012). The BEDA impact project: A preliminary investigation of a bilingual programme in Spain. Research notes, 50, 34-41.
4. Bader, M., Burner, T., Iversen S., & Varga Z. (2019). Student perspectives on formative feedback as part of writing portfolios. Assessment & Evaluation in Higher Education, 44(7), 1017-1028. https://doi.org/10.1080/02602938.2018.1564811
5. Barker, (2013). Learning oriented assessment. Retrieved from: https://www.cambridgeenglish.org/research-and-validation/fitness-for-purpose/loa/
6. Bawarshi, A., & Reiff, M.J. (2010). Genre: An introduction to the history, theory research, and pedagogy. Parlor press/WAC Clearinghouse.
7. Beuningen, V., De Jong, N.H., & Kuiken, F. (2012). Evidence on the effectiveness of comprehensive error correction in second language writing. Language Learning, 62, 1-41. https://doi.org/10.1111/j.1467-9922.2011.00674.x
8. Boud, D., & Falchinov, N. (2006). Aligning assessment with long-term learning. Assessment and Evaluation in Higher Education, 31(4), 399 413. https://doi.org/10.1080/02602930600679050

9. Carless, D. (2006). Differing perceptions in the feedback process. Studies in Higher Education, 31(2), 219–233.
10. Carless, D. (2007). Learning‐oriented assessment: Conceptual bases and practical implications. Innovations in Education and Teaching International, 44(1), 57 66. https://doi.org/10.1080/14703290601081332
11. Carless, D. (2009). Learning Oriented Assessment: Principles, practice and a project. In Meyer, L.H, Davidson, H, Anderson, R, Fletcher, PM, Johnston & Rees, M. (Eds), Tertiary Assessment & Higher Education Student Outcomes: Policy, Practice & Research. Wellington.
12. Carless, D. (2014). Exploring learning-oriented assessment processes. Higher Education, 69, pp.2-5. https://doi.org/10.1007/s10734-014-9816-z
13. Carless, D., Joughin, G., & Liu, N. (2006). How assessment supports learning: Learning oriented assessment in action. Hong Kong University Press.
14. Chao, Y.C.J., & Lo, H.C. (2011). Students' Perceptions of Wiki-Based Collaborative Writing for Learners of English as a Foreign Language. Interactive Learning Environments, 19(4), 395-411, https://www.learntechlib.org/p/167206/.
15. Chen, T. (2016). Technology-supported peer feedback in ESL/EFL writing classes: A research synthesis. Computer Assisted Language Learning, 29(2), 365 397, https://doi.org/10.1080/09588221.2014.960942
16. Cho, H. (2017). Synchronous web-based collaborative writing: Factors mediating interaction among second-language writers, University of Toronto. Journal of Second Language Writing. 36, 37-51, https://doi.org/10.1016/j.jslw.2017.05.013
17. Cook, T.D., & Campbell, D.T. (1979). Quasi-experimentation: Design & analysis issues in field settings. Houghton Mifflin.
18. Dalrymple, R., & Smith, P. (2008). Patchwork Text: Enabling discursive writing and reflective practice on a foundation module in work-based learning. Innovations in Education and Teaching International, 45(1), 47-54. https://doi.org/10.1080/14703290701757443
19. Elola, I., & Oskoz, A. (2010). Collaborative writing: Fostering foreign language and writing conventions development. Language Learning & Technology, 14(3), 51–71, http://llt.msu.edu/vol14num3/elolaoskoz.pdf.
20. Eridafithri, E. (2015). The Application of portfolios to assess progress in writing of EFL students at secondary schools in banda aceh. Studies in English Language and Education, 2, 1-16. https://doi.org/10.24815/siele.v2i1.2231
21. Farahian M., & Avarzamani F. (2018). The impact of portfolio on EFL learners’ metacognition and writing performance. Cogent Education, 5(1). https://doi.org/10.1080/2331186X.2018.1450918
22. Gielen, S., Peeters, E., Dochy, F., Onghena, P., & Struyven, K. (2010). Improving the effectiveness of peer feedback for learning. Learning and Instruction, 20(4), 304–315. https://doi.org/10.1016/j.learninstruc.2009.08.007
23. Grabe, W., & Kaplan, R.B. (1996). Theory and Practice of Writing: An Applied Linguistic Perspective. Longman, New York, p. 91, 93, 114, 115, 202, 245.
24. Grabe, W., & Zhang C. (2016). Reading-writing relationships in first and second language academic literacy development. Language Teaching, 49, 339 355 https://doi.org/10.1017/S0261444816000082
25. Green, A. (2017). Learning-oriented language test preparation materials: A contradiction in terms?. Papers in Language Testing and Assessment, 6(1), 112 132. http://hdl.handle.net/10547/622430
26. Gibbons, S., & Kankkonen, B. (2011). Assessment as learning in physical education: Making assessment meaningful for secondary school students, Physical and Health Education Journal, 76(4), 6–12.
27. Hamp-Lyons, L. (2017). Language assessment literacy for language learning-oriented assessment. Papers in Language Testing and Assessment, 6(1), 88-110. http://hdl.handle.net/10547/622445
28. Han, Y., & Hyland, F. (2015). Exploring learner engagement with written corrective feedback in a Chinese tertiary EFL classroom. Journal of Second Language Writing, 30, 31–44. https://doi.org/10.1016/j.jslw.2015.08.002
29. Hayes, J.R., & Chenoweth, N.A. (2001). Fluency in Writing: Generating Text in L1 and L2. Written Communication, 18(1), 80–98. https://doi.org/10.1177/0741088301018001004
30. Hayes, J.R., & Flower, L.S. (1980). Identifying the organization of writing processes. In L. Gregg and E. R. Steinberg (Eds.), Cognitive Processes in Writing (pp. 3-30). Lawrence Erlbaum.
31. Horstmanshof, L., & Brownie, S. (2013). 'A scaffolded approach to discussion board use for formative assessment of academic writing skills'. Assessment & Evaluation in Higher Education, 38(1)1, 61-73, https://www.learntechlib.org/p/132972/.
32. Huang, F.Q. (2003). On curriculum for learning: review from perspective of cultural Philosophy. Peking University Education Review, 4(90–94), 99.
33. Hume, A., & Coll, R. (2009). Assessment of learning, for learning, and as learning: New Zealand case studies. Assessment in Education, 16(3), 269 290, https://doi.org/10.1080/09695940903319661
34. Hyland, K. (2014). Writing: Texts, Processes and Practices, Routledge, Taylor and Francis Group.
35. Ibrahim, H.A. (2013). In Search for implementing Learning-Oriented Assessment in an EFL Setting. World Journal of English Language, 3(4), 12-14, ISSN 1925-ƒ0703, https://doi.org/10.5430/wjel.v3n4p11
36. Jones, N., & Seville, N. (2016). Learning Oriented Assessment: A systematic approach (Vol 45). Cambridge University Press, p.14.
37. Kathpalia, S., & Heah, C. (2008). Reflective writing: Insights into what lies beneath. RELC Journal, 39(3), 300–317, https://doi.org/10.1177/0033688208096843
38. Katstra, J., Tollefson, N., & Gilbert, E. (1987). The effects of peer evaluation on attitude toward writing and writing fluency of ninth grade students. Journal of Educational Research, 80(3), https://doi.org/10.1080/00220671.1987.10885745
39. Keppell, M., Au, E., Ma, A. & Chan, C. (2006). Peer learning and learning‐oriented assessment in technology‐enhanced environments. Assessment & Evaluation in Higher Education, 31(4), 453-464. https://doi.org/10.1080/02602930600679159
40. Krause, K-L, & O’Brien, D. (1999). A sociolinguistic study of the argumentative writing of Chinese students. Education Journal, 27(2), 43-64.
41. Lam R. (2019). Writing portfolio assessment in practice: individual, institutional, and systemic issues, Pedagogies: An International Journal, 15(3), 169 182. https://doi.org/10.1080/1554480X.2019.1696197
42. Manzi, J., Flotts, P., & Preiss, D.D. (2012). Design of a college level test of written communication: Theoretical and methodological challenges. In E.L. Grigorenco, E. Mambrino & D.D. Preiss (eds), Writing: A mosaic of new perspectives. Taylor and Francis Group, LLC, 385-400.
43. Martinez, M., & Lipson, J. (1989). Assessment for learning. Educational Leadership, 46(7), 73 75. https://files.ascd.org/staticfiles/ascd/pdf/journals/ed_lead/el_198904_martinez.pdf
44. Mok, M.M.C. (2013). Self-directed learning-oriented assessments in the Asia-Pacific. Springer
45. Nicolás-Conesa, F., Roca de Larios, J., & Coyle, Y (2014). Development of EFL students’ mental models of writing and their effects on performance. Journal of Second Language Writing. 24. http://dx.doi.org/10.1016/j.jslw.2014.02.004
46. Paek, J.K., & Kang, Y. (2017). Investigation of content features that determine Korean EFL learners' argumentative writing qualities. English Teaching, 72(2), 101-122.
47. Purpura, J.E., & Turner, C.E. (2014). A learning-oriented assessment approach to understanding the complexities of classroom-based language assessment, Presentation at the Roundtable on Learning-Oriented Assessment in Language Classrooms and Large-Scale Contexts, Teachers College, Columbia University, New York.
48. Romova, Z., & Andrew, M. (2011). Teaching and assessing academic writing via the portfolio: Benefits for learners of English as an additional language. Assessing Writing, 16, 111-122. https://doi.org/10.1016/j.asw.2011.02.005
49. Swales, J.M. (1990). Genre Analysis: English in academic and research settings. Cambridge University Press.
50. Stiggings, R. (2005). From formative assessment to assessment for learning: A path to success in standards-based schools. Phi Delta Kappan, 87(4), 324–328.
51. Storch, N. (2005). Collaborative writing: Product, process, and students’ reflections. Journal of Second Language Writing, 14, 153–173. https://doi.org/10.1016/j.jslw.2005.05.002
52. Storch, N. (2019). Collaborative writing. Language Teaching, 52(1), 40 59. https://doi.org/10.1017/S0261444818000320
53. Strijbos, J.W., Narciss, S., & Dünnebier, K. (2010). Peer feedback content and sender's competence level in academic writing revision tasks: Are they critical for feedback perceptions and efficiency? Learning and Instruction, 20(4), 291–303. https://doi.org/10.1016/j.learninstruc.2009.08.008
54. Trevelyan, R., & Wilson, A. (2012). Using Patchwork Texts in assessment: Clarifying and categorising choices in their use. Assessment & Evaluation in Higher Education, 37(4), 487-498. https://doi.org/10.1080/02602938.2010.547928
55. Wang, L. (2019). Effects of regulation on interaction pattern in web-based collaborative writing activity, Computer Assisted Language Learning, https://doi.org/10.1080/09588221.2019.1667831
56. Westbury, I., Hopmann, S., & Riquarts, K. (2000). Teaching as a reflective practice: The German didaktik tradition. Routledge.
57. Wu, W.V., Petit, E., & Chen, C. (2015). EFL writing revision with blind expert and peer review using a CMC open forum. Computer Assisted Language Learning, 28(1), 58-80. https://doi.org/10.1080/09588221.2014.937442
58. William, D., & Thompson, M. (2007). Integrating assessment with learning: what will it take to make it work? In: Dwyer, CA, (ed.) The Future of Assessment: Shaping Teaching and Learning. (pp. 53-82). Lawrence Erlbaum Associates.
59. Winter, R. (2003). Contextualizing the Patchwork Text: Addressing problems of course assessment in higher education. Innovations in Education and Teaching International, 40(2), 112-122. https://doi.org/10.1080/1470329031000088978
60. Vallberg Roth, A.C. (2014). Bedömning i förskolans dokumentationspraktiker: Fenomen, begrepp ochreglering. Göteborgs universitet. Institutionen för pedagogik och didaktik, 19(4–5), 403–437.
61. Zeng, W., Huang, F., Yu, L., & Chen, S. (2018). Towards a learning-oriented assessment to improve students’ learning a critical review of literature. Educational Assessment Evaluation and Accountability, 30, 211–250. https://doi.org/10.1007/s11092-018-9281-9