Formative Assessment of Writing (FAoW): A Confirmatory Factor Structure Study

Abstract

This validation study was undertaken to evaluate the construct of Formative Assessment of Writing (FAoW) operationalized by an instrument with 50 Likert scale items. To identify the EFL learners’ experiences of FAoW practices, the instrument was first piloted with three EFL learners, and subsequently administered on a sample of 255 EFL learners selected based on purposive sampling. A five-factor solution with five latent variables (i.e. clarifying criteria, evidence on students’ current learning, feedback to move learners forward, peer-assessment and autonomy) was evaluated through Confirmatory Factor Analysis (CFA) with AMOS 22. Model fit showed that the five-factor structure of FAoW could only be supported in terms of absolute and parsimony fit indices. The model with three factors (i.e. clarifying criteria, peer-assessment and feedback) in two stages of pre- and while-writing, however, provided higher discriminant validity in addition to absolute and parsimony fit indices. In other words, FAoW was not found to be practiced within its full potential with five components in the context of this study. A conceptual model was developed based on the findings and the literature to show pedagogical application of  FAoW and how it can be practiced in line with Black and Wiliam’s (2009).

Keywords

• Formative Assessment of Writing
• Factor Solution
• Formative Assessment
• Confirmatory Factor Analysis (CFA)

Formative Assessment of Writing (FAoW): A Confirmatory Factor Structure Study

Abstract

This validation study was undertaken to evaluate the construct of Formative Assessment of Writing (FAoW) operationalized by an instrument with 50 Likert scale items. To identify the EFL learners’ experiences of FAoW practices, the instrument was first piloted with three EFL learners, and subsequently administered on a sample of 255 EFL learners selected based on purposive sampling. A five-factor solution with five latent variables (i.e. clarifying criteria, evidence on students’ current learning, feedback to move learners forward, peer-assessment and autonomy) was evaluated through Confirmatory Factor Analysis (CFA) with AMOS 22. Model fit showed that the five-factor structure of FAoW could only be supported in terms of absolute and parsimony fit indices. The model with three factors (i.e. clarifying criteria, peer-assessment and feedback) in two stages of pre- and while-writing, however, provided higher discriminant validity in addition to absolute and parsimony fit indices. In other words, FAoW was not found to be practiced within its full potential with five components in the context of this study. A conceptual model was developed based on the findings and the literature to show pedagogical application of FAoW and how it can be practiced in line with Black and Wiliam’s (2009).

Keywords

• Formative Assessment of Writing
• Factor Solution
• Formative Assessment
• Confirmatory Factor Analysis (CFA )

References

1. Arbuckle, J. L. (2012). Amos [Computer software]. Chicago, IL: SPSS.
2. Assessment Reform Group. (2002). Assessment for learning: 10 principles. Research based principles to guide classroom practice. London, UK: Retrieved from http://languagetesting.info/features/afl/4031afl principles.pdf
3. Abdollahzadeh (2010). Undergraduate Iranian EFL learners’ use of writing strategies. Writing & Pedagogy, 2(1), 65-90.
4. Bennett, R. E. (2011). Formative assessment: A critical review. Assessment in Education: Principles, Policy & Practice, 18(1), 5-25.
5. Birjandi, P., & Hadidi Tamjid, N. (2012). The role of self-, peer and teacher assessment in promoting Iranian EFL learners’ writing performance. Assessment & Evaluation in Higher Education, 37(5), 513-533.
6. Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7-73.
7. Black, P. & Wiliam, D. (2006). Assessment for learning in the classroom. In J. Gardner (Ed.), Assessment and learning, 9-25. London: Sage.
8. Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability (formerly: Journal of Personnel Evaluation in Education), 21(1), 5.

9. Brown, T. (2015). Confirmatory analysis for applied research (2nd ed.). New York: The Guilford Press.
10. Brookhart, S. M. (2001). Successful students' formative and summative uses of assessment information. Assessment in Education: Principles, Policy & Practice, 8(2), 153-169.
11. Burner, T. (2015). Processes of change when using portfolios to enhance formative assessment of writing. Assessment Matters, 9(2), 53-79.
12. Burner, T. (2016). Formative assessment of writing in English as a foreign language. Scandinavian Journal of Educational Research, 60(6), 626-648.
13. Elahinia, H. (2004). Assessment of writing through portfolios and achievement tests. Unpublished Masters thesis, Teacher Training University, Iran.
14. Carless, D. (2007). Learning‐oriented assessment: Conceptual bases and practical implications. Innovations in Education and Teaching International, 44(1), 57-66.
15. Feng, H. (2007). Senior ESOL students' experiences of and attitudes towards formative assessment in mainstream secondary classrooms. Unpublished Masters thesis, University of Canterbury, New Zealand.
16. Ghoorchaei, B. Tavakoli, M. & Nejad Ansari, D. (2010). The impact of portfolio assessment on Iranian EFL students‟ essay writing: A process-oriented approach. GEMA Online Journal of Language Studies, 10 (3), 35-51.
17. Graham, S., Harris, K.R., Fink, B., & MacArthur, C.A., (2001). Teacher Efficacy in Writing: A Construct Validation with Primary Grade Teachers, Scientific Studies of Reading, 5(2), 177-202.
18. Hasim, Z. (2014). An integration of a process approach and formative assessment into the development of teaching and learning of ESL writing in a Malaysian university: A sociocultural perspective (Doctoral dissertation, University of Waikato).
19. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112.
20. Havnes, A., Smith, K., Dysthe, O., & Ludvigsen, K. (2012). Formative assessment and feedback: Making learning visible. Studies in Educational Evaluation, 38(1), 21-27.
21. Hawe, E., & Parr, J. M. (2013). Assessment for Learning-Form and Substance in Writing Lessons. In European Conference on Educational Research (ECER) Conference. Istanbul, Turkey.
22. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modeling: a multidisciplinary journal, 6(1), 1-55.
23. Javaherbakhsh, M. R. (2010). The impact of self-assessment on Iranian EFL learners’ writing skill. English Language Teaching, 3(2), 213-218.
24. Johnson, R. C., & Riazi, A. M. (2017). Validation of a locally created and rated writing test used for placement in a higher education EFL program. Assessing Writing, 32, 85-104.
25. Ketabi, S. (2015). Different methods of assessing writing among EFL teachers in Iran. International Journal of Research Studies in Language Learning, 5(2), 3-15.
26. Lee, I. (2003). L2 writing teachers’ perspectives, practices and problems regarding error feedback. Assessing Writing, 8(3), 216-237.
27. Lee, I. (2007). Assessment for learning: integrating assessment, teaching, and learning in the ESL/EFL writing classroom. The Canadian Modern Language Review, 64(1), 199-213.
28. Lee, I. (2011). Formative Assessment in EFL Writing: An Exploratory Case Study, Changing English: Studies in Culture and Education, 18(1), 99-111.
29. Lee, I. & Coniam, D. (2013). Introducing assessment for learning for EFL writing in an assessment of learning examination-driven system in Hong Kong. Journal of Second Language Writing, 22(1), 34-50.
30. Mak, P., & Lee, I. (2014). Implementing assessment for learning in L2 writing: An activity theory perspective. System, 47, 73-87.
31. Moradan, A., & Hedayati, N. (2011). The impact of portfolios and conferencing on Iranian EFL writing skill. Journal of English Language Teaching and Learning, 8, 115-141.
32. Mosmery, P., & Barzegar, R. (2015). The effects of using peer, self, and teacher-assessment on Iranian EFL learners’ writing ability at three levels of task complexity. International Journal of Research Studies in Language Learning, 4(4), 15-27.
33. Naghdipour, B. (2016). English writing instruction in Iran: Implications for second language writing curriculum and pedagogy. Second Language Writing Journal, 32, 81-87.
34. Naghdipour, B. (2017). Incorporating formative assessment in Iranian EFL writing: A case study. The Curriculum Journal, 28(2), 283-299.
35. Naghdipour, B., & Koç, S. (2015). The evaluation of a teaching intervention in Iranian EFL writing. The Asia-Pacific Education Researcher, 24(2), 389-398.
36. Nezakatgoo, B. (2005). The effects of writing and portfolio on final examination scores and mastering mechanics of writing of EFL students. Unpublished Master thesis, Allame Tabtba'i University, Tehran, Iran.
37. Pallant, J. (2007). SPSS Survival Manual, A Step by Step Guide to Data Analysis using SPSS for Windows, third edition, In SPSS Survival Manual. Open University Press, New York.
38. Pallant, J., & Manual, S. S. (2007). A step by step guide to data analysis using SPSS for windows. In SPSS Survival Manual. Open University Press, New York.
39. Rahimi, M. (2009). The role of teacher’s corrective feedback in improving Iranian EFL learners’ writing accuracy over time: Is learner’s mother tongue relevant? Reading and Writing, 22(2), 219-243.
40. Rahimi, M. (2013). Is training student reviewers worth its while? A study of how training influences the quality of students’ feedback and writing. Language Teaching Research, 17(1), 67-89.
41. Sadeghi, K., & Rahmati, T. (2017). Integrating assessment as, for, and of learning in a large-scale exam preparation course. Assessing Writing, 34, 50-61.
42. Saliu-Abdulahi, D. (2017). Scaffolding writing development: How formative is the feedback?. Moderna språk, 111(1), 127-155.
43. Saliu-Abdulahi, D., Hellekjær, G. O., & Hertzberg, F. (2017). Teachers'(Formative) Feedback Practices in EFL Writing Classes in Norway. Journal of Response to Writing, 3(1), 31-55.
44. Sharifi, A., & Hassaskhah, J. (2011). The role of portfolios assessment and reflection on process writing. Asian EFL Journal, 13(1), 192-229.
45. Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Needham Heights, MA: Allyn and Bacon.
46. Wingate, U. (2010). The impact of formative feedback on the development of academic writing. Assessment & Evaluation in Higher Education, 35(5), 519-533.