A quantitative test analysis: Implementing an interpretive approach to validity argument

Year 2015, Volume: 36 Issue: 3, 3530 - 3537, 13.05.2015

Forough Rahımı

Abstract

Abstract. Validity and test fairness have concerned many researchers and scholars in the field of language testing due to its importance in decision making process (Kunnan, 2008; Shohamy, 2001, 2007; Bachman, 2005) as well as the consequences that an unfair practice may bring to test takers ( Kane, 2004, 2006; Xi, 2005a, 2005b). This study was a quantitative investigation of test validity and fairness using an argument-based approach developed by Xi (2010). The main purpose of this study was to quantitatively assess and determine the relationship between a series of inferences in an argument-based framework that can potentially lead to degrees of unfairness and lack of validity in language testing. To this end samples of IELTS test were analyzed and the obtained results determined degrees of fairness which existed among the inferences in the interpretive model.

Keywords

test validity, test fairness, interpretive approach, quantitative analysis

References

• Bachman, L.F. (1990).Fundamental considerations in language testing. Oxford University Press, Oxford.
• Bachman, L. F. (2005). Building and supporting a case for test use. Language Assessment Quarterly,2, 1–34.
• Bachman, L.F. & Palmer, A. (1996).Language testing in practice: Designing and developing useful language tests, Oxford University Press, Oxford.
• Chapelle, C. (1999). Validity in language testing. Annual Review of Applied Linguistics, 19, 254-274.
• Cheng, L. (2005). Changing language teaching through language testing: A Washback study. Studies in Language Testing: Volume 21, Cambridge University Press, Cambridge.
• Davies, A. (2003). Three heresies of language testing research. Language Testing, 20(4), 355 –368.
• Kane, M. T. (2004). Certification testing as an illustration of argument-based validation. Measurement: Interdisciplinary Research and Perspectives, 2, 135–170.
• Kane, M. T. (2006). Validation.In Brennan, R. L. (Ed.), Educational measurement, 4th edn. (pp.18–64). Washington, DC: American Council on Education/ Praeger.
• Kunnan, A. J. (2008). Large-scale language assessment. In E. Shohamy & N. Hornberger (Ed). Encyclopedia of language and education, 2nd Edition, Volume 7: Language Testing and assessment (pp. 135-155). Amsterdam: Springer Science.
• McNamara, T. F. & Roever, C. (2006).Language testing: The social dimension. Oxford: Blackwell.
• Messick, S. (1980). Test validity and the ethics of assessment. American Psychologist, 35(11), 1012-1027.
• Messick, S. (1988). The once and future issues of validity. Assessing the meaning and consequences of measurement. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 33 - 45). Hillsdale, N.J. :: Lawrence Erlbaum Associates.
• Messick, S. (1989). Validity. In Linn, R. L. (Ed.), Educational measurement, 3rd edn. (pp. 13–103). New York: American Council on Education and Macmillan.
• Messick, S. (1996). Validity and washback in language testing. Language Testing, 13(3), 241–256.
• Shohamy, E. (2001). The power of tests: A critical perspective of the uses of language tests. London: Longman.
• Shohamy, E. (2007). Language tests as language policy tools. Assessment in Education: Principles, Policy & Practice, 14, 117-130.
• Xi, X. (2005a).An argument-based approach to investigating fairness for the new TOEFL test. Unpublished manuscript, Educational Testing Service, Princeton, NJ.
• Xi, X. (2005b). Do visual chunks and planning impact performance on the graph description task in the SPEAK Exam? Language Testing, 22(4), 463–508.
• Xi, X. (2010).How do we go about investigating test fairness? Language Testing, 27(2), 147–170.