Exploring the Validity of a CEFR-Aligned Placement Test through Norm-Referenced and Criterion-Referenced Approaches

Öz

Researchers in the field of language assessment have generally overlooked the area of placement test validation despite the importance of such measures. This paper describes the second phase of the research which aims at collecting evidence to judge the degree to which the use of an in-house selected response English as a foreign language (EFL) placement test. To serve this purpose, data were collected from a variety of subjects employing various instruments. Initially, the placement test was administered to 789 EFL university students at four proficiency levels employing a pre- and post-test design, with an 8 week interval to collect construct validity evidence. Added to this, the course teachers (n= 43) have been requested to report whether each student in their classroom has been placed accurately based on placement test scores. The paper concludes by discussing the findings to evaluate the validity of the test.

Anahtar Kelimeler

• language assessment
• validity
• NRT
• placement test

CEFR Uyumlu Bir Seviye Belirleme Sınavının Geçerliliğinin Norm Referanslı ve Kriter Referanslı Yaklaşımlar Aracılığıyla İncelenmesi"

Öz

Researchers in the field of language assessment have generally overlooked the area of placement test validation despite the importance of such measures. This paper describes the second phase of the research which aims at collecting evidence to judge the degree to which the use of an in-house selected response English as a foreign language (EFL) placement test. To serve this purpose, data were collected from a variety of subjects employing various instruments. Initially, the placement test was administered to 789 EFL university students at four proficiency levels employing a pre- and post-test design, with an 8 week interval to collect construct validity evidence. Added to this, the course teachers (n= 43) have been requested to report whether each student in their classroom has been placed accurately based on placement test scores. The paper concludes by discussing the findings to evaluate the validity of the test.

Anahtar Kelimeler

• validity
• language assessment
• norm referenced test
• placement test

Kaynakça

1. Alderson, J. C., Clapham, C., & Wall, D. (1995). Language test construction and evaluation. Cambridge University Press.
2. Bachman, L. F. (2000). Modern language testing at the turn of the century: Assuring that what we count counts. Language Testing, 17(1), 1–42.
3. Bernhardt, E. B., Rivera, R. J., & Kamil, M. L. (2004). The practicality and efficiency of web‐based placement testing for college‐level language programs. Foreign Language Annals, 37(3), 356–365.
4. Brown, J. D. (1989). Improving ESL placement tests using two perspectives. TESOL Quarterly, 23(1), 65–83.
5. Brown, J. D. (1990). Where do tests fit into language programs. JALT Journal, 12(1), 121–140.
6. Brown, J. D. (2005). Testing in language programs: A comprehensive guide to English language assessment. McGraw-Hill College.
7. Brown, J. D. (2008). Testing-context analysis: Assessment is just another part of language curriculum development. Language Assessment Quarterly, 5(4), 275–312.
8. Brown, J. D., & Hudson, T. (2002). Criterion-referenced language testing. Cambridge University Press.

9. Chapelle, C. A., Jamieson, J., & Hegelheimer, V. (2003). Validation of a web-based ESL test. Language Testing, 20(4), 409–439.
10. Choe, M. (2010). Rasch analysis of a nationwide English placement test in Korea. English Teaching, 65(2), 3–23.
11. Clark, M. (2004). By the numbers: The rationale for Rasch analysis in placement testing. Second Language Studies, 22(2), 61–90.
12. Culligan, B., & Gorsuch, G. (1999). Using a commercially produced proficiency test in a one-year core EFL curriculum in Japan for placement purposes. JALT Journal, 21, 7–25.
13. Eda, S., Itomitsu, M., & Noda, M. (2008). The Japanese Skills Test as an On‐Demand Placement Test: Validity Comparisons and Reliability. Foreign Language Annals, 41(2), 218–236.
14. Feak, C. B., & Salehzadeh, J. (2001). Challenges and issues in developing an EAP video listening placement assessment: A view from one program. English for Specific Purposes, 20, 477–493.
15. Fulcher, G. (1997). An English language placement test: Issues in reliability and validity. Language Testing, 14(2), 113–139.
16. Green, A. B., & Weir, C. J. (2004). Can placement tests inform instructional decisions? Language Testing, 21(4), 467–494.
17. Harsch, C., & Rupp, A. A. (2011). Designing and scaling level-specific writing tasks in alignment with the CEFR: A test-centered approach. Language Assessment Quarterly, 8(1), 1–33.
18. Hudson, T., & Clark, M. (Eds.). (2008). Case studies in foreign language placement: Practices and possibilities. National Foreign Language Resource Center.
19. Hudson, T., & Lynch, B. (1984). A criterion-referenced measurement approach to ESL achievement testing. Language Testing, 1(2), 171–201.
20. James, C., & Templeman, E. (2009). A case for faculty involvement in EAP placement testing. TESL Canada Journal, 26(2), 82–99.
21. Li, Z. (2015). An argument-based validation study of the English Placement Test (EPT): Focusing on the inferences of extrapolation and ramification (Doctoral dissertation, Iowa State University).
22. Long, A., Shin, S. Y., Geeslin, K., & Willis, E. (2018). Does the test work? Evaluating a web-based language placement test. Language Learning & Technology, 22(1), 137–151.
23. Lynch, B. K., & Davidson, F. (1994). Criterion‐referenced language test development: Linking curricula, teachers, and tests. TESOL Quarterly, 28(4), 727–743.
24. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). Macmillan.
25. Pardo-Ballester, C. (2010). Measuring the effectiveness of a web-based computer placement test. CALICO Journal, 27(2), 269–287.
26. Popham, W. J. (1978). Criterion-referenced measurement. Prentice-Hall.
27. Sasaki, M. (1991). A comparison of two methods for detecting differential item functioning in an ESL placement test. Language Testing, 8(2), 95–111.
28. Sireci, S. G. (1998). The construct of content validity. Social Indicators Research, 45, 83–117.
29. Wall, D., Clapham, C., & Alderson, J. C. (1994). Evaluating a placement test. Language Testing, 11(3), 321–344.
30. Weir, C. J. (2005). Language testing and validation. Hampshire: Palgrave McMillan, 10, 9780230514577.