Assessing Reading in Turkish as a Second Language: Scoring and Criterion-Related Validity

Abstract

Justifying the appropriateness of test use for a particular purpose is critical in language testing. The tools used for assessment need to be continually evaluated before and after their use. Arguments regarding the validity of interpretations based on test performances can be developed from various aspects. Two of these aspects are justification for scoring and comparison against external measures. The current paper reports on an investigation of scoring validity and criterion related validity with regard to a set of reading tasks developed for second language (L2) learners of Turkish. The findings provide a preliminary base to develop validity arguments regarding the tasks, but also call for revisions.

Keywords

• Scoring validity,criterion-related validity,assessing reading in Turkish as a second language

References

1. Akbari, R. (2012). Validity in language testing. In C.Coombe, P.Davidson, B.O’Sullivan & S.Stoynoff (Eds.). The Cambridge Guide to Second Language Assessment (pp.30-36). Cambridge: Cambridge University Press.
2. Aksan, Y., Aksan, M., Mersinli, Ü., Demirhan, U. U., & Yılmazer, H. (2012). Turkish national corpus (TNC) demo version work frequency lists (Report No. 1). Mersin: Mersin University.
3. Alderson, J. C. (2000). Assessing reading. Cambridge: Cambridge University Press.
4. Alderson, J. C., Clapham, C., & Wall, D. (1995). Language test construction and evaluation. Cambridge: Cambridge University Press.
5. Bachman, L. F. (2004). Statistical analyses for language assessment. Cambridge: Cambridge University Press.
6. Bachman, L. F. (2005). Building and supporting a case for test use. Language Assessment Quarterly: An International Journal, 2(1), 1-34.
7. Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford University Press.
8. Bossers, B. (1991). On thresholds, ceilings, and short-circuits: The relation between L1 reading, L2 reading and L2 knowledge. In J. H. Julstijn, & J. F. Matters (Eds.), AILA Review, 8, (pp. 45-60). Amsterdam: Free University Press.

9. Carrell, P.L. (1987). Content and formal schemata in ESL reading. TESOL Quarterly 21, 461-481.
10. Council of Europe (2001). Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge: Cambridge University Press.
11. Downing, S. M. (2006). Twelve Steps for Effective Test Development. In S. M. Downing & T. M. Haladyna (Eds.), Handbook of test development (pp. 3-25). Mahwah, NJ, US: Lawrence Erlbaum Associates Publishers.
12. Enright, M., Grabe, W., Koda, K., Mosenthal, P., Mulcahy-Ernt, P., & Schedl, M. (2000). TOEFL 2000 reading framework. Princeton, NJ: Educational Testing Service.
13. Frisbie, D. A., & Ebel, R. L. (1991). Essentials of educational measurement (5th ed.). Englewood Cliffs, NJ: Prentice-Hall.
14. Fulcher, G., & Davidson, F. (2007). Language testing and assessment. London, England: Routledge.
15. Goodman, K. S. (2001). On reading. Portsmouth, NH: Heinemann.
16. Gough, P. B., & Tunmer, W. E. (1986). Decoding, reading, and reading disability. Remedial and Special Education, 7(1), 6-10.
17. Grabe, W., & Stoller, F. L. (2002). Teaching and researching reading. London: Longman.
18. Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309-334.
19. Hsueh-chao, M. H., & Nation, P. (2000). Unknown vocabulary density and reading comprehension. Reading in a Foreign Language, 13, 403-430.
20. Hughes, A. (2003). Testing for language teachers. New York, NY: Cambridge University Press.
21. Jaliehvand, M., & Moses, M. (2014). Influence of rhetorical pattern on improving EFL sudents’ reading comprehension. Journal of Studies in Social Sciences, 7, 210-225.
22. Jiang, X. (2011). The role of first language literacy and second language proficiency in second language reading comprehension. The Reading Matrix, 11, 177-190.
23. Kieffer, M. J. (2010). Socioeconomic status, English proficiency, and late-emerging reading difficulties. Educational Researcher, 39(6), 484-486.
24. Khalifa, H., & Weir, C. J. (2009). Examining reading: Research and practice in assessing second language reading. Studies in Language Testing 29. Cambridge: UCLES/Cambridge University Press.
25. Kintsch, W., & Rawson, K. A. (2005). Comprehension. In M. J. Snowling, & C. Hulme (Eds.), The science of reading: A handbook (pp. 209-226). Malden, MA: Blackwell.
26. Koda, K. (2005). Insights into second language reading. New York, NY: Cambridge University Press.
27. Kurt, Y. (2015). Development of a reading test for second language learners of Turkish (Unpublished master's thesis). Boğaziçi University, İstanbul, Turkey.
28. Lee, S. K. (2007). Effects of textual enhancement and topic familiarity on Korean EFL students' reading comprehension and learning of passive form. Language Learning, 57(1), 87-118.
29. Leeser, M. J. (2007). Learner-based factors in L2 reading comprehension and processing grammatical form: Topic familiarity and working memory. Language Learning, 57, 229-270.
30. McNamara, T. F., & Roever, C. (2006). Language testing: The social dimension. Malden, MA: Blackwell.
31. McNeil, L. (2011). Investigating the contributions of background knowledge and reading comprehension strategies to L2 reading comprehension: an exploratory study. Reading and Writing, 24, 883-902.
32. Messick, S. (1989a). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). New York, NY: Macmillan.
33. Messick, S. (1989b). Meaning and values in test validation: The science and ethics of assessment. Educational Researcher, 18(2), 5-11.
34. Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749.
35. Morvay, G. (2012). The relationship between syntactic knowledge and reading comprehension in EFL learners. Studies in Second Language Learning and Teaching, 2, 415-438.
36. Qian, D. D. (2002). Investigating the relationship between vocabulary knowledge and academic reading performance: an assessment perspective. Language Learning 52(3), 513–536.
37. Rozimela, Y. (2014). The students’ genre awareness and their reading comprehension of different text types. International Journal of Asian Social Science, 4, 460-469.
38. Sabatin, I. M. (2013). The effect of cultural background knowledge on learning English language. International Journal of Science Culture and Sport, 1(4), 22-32. Urquhart, S., & Weir, C. J. (1998). Reading in a second language: Process, product and practice. New York: Longman.
39. Ünaldı, A. (2010) Investigating reading for academic purposes: sentence, text, and multiple texts (Unpublished doctoral dissertation). University of Bedforshire, Bedforshire, England.
40. Weir, C. J. (2005). Language testing and validation. Hampshire: Palgrave McMillan.
41. Weir, C. J., & Khalifa, H. (2008). A cognitive processing approach towards defining reading comprehension, Cambridge ESOL: Research Notes, 31, 2-10.
42. Zhang, X. (2008). The effects of formal schema on reading comprehension: An experiment with Chinese EFL readers. Computational Linguistics and Chinese Language Processing, 13(2), 197-214.
43. Zhang, D. (2012). Vocabulary and grammatical knowledge in L2 reading comprehension: A structural equation modeling study. The Modern Language Journal, 96, 554-571.
44. Zhou, L. (2011). Effects of text types on advanced EFL learners' reading comprehension. Journal of Language and Culture, 30(2), 45–56.