An integrated approach to establish validity and reliability of reading tests

Abstract

This study presents the processes of developing and establishing reliability and validity of a reading test by administering an integrative approach as conventional reliability and validity measures superficially reveals the difficulty of a reading test. In this respect, analysing vocabulary frequency of the test is regarded as a more eligible way of measuring validity. A study was conducted at Dokuz Eylül University and Çanakkale Onsekiz Mart University with three colleagues and 100 undergraduate students to establish validity and reliability along with readability and vocabulary frequency of a 32-item reading test which was developed by the researcher. Such detailed assessment is highly recommended for researchers who are in need of preparing pre and post tests which are different from each other.

Keywords

• assessing reading, reliability, validity; multiple choice; item analysis; item difficulty

References

1. Aiken, L. R. (2003). Psychological testing and assessment (11th ed.). Boston: Allyn and Bacon.
2. Alderson, J. C. (2000). Assessing reading. Cambridge: Cambridge University Press.
3. Brown, H. D. (2001). Teaching by principles: An interactive approach to language pedagogy. New York: Longman.
4. Brown, S. (1994). Assessment: A changing practice. In B. Moon, & A. Shelton Mayes (Eds.), Teaching and learning in the secondary school (pp. 266-272). London: Routledge.
5. Campell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.
6. Capel, S., Leask, M., & Turner, T. (1995). Learning to teach in the secondary school: A companion to school experience. London: Routledge.
7. Carducci, B. J. (2009). The psychology of personality (2nd ed.). Chichester: Wiley-Blackwell.
8. Carmines, E. G., & Zeller, R.A. (1991). Reliability and validity assessment. Newbury Park: Sage Publications.

9. Chastain, K. (1988). Developing second-language skills theory and practice (3rd ed.). Orlando: Harcourt Brace Jovanovich.
10. Conrad, S. (2005). Corpus linguistics and L2 teaching. In E. Hinkel (Ed.), Handbook of research in second language teaching and learning (pp. 393-409). New Jersey: Lawrence Erlbaum.
11. Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Forth Worth: Harcourt Brace Jovanovich.
12. Frith, U. (1985). Beneath the surface of developmental dyslexia. K. Patterson, J. Marshall, & M. Coltheart (Eds.), Neuropsychological and cognitive studies of phonological reading (pp. 301-303). London: Lawrence Erlbaum Associates.
13. Fry, E. B. (1977a). Elementary reading instruction. New York: McGraw-Hill.
14. Fry, E. B. (1977b). Fry’s readability graph: Clarification, validity, and extension to level 17. Journal of Reading, 20, 242-252.
15. Goodwin, L. D. (2001). Interrater agreement and reliability. Measurement in Psychical Education and Exercises Science, 5, 13-14.
16. Hadley, A. O. (2003). Teaching language in context (3rd ed.). Boston, Massachusetts: Heinle & Heinle Publishers.
17. Hannon, P., & McNally, J. (1986). Children’s understanding and cultural factors in reading test performance. Educational Review, 38, 237-246.
18. Kaplan, R. M., & Saccuzzo, D. P. (2001). Psychological testing: Principle, applications and issues (5 ed.). Belmont, CA: Wadsworth
19. th ed.). Belmont, CA: Wadsworth
20. Lewis, M. (1993). The lexical approach. Hove: Language Teaching Publications.
21. McEnery, T., & Wilson, A. (1996). Corpus linguistics. Edinburgh: Edinburgh University Press.
22. McLaughlin, G. H. (1969). SMOG grading: A new readability formula. Journal of Reading, 12, 639-646.
23. Murphy, P. (1994). Assessment and gender. In B. Moon & A. Shelton Mayes (Eds.), Teaching and learning in the secondary school (pp. 287-298). London: Routledge.
24. Noda, M. (2003). Evaluation in reading. In H. Nara & M. Noda (Eds.), Acts of reading: Exploring connections in pedagogy of Japanese (pp. 197-222). Honolulu: University of Hawai’i Press.
25. Pearson, P. D., & Johnson, D. D. (1978). Teaching reading comprehension. New York: Holt, Rinehart and Winston.
26. Razı, S. (2005). A fresh look at the evaluation of ordering tasks in reading comprehension: Weighted marking protocol. The Reading Matrix, 5, 1-15.
27. Razı, S. (2007). Okuma becerisi öğretimi ve değerlendirilmesi [Teaching and assessing reading]. İstanbul: Kriter Yayınları.
28. Richards, J. C., & Rodgers, T. S. (2001). Approaches and methods in language teaching. Cambridge: Cambridge University Press.
29. Wallace, C. (1992). Reading. Oxford: Oxford University Press.