Correcting Fallacies about Validity as the Most Fundamental Concept in Educational and Psychological Measurement

Year 2022, , 148 - 154, 11.11.2022

Vahit Bademci

https://doi.org/10.31458/iejes.1140672

Cited By: 1

Abstract

Validity is the most fundamental cerebration in educational and psychological testing. That is to say, validity is a crucial concept in psychometrics, but it is still misunderstood and misused. Validity has changed in the last 100 years, in other words, evolved. Validity is the degree to which evidence and theory support the adequacy and appropriateness of the proposed interpretations and uses of the scores obtained from the test or measurement instrument applied to a particular population or sample. In short, validity is not a property of a test or measurement instrument itself, but it is a property of the proposed interpretations and uses of the scores. Thus, such statements as ‘the test is valid’, ‘the validity of scale’ or ‘the scores are valid’ should not be used. The most authoritative source regarding the development and evaluation of educational and psychological tests is published by name of the Standards for Educational and Psychological Testing and briefly referred to as the Standards. The view of content validity, criterion-related validity and construct validity supported in 1966 Standards was quitted in 1999 Standards.

Keywords

Validity, validation, sources of validity evidence, reliability, misconceptions in educational and psychological testing

References

• Algina, J., & Penfield, R. D. (2009). Classical test theory. In R. Millsap, & A. Maydeu-Olivares (Eds.), The Sage handbook of quantitative methods in psychology (pp. 93-122). Los Angeles: Sage.
• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (AERA, APA, & NCME) (1985). Standards for educational and psychological testing. Washington, DC: American Psychological Association.
• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (AERA, APA, & NCME) (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (AERA, APA, & NCME) (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
• American Psychological Association, American Educational Research Association, & National Council on Measurement in Education (APA, AERA, & NCME) (1966). Standards for educational and psychological tests and manuals. Washington, DC: American Psychological Association.
• Angoff, W. H. (1988). Validity: An evolving concept. In H. Wainer, & H. I. Braun (Eds.), Test validity (pp. 19-32). Hillsdale, New Jersey: Lawrence Erlbaum.
• Bademci, V. (1999). Türkiye’de eğitim fakülteleri ve öğretmen yetiştirme: Öğretmen yetiştiren programlar nasıl olmalı? [Education faculties and teacher training in Turkey: How should teacher training programs be?] Panel. Düzenleyen: ESEF İşletme Araştırma Topluluğu. Ankara: G.Ü. Mesleki Eğitim Fakültesi Konferans Salonu, 21 Mayıs 1999.
• Bademci, V. (2004). Testin güvenirliği” veya “test güvenilirdir” diye ifade etmek doğru değildir [It is incorrect to express of “the reliability of the test” or “ the test is reliable”]. Türk Eğitim Bilimleri Dergisi, 2, 367-373.
• Bademci, V. (2007). Ölçme ve araştırma yöntembiliminde paradigma değişikliği: Testler güvenilir değildir [A paradigm change in measurement and research methodology: Tests are not reliable]. Ankara: Yenyap.
• Bademci, V. (2011). Türk eğitim ve biliminde bilimsel devrim: Testler ya da ölçme araçları güvenilir ve geçerli değildir [Scientific revolution in Turkish education and science: Tests or measurement instruments are not reliable and valid]. Dicle Üniversitesi Ziya Gökalp Eğitim Fakültesi Dergisi, 16, 116-132.
• Bademci, V. (2013). Değerbiçiciler arası (interrater) ölçüm güvenirliğinin Cronbach’ın alfası ile kestirilmesi [Estimation of interrater score reliability by the Cronbach’s alpha]. Gazi Üniversitesi Endüstriyel Sanatlar Eğitim Fakültesi Dergisi, 30, 55-62.
• Bademci, V. (2014). Cronbach’s alpha is not a measure of unidimensionality or homogeneity. Journal of Computer and Educational Research, 2(3), 19-27.
• Bademci, V. (2017a). Ölçme ve araştırma yöntembiliminde çağdaş gelişmeler ve yeni standartlar 1: Geçerlik, ölçümlerin kullanımlarının ve önerilen yorumlarının bir özelliğidir [Contemporary developments and new standards in measurement and research methodology 1: Validity is a property of the proposed interpretations and uses of scores]. JRES, 4(1), 63-80.
• Bademci, V. (2017b). Ölçme ve araştırma yöntembiliminde çağdaş gelişmeler ve yeni standartlar 2: Geçerlikte üçleme (kapsam, ölçüt ilişkili ve yapı geçerlikleri) öğretisinin reddi ve geçerlik kanıtının kaynakları [Contemporary developments and new standards in measurement and research methodology 2: Rejection of the trinitarian (content, criterion-related, and construct validities) doctrine in validity and sources of validity evidence]. JRES, 4(1), 81-97.
• Bademci, V. (2019). Geçerlik: Nedir? Ne değildir? [Validity: What is it? What is it not?] JRES, 6(2), 373-385.
• Cizek, G. J. (2016). Validating test score meaning and defending test score use: Different aims, different methods. Assessment in Education: Principles, Policy & Practice, 23(2), 212-225.
• Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Fort Worth: Holt, Rinehart and Winston.
• Cronbach, L. J. (1971). Test validation. In R. L. Thorndike (Ed.), Educational measurement (2nd ed.) (pp. 443-507). Washington, DC: American Council on Education.
• Ercikan, K., & Pellegrino, J. W. (2017). Validation of score meaning using examinee response processes for the next generation of assessments. In K. Ercikan, & J. W. Pellegrino (Eds.), Validation of score meaning for the next generation of assessments (pp. 1-8). New York: Routledge.
• Frisbie, D. A. (2005). Measurement 101: Some fundamentals revisited. Educational Measurement: Issues and Practice, 24(3), 21-28.
• Furr, R. M., & Bacharach, V. R. (2008). Psychometrics: An introduction. Los Angeles: Sage.
• Goodwin, L. D., & Goodwin, W. L. (1999). Measurement myths and misconceptions. School Psychology Quarterly, 14(1), 408-427.
• Guion, R. M. (1980). On trinitarian doctrines of validity. Professional Psychology, 11(3), 385-398.
• Kane, M. T. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38(4), 319-342.
• Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed.) (pp. 17-64). Westport, CT: American Council on Education & Praeger.
• Kane, M. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50 (1), 1-73.
• Koretz, D. (2008). Measuring up: What educational testing really tells us. Cambridge, Massachusetts: Harvard University Press.
• Linn, R. L. (2010). Validity. In P. Peterson, E. Baker, & B. McGaw (Eds.), International Encyclopedia of Education, Volume 4 (pp. 181-185). Oxford: Elsevier.
• Linn, R. L., & Gronlund, N. E. (1995). Measurement and assessment in teaching (7th ed.). Upper Saddle River, New Jersey: Prentice-Hall.
• Linn, R. L., & Miller, M. D. (2005). Measurement and assessment in teaching (9th ed.).Upper Saddle River, New Jersey: Pearson.
• Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed.) (pp. 13-103). New York: American Council on Education and Macmillan Publishing Company.
• Mislevy, R. J. (2018). Sociocognitive foundations of educational measurement. New York: Routledge.
• Newton, P. E. (2012). Clarifying the consensus definition of validity. Measurement, 10(1-2), 1-29.
• Newton, P. E., & Shaw, S. D. (2014). Validity in educational & psychological assessment. London: Sage.
• Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.
• Osterlind, S. J. (2006). Modern measurement: Theory, principles, and applications of mental appraisal. Upper Saddle River, New Jersey: Pearson.
• Phelps, R. P. (Ed.). (2009). Correcting fallacies about educational and psychological testing. Washington, DC: American Psychological Association.
• Reynolds, C. R., & Livingston, R. B. (2012). Mastering modern psychological testing: Theory & methods. Boston: Pearson.
• Rogers, T. B. (1995). The psychological testing enterprise: An introduction. Pasific Grove, California: Brooks/Cole.
• Sijtsma, K. (2009). Correcting fallacies in validity, reliability, and classification. International Journal of Testing, 9, 167-194.
• Sireci, S. G., & Faulkner-Bond, M. (2015). Promoting validity in the assessment of English learners. Review of Research in Education, 39 (1), 215-252.
• Sireci, S. G., & Soto, A. (2016). Validity and accountability: Test validation for 21st-century educational assessments. In H. Braun (Ed.), Meeting the challenges to measurement in an era of accountability (pp. 149-167). New York: Routledge.
• Suen, H. K., & Rzasa, S. E. (2004). Psychometric foundations of behavioral assessment. In S.N. Haynes, & E. M. Heiby (Eds.), M. Hersen (Series Ed.), Comprehensive handbook of psychological assessment, Volume 3 (pp. 37- 56). Hoboken, New Jersey: John Wiley & Sons.
• Thompson, B. (2003). Understanding reliability and coefficient alpha, really. In B. Thompson (Ed.), Score reliability (pp. 3-23). Thousand Oaks, California: Sage.
• Urbina, S. (2014). Essentials of psychological testing (2nd ed.). Hoboken, New Jersey: Wiley.
• Viswanathan, M. (2005). Measurement error and research design. Thousand Oaks, California: Sage.