Implications of current validity frameworks for classroom assessment

Year 2023, , 164 - 173, 27.12.2023

Ezgi Mor , Rabia Karatoprak Erşen

https://doi.org/10.21449/ijate.1368458

Abstract

The argument-based approach is the current framework for validity and validation. One of the criticisms is that understanding and applying this approach to practice are complicated and require abstract thinking. Teachers or school administrators in teaching and learning need support in their validation practice. Due to the abstract structure of validity, the test users and instructors who are not familiar with psychometrics may face problems in gathering validity evidence. Especially in classroom assessment, teachers may deal with understanding the complex methods of validation. In line with this need, the purpose of this study is to help instructors validate their assessment practices by providing a pathway to guide them through their validation processes and to make the validation process more obvious in classroom assessment. For this purpose, a checklist including the validity indicators for classroom assessment is developed. In this development process, Sireci's (2020) 4-step validation which is based on AERA et al. (2014) Standards and Bonner's (2013) study as a framework were followed. The validity indicators were composed by simplifying the AERA’s standards and the ones which are relevant to classroom assessment were selected. In addition to the standards, the aforementioned studies were investigated and the validity indicators that may be applicable in classroom assessment were determined.

Keywords

Validity, Classroom assessment, Educational assessment, Argument-based approach, Educational standards

References

• American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.
• Bonner, S.M. (2013). Validity in classroom assessment: Purposes, properties, and principles. In J.H. McMillan (Ed.), SAGE handbook of research on classroom assessment, (pp. 87-106). SAGE.
• Cizek, G.J. (2012). Defining and distinguishing validity: Interpretations of score meaning and justifications of test use. Psychological Methods, 17(1), 31. https://doi.org/10.1037/a0026975
• Cronbach, L.J. (1988). Five perspectives on validity argument. In H. Wainer & H. Braun (Eds.), Test validity (pp. 3–17). Lawrence Erlbaum Associates, Inc.
• Cronbach, L.J., & Meehl, P.E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 174-203.
• Kane, M.T. (1992). An argument-based approach to validity. Psychological Bulletin, 112(3), 527–535. https://doi.org/10.1037/0033-2909.112.3.527
• Kane, M.T. (2006). Validation. In R.L. Brennan (Ed.), Educational measurement (4th ed., pp. 17–64). American Council on Education/Praeger.
• Kane, M.T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. https://doi.org/10.1111/jedm.12000
• Kane, M., & Bridgeman, B. (2021). The evolution of the concept of validity. In B.E. Clauser & M.B. Bunch (Eds.), The history of educational measurement (pp. 181-205). Routledge.
• Markus, K.A. (2016). Alternative vocabularies in the test validity literature. Assessment in Education: Principles, Policy & Practice, 23(2), 252 267. https://doi.org/10.1080/0969594X.2015.1060191
• Messick, S. (1980). Test validity and the ethics of assessment. American Psychologist, 35(11), 1012–1027. https://doi.org/10.1037/0003-066X.35.11.1012
• Messick, S. (1989). Validity. In R.L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). Macmillan.
• Moss, P.A. (2013). Validity in action: Lessons from studies of data use. Journal of Educational Measurement, 50(1), 91-98. https://doi.org/10.1111/jedm.12003
• Moss, P. (2016). Shifting the focus of validity for test use. Assessment in Education: Principles, Policy & Practice, 23(2), 236-251. https://doi.org/10.1080/0969594X.2015.1072085
• Newton, P.E. (2013). Two kinds of argument?. Journal of Educational Measurement, 50(1), 105–109. https://doi.org/10.1111/jedm.12004
• Newton, P., & Shaw, S. (2014). The deconstruction of validity: 2000–2012. In Validity in educational and psychological assessment (pp. 135-182). Sage.
• Newton, P., & Shaw, S. (2016). Disagreement over the best way to use the word ‘validity’ and options for reaching consensus. Assessment in Education: Principles, Policy & Practice, 23(2), 178-197. https://doi.org/10.1080/0969594X.2015.1037241
• Sireci, S.G., & Benítez, I. (2023). Evidence for test validation: A guide for practitioners. Psicothema 35(3) 217-226. https://doi.org/10.7334/psicothema2022.477
• Sireci, S.G. (2013). Agreeing on validity arguments. Journal of Educational Measurement, 50(1), 99-104. https://doi.org/10.1111/jedm.12005
• Sireci, S.G. (2020). De-“constructing” test validation. Chinese/English Journal of Educational Measurement and Evaluation, 1(1), Article 3. https://doi.org/10.59863/CKHH8837
• Welch, C.J. (2021). Rethinking measurement 101: Lessons learned from teachers. Educational Measurement: Issues and Practice, 40(4), 13-17. https://doi.org/10.1111/emip.12479