Implications of current validity frameworks for classroom assessment

Abstract

The argument-based approach is the current framework for validity and validation. One of the criticisms is that understanding and applying this approach to practice are complicated and require abstract thinking. Teachers or school administrators in teaching and learning need support in their validation practice. Due to the abstract structure of validity, the test users and instructors who are not familiar with psychometrics may face problems in gathering validity evidence. Especially in classroom assessment, teachers may deal with understanding the complex methods of validation. In line with this need, the purpose of this study is to help instructors validate their assessment practices by providing a pathway to guide them through their validation processes and to make the validation process more obvious in classroom assessment. For this purpose, a checklist including the validity indicators for classroom assessment is developed. In this development process, Sireci's (2020) 4-step validation which is based on AERA et al. (2014) Standards and Bonner's (2013) study as a framework were followed. The validity indicators were composed by simplifying the AERA’s standards and the ones which are relevant to classroom assessment were selected. In addition to the standards, the aforementioned studies were investigated and the validity indicators that may be applicable in classroom assessment were determined.

Keywords

• Validity
• Classroom assessment
• Educational assessment
• Argument-based approach
• Educational standards

References

1. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.
2. Bonner, S.M. (2013). Validity in classroom assessment: Purposes, properties, and principles. In J.H. McMillan (Ed.), SAGE handbook of research on classroom assessment, (pp. 87-106). SAGE.
3. Cizek, G.J. (2012). Defining and distinguishing validity: Interpretations of score meaning and justifications of test use. Psychological Methods, 17(1), 31. https://doi.org/10.1037/a0026975
4. Cronbach, L.J. (1988). Five perspectives on validity argument. In H. Wainer & H. Braun (Eds.), Test validity (pp. 3–17). Lawrence Erlbaum Associates, Inc.
5. Cronbach, L.J., & Meehl, P.E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 174-203.
6. Kane, M.T. (1992). An argument-based approach to validity. Psychological Bulletin, 112(3), 527–535. https://doi.org/10.1037/0033-2909.112.3.527
7. Kane, M.T. (2006). Validation. In R.L. Brennan (Ed.), Educational measurement (4th ed., pp. 17–64). American Council on Education/Praeger.
8. Kane, M.T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. https://doi.org/10.1111/jedm.12000

9. Kane, M., & Bridgeman, B. (2021). The evolution of the concept of validity. In B.E. Clauser & M.B. Bunch (Eds.), The history of educational measurement (pp. 181-205). Routledge.
10. Markus, K.A. (2016). Alternative vocabularies in the test validity literature. Assessment in Education: Principles, Policy & Practice, 23(2), 252 267. https://doi.org/10.1080/0969594X.2015.1060191
11. Messick, S. (1980). Test validity and the ethics of assessment. American Psychologist, 35(11), 1012–1027. https://doi.org/10.1037/0003-066X.35.11.1012
12. Messick, S. (1989). Validity. In R.L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). Macmillan.
13. Moss, P.A. (2013). Validity in action: Lessons from studies of data use. Journal of Educational Measurement, 50(1), 91-98. https://doi.org/10.1111/jedm.12003
14. Moss, P. (2016). Shifting the focus of validity for test use. Assessment in Education: Principles, Policy & Practice, 23(2), 236-251. https://doi.org/10.1080/0969594X.2015.1072085
15. Newton, P.E. (2013). Two kinds of argument?. Journal of Educational Measurement, 50(1), 105–109. https://doi.org/10.1111/jedm.12004
16. Newton, P., & Shaw, S. (2014). The deconstruction of validity: 2000–2012. In Validity in educational and psychological assessment (pp. 135-182). Sage.
17. Newton, P., & Shaw, S. (2016). Disagreement over the best way to use the word ‘validity’ and options for reaching consensus. Assessment in Education: Principles, Policy & Practice, 23(2), 178-197. https://doi.org/10.1080/0969594X.2015.1037241
18. Sireci, S.G., & Benítez, I. (2023). Evidence for test validation: A guide for practitioners. Psicothema 35(3) 217-226. https://doi.org/10.7334/psicothema2022.477
19. Sireci, S.G. (2013). Agreeing on validity arguments. Journal of Educational Measurement, 50(1), 99-104. https://doi.org/10.1111/jedm.12005
20. Sireci, S.G. (2020). De-“constructing” test validation. Chinese/English Journal of Educational Measurement and Evaluation, 1(1), Article 3. https://doi.org/10.59863/CKHH8837
21. Welch, C.J. (2021). Rethinking measurement 101: Lessons learned from teachers. Educational Measurement: Issues and Practice, 40(4), 13-17. https://doi.org/10.1111/emip.12479