Year 2020, Volume 7 , Issue 2, Pages 145 - 158 2020-06-13

Wald Test Formulations in DIF Detection of CDM Data with the Proportional Reasoning Test

Likun HOU [1] , Ragıp TERZİ [2] , Jimmy DE LA TORRE [3]


This study aims to conduct differential item functioning analyses in the context of cognitive diagnosis assessments using various formulations of the Wald test. In implementing the Wald test, two scenarios are considered: one where the underlying reduced model can be assumed; and another where a saturated CDM is used. Illustration of the different Wald test to detect DIF in CDM data was based on the items’ performance of the Proportional Reasoning test among low- and high-performing school students. A benchmark simulation study was included to compare the performance of the Wald test in each scenario. The agreement of the latent attribute classification based on different cognitive diagnosis models was also discussed.
Cognitive Diagnosis Model, Proportional Reasoning, Wald Test, DIF
  • Camilli, G. (2006). Test fairness. Educational Measurement, 4, 221-256.
  • Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.
  • de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179-199.
  • de la Torre, J., & Douglas, J. A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69, 333-353.
  • de la Torre, J., & Lee, Y. S. (2013). Evaluating the wald test for item-level comparison of saturated and reduced models in cognitive diagnosis. Journal of Educational Measurement, 50, 355-373.
  • Doornik, J. A. (2002). An object-oriented matrix programming using Ox (Version 3.1) [Computer software]. London, UK: Timberlake Consultants Press.
  • Haertel, E. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 301-321.
  • Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (p. 129–145). Hilldale, NJ: Lawrence Earlbaum Associates.
  • Hou, L., de la Torre, J., & Nandakumar, R. (2014). Differential item functioning assessment in cognitive diagnostic modeling: Application of the wald test to investigate DIF in the DINA model. Journal of Educational Measurement, 51, 98-125.
  • Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258-272.
  • Li, F. (2008). A modified higher-order DINA model for detecting differential item functioning and differential attribute functioning (Doctoral dissertation). University of Georgia, Athens, GA.
  • Ma, W., Iaconangelo, C., & de la Torre, J. (2016). Model similarity, model selection, and attribute classification. Applied Psychological Measurement, 40, 200-217.
  • Milewski, G. B., & Baron, P. A. (2002). Extending DIF methods to inform aggregate reports on cognitive skills. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans, LA.
  • Morrison, D. F. (1967). Multivariate statistical methods. New York, NY: McGraw-Hill.
  • Shealy, R., & Stout, W. (1993). A model-based standardization approach that separates true bias/dif from group ability differences and detects test bias/dtf as well as item bias/dif. Psychometrika, 58, 159-194.
  • Tatsuoka, K. K. (1984). Analysis of errors in fraction addition and subtraction problems. Computer-based Education Research Laboratory, University of Illinois.
  • Terzi, R. (2017). New Q-matrix validation procedures (Doctoral dissertation). Rutgers, The State University of New Jersey, New Brunswick, NJ.
  • Tjoe, H., & de la Torre, J. (2014). The identification and validation process of proportional reasoning attributes: An application of a cognitive diagnosis modeling framework. Mathematics Education Research Journal, 26, 237-255.
  • Zhang, W. (2006). Detecting differential item functioning using the DINA model (Doctoral dissertation). University of North Carolina at Greensboro, Greensboro, NC.
  • Zumbo, B. D. (2007). Three generations of dif analyses: Considering where it has been where it is now, and where it is going. Language Assessment Quarterly, 4, 223-233.
Primary Language en
Subjects Education, Scientific Disciplines
Published Date June
Journal Section Articles
Authors

Orcid: 0000-0002-1381-8907
Author: Likun HOU
Institution: Educational Testing Service, NJ
Country: United States


Orcid: 0000-0003-3976-5054
Author: Ragıp TERZİ (Primary Author)
Institution: Harran University, Faculty of Education
Country: Turkey


Orcid: 0000-0002-0893-3863
Author: Jimmy DE LA TORRE
Institution: The University of Hong Kong
Country: Hong Kong


Dates

Publication Date : June 13, 2020

APA Hou, L , Terzi̇, R , De La Torre, J . (2020). Wald Test Formulations in DIF Detection of CDM Data with the Proportional Reasoning Test . International Journal of Assessment Tools in Education , 7 (2) , 145-158 . DOI: 10.21449/ijate.689752