TY  - JOUR
T1  - The Impact of Missing Data on the Performances of DIF Detection Methods
AU  - Akcan, Rabia
AU  - Atalay Kabasakal, Kübra
PY  - 2023
DA  - March
Y2  - 2023
DO  - 10.21031/epod.1183617
JF  - Journal of Measurement and Evaluation in Education and Psychology
JO  - JMEEP
PB  - Association for Measurement and Evaluation in Education and Psychology
WT  - DergiPark
SN  - 1309-6575
SP  - 95
EP  - 105
VL  - 14
IS  - 1
LA  - en
AB  - This study analyzed the impact of missing data techniques on performances of two differential item functioning (DIF) detection methods (Mantel Haenszel and Multiple Indicator and Multiple Causes) under missing completely at random missing data mechanism. Percentage of missing data was set at 5% and 15%. Zero imputation, listwise deletion and fractional hot-deck imputation were used to handle missing data. The data set of the study consisted of 17 items in the S12 item cluster of Programme for International Student Assessment (PISA) 2015 science test. Results showed that fractional hot-deck imputation produced the best results in identifying DIF items in all conditions and it had also the closest DIF values to the values obtained from complete data set. It was also found that multiple indicator and multiple causes method was more adversely affected than Mantel Haenszel by the presence of missing data.
KW  - Differential item functioning
KW  - Mantel Haenszel
KW  - MIMIC
KW  - missing data
CR  - Banks, K. (2015). An introduction to missing data in the context of differential item functioning. Practical Assessment, Research & Evaluation, 20(12), 1-10. https://eric.ed.gov/?id=EJ1059748
CR  - Banks, K., & Walker, C. (2006, April). Performance of SIBTEST when focal group examinees have missing data. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco, CA.
CR  - Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items. London Sage.
CR  - Clauser, B. E., & Mazor, K. M. (1998). Using statistical procedures to identify differentially functioning test items. Educational Measurement Issues and Practice, 17(1), 31-44. https://eric.ed.gov/?id=EJ564712
CR  - Doğanay Erdoğan, B. (2012). Çoklu atama yöntemlerinin Rasch modelleri için performansının benzetim çalışması ile incelenmesi [Assessing the performance of multiple imputation techniques for Rasch models with a simulation study] (Publication No. 314412) [Doctoral dissertation, Ankara University]. Council of Higher Education Thesis Center.
CR  - Dong, Y., & Peng, C. Y. (2013). Principled missing data methods for researchers. Springer Plus, 2(1), 222. https://doi.org/10.1186/2193-1801-2-222
CR  - Emenogu, B. C., Falenchuk, O., & Childs, R. A. (2010). The effect of missing data treatment on Mantel-Haenszel DIF detection. The Alberta Journal of Educational Research, 56(4), 459-469. https://doi.org/10.11575/ajer.v56i4.55429
CR  - Finch, H. (2011a). The use of multiple imputation for missing data in uniform DIF analysis: Power and type I error rates. Applied Measurement in Education, 24(4), 281-301. https://doi.org/10.1080/08957347.2011.607054
CR  - Finch, H. (2011b). The impact of missing data on the detection of nonuniform differential item functioning. Educational and Psychological Measurement, 71(4), 663-683. https://doi.org/10.1177/0013164410385226
CR  - Finch, H. (2005). The MIMIC model as a method for detecting DIF: Comparison with Mantel-Haenszel, SIBTEST, and the IRT likelihood ratio. Applied Psychological Measurement, 29(4), 278-295. https://doi.org/10.1177/0146621605275728
CR  - Finch, H. W., & French, B. F. (2007). Detection of crossing differential item functioning a comparison of four methods. Educational and Psychological Measurement, 67(4), 565-582. https://doi.org/10.1177/0013164406296975
CR  - Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education McGraw-hill.
CR  - Garrett, P. L. (2009). A monte carlo study investigating missing data, differential item functioning, and effect size (Publication No. 3401601) [Doctoral dissertation, Georgia State University]. ProQuest Dissertations Publishing.
CR  - Hallquist, M., & Wiley, J. (2018). MplusAutomation: An R package for facilitating large-scale latent variable analyses in Mplus. Structural Equation Modeling: A Multidisciplinary Journal, 25(4), 621-638. https://doi.org/10.1080/10705511.2017.1402334
CR  - Harrington, D. (2009). Confirmatory factor analysis. Oxford University Press.
CR  - Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer, & H. I. Braun, Test Validity (pp. 129-145). Lawrence Erlbaum.
CR  - Holland, P. W., & Wainer, H. (1993). Differential item functioning. Lawrence Erlbaum.
CR  - Im, J., Cho, I. H., & Kim, J. K. (2018). FHDI: Fractional hot deck and fully efficient fractional imputation. https://CRAN.R-project.org/package=FHDI
CR  - Im, J., Kim, J. K., & Fuller, W. A. (2015). Two-phase sampling approach to fractional hot deck imputation. In Proceedings of the Survey Research Methods Section, pages 1030-1043. http://www.asasrms.org/Proceedings/y2015/files/233957.pdf
CR  - Jin, KY., & Chen, HF. (2020). MIMIC approach to assessing differential item functioning with control of extreme response style. Behavior Research Methods, 52, 23-35. https://doi.org/10.3758/s13428-019-01198-1
CR  - Kalton, G., & Kish, L. (1984). Some efficient random imputation methods. Communications in Statistics-Theory and Methods ,13(16), 1919-1939. https://doi.org/10.1080/03610928408828805
CR  - Kim, J. K., & Fuller, W. (2004). Fractional hot deck imputation. Biometrika, 91(3), 559-578. https://doi.org/10.1093/biomet/91.3.559
CR  - Magis, D., Beland, S., Tuerlinckx, F., & De Boeck, P. (2010). A general framework and an R package for the detection of dichotomous differential item functioning. Behavior Research Methods, 42(3), 847-862. https://doi.org/10.3758/BRM.42.3.847
CR  - Montoya, A. K., & Jeon, M. (2020). MIMIC models for uniform and nonuniform DIF as moderated mediation models. Applied Psychological Measurement, 44(2), 118-136. https://doi.org/10.1177/0146621619835496
CR  - Peugh, J. L., & Enders, C. K. (2004). Missing data in educational research:A review of reporting practices and suggestions for improvement. Review of Educational Research, 74(4), 525-556. https://journals.sagepub.com/doi/pdf/10.3102/00346543074004525
CR  - Robitzsch, A., & Rupp, A. A. (2009). Impact of missing data on the detection of differential item functioning: The case of Mantel-Haenszel and logistic regression analysis. Educational and Psychological Measurement, 69(1), 18-34. https://doi.org/10.1177/0013164408318756
CR  - Rousseau, M., Bertrand, R., & Boiteau, N. (2004, April). Impact of missing data on robustness of DIF IRT-based and non IRT-based methods. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.
CR  - Rubin, D. B. (1976). Inference and missing data. Biometrika, 63(3), 581-592. https://doi.org/10.1093/biomet/63.3.581
CR  - Scheuneman, J. (1979). A method of assessing bias in test items. Journal of Educational Measurement, 16, 143–152. https://www.jstor.org/stable/1433816
CR  - Shih, C. L., & Wang, W. C. (2009). Differential item functioning detection using multiple indicators, multiple causes method with a pure short anchor. Applied Psychological Measurement, 33(3), 184-199. https://doi.org/10.1177/0146621608321758
CR  - Tamcı, P. (2018). Kayıp veriyle baş etme yöntemlerinin değişen madde fonksiyonu üzerindeki etkisinin incelenmesi [Investigation of the impact of techniques of handling missing data on differential item functioning] (Publication No. 517260) [Master's dissertation, Hacettepe University]. Council of Higher Education Thesis Center.
CR  - Uğurlu, S., & Atar, B. (2020). Performances of MIMIC and logistic regression procedures in detecting DIF. Journal of Measurement and Evaluation in Education and Psychology, 11(1), 1-12. https://doi.org/10.21031/epod.531509
CR  - Woods, C. M. (2009). Evaluation of MIMIC-model methods for DIF testing with comparison to two-group analysis. Multivariate Behavioral Research, 44(1), 1-27. https://doi.org/10.1080/00273170802620121
CR  - Zieky, M. (1993). Practical questions in the use of DIF statistics in test development. In P. W. Holland, & H. Wainer, Differential Item Functioning (pp. 337-347). Lawrence Erlbaum.
CR  - Zumbo, B. D. (2007). Three generation of DIF analyses: Considering where it has been, where it is now, and where it is going. Language Assessment Quarterly, 4(2), 223–233. https://doi.org/10.1080/15434300701375832
UR  - https://doi.org/10.21031/epod.1183617
L1  - https://dergipark.org.tr/en/download/article-file/2685513
ER  -