Seviye Belirleme Sınavında Değişen Madde ve Değişen Çeldirici Fonksiyonu Analizleri

Abstract

Testin adil olması başarı testi hazırlamadaki vazgeçilmez unsurlardan biridir. Değişen madde fonksiyonu (DMF) ve değişen çeldirici fonksiyonu (DÇF) analizleri test adilliğini değerlendirmede birbirlerini tamamlayan rollere sahiptirler. Bu çalışma Türkiye’de yapılan geniş ölçekli bir teste katılan öğrencilerin cinsiyetlerine göre doğru ve yanlış yanıtlarını ve çeldirici seçimlerini DMF ve DÇF yöntemleri ile incelemeyi amaçlamaktadır. Bu amaçla 2011 yılı Seviye Belirleme Sınavı Matematik bölümü DMF ve DÇF açısından analiz edilmiştir. DMF analizleri için, Mantel-Haenszel ve Lojistik Regresyon metotları kullanılmıştır. DÇF analizleri için, iki parametreli lojistik kümelenmiş logit ve nominal yanıt modellerinin altındaki iki olasılık oranı yaklaşımları kullanılmıştır. Sonuçlara göre 500 ve 1,000 örneklem büyüklüğünde DMF bulunmazken sadece 2,000 örneklem büyüklüğünde anlamlı DMF sonuçları bulunmuştur. 20 maddelik testin beş maddesinde DMF ve bu beş maddenin üçünde de DÇF görülmüştür. 

Keywords

• DMF,DÇF,SBS

Differential Item and Differential Distractor Functioning Analyses on Turkish High School Entrance Exam

Abstract

Test fairness is one of the most critical elements in creating assessments. Differential item functioning (DIF) and differential distractor functioning (DDF) analyses play complementary roles in justifying test fairness. This study aims to investigate the correct (i.e., DIF) and incorrect (i.e., DDF) response choices of students based on gender in a standardized high-stakes test administered in Turkey. Given the purpose of this study, the Math section of 2011 Turkish High School Entrance Exam was investigated. For DIF analyses, Mantel Haenszel and Logistic Regression methods were used. For DDF analyses, two odds ratio approaches under the two-parameter logistic-nested logit model and nominal response model were used. According to the findings, in 500 and 1,000 sample sizes, DIF was not detected, however, only a 2,000 sample size indicated significant DIF results. Five DIF items were observed among 20 items, where three out of those five DIF items also showed DDF.

Keywords

• DIF,DDF,SBS

References

1. Abedi, J., Leon, S., & Kao, J. (2007). Examining differential distractor functioning in reading assessments for students with disabilities. Minneapolis, MN: University of Minnesota, Partnership for Accessible Reading Assessment.
2. Arikan, Ç. A., Uğurlu, S., & Atar, B. (2016) Mimic, Sibtest, Lojistik Regresyon ve Mantel-Haenszel Yöntemleriyle Gerçekleştirilen DMF ve Yanlılık Çalışması . Hacettepe Eğitim Fakültesi Dergisi 31(1): 34-52.
3. Camilli, G. (2006). Test fairness. Educational measurement, 4, 221-256.
4. DeMars, C. E., & Lau, A. (2011). Differential item functioning detection with latent classes: How accurately can we detect who is responding differentially? Educational and Psychological Measurement, 71, 597–616.
5. Doğan, N., & Öğretmen, T. (2010). Değişen Madde Fonksiyonunu Belirlemede Mantel‐Haenszel, Ki‐Kare ve Lojistik Regresyon Tekniklerinin Karşılaştırılması. Eğitim ve Bilim, 33(148), 100-112.
6. Doornik, J.A. (2002), Object-Oriented Matrix Programming Using Ox, 3rd ed. London: Timberlake Consultants Press and Oxford: www.nuff.ox.ac.uk/Users/Doornik.
7. Dorans, N. J., & Holland, P. W. (1993). DIF detection and description: Mantel-Haenszel and standardization. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 35–66). Hillsdale, NJ: Erlbaum.Penfield, R. D. (2008). An odds ratio approach for assessing differential distractor functioning effects under the nominal response model. Journal of Educational Measurement, 45, 247–269.
8. Gierl, M., Khaliq, S. N., Bougthon, K. (1999). Gender differential item functioning in mathematics and science: Prevalence and policy implications. Paper presented at the symposium entitled “Improviming large – scale assessment in education” at the Annual Meeting of the Canadian Society for the Study of Education, Canada, June, 1999.

9. Green, B. F., Crone, C. R., & Folk, V. G. (1989). A method for studying differential distractor functioning. Journal of Educational Measurement, 26, 147–160.
10. Haladyna, T. M., & Downing, S. M. (2004). Construct-irrelevant variance in high-stakes testing Educational Measurement: Issues and Practice, 23, 17–27.
11. Hidalgo, M. D., & LÓPez-Pina, J. A. (2004). Differential item functioning detection and effect size: A comparison between logistic regression and Mantel-Haenszel procedures. Educational and Psychological Measurement, 64(6), 903-915.
12. Holland, P. W., & Thayer, D. T. (1986). Differential Item Functioning and the Mantel‐Haenszel Procedure. ETS Research Report Series, 1986(2).
13. Kan, A., Sünbül, Ö., & Ömür, S., (2013). 6.-8. Sınıf seviye belirleme sınavları alt testlerinin çeşitli yöntemlere göre değişen madde fonksiyonlarının incelenmesi. Mersin Üniversitesi Eğitim Fakültesi Dergisi, 9(2).
14. Karakaya, İ. (2012). Seviye Belirleme Sınavındaki Fen ve Teknoloji ile Matematik alt testlerinin madde yanlılığı açısından incelenmesi. Kuram ve Uygulamada Eğitim Bilimleri, 12(1), 222-229.
15. Kelecioğlu, H., Karabay, B., & Karabay, E. (2014). Seviye belirleme sınavı’nın madde yanlılığı açısından incelenmesi. İlköğretim Online, 13(3). Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the national cancer institute, 22(4), 719-748.
16. Marshall, S. P. (1983). Sex differences in mathematical errors: An analysis of distractor choices. Journal for Research in Mathematics Educations, 14, 325–336.
17. Mazor, K. M., Clauser, B. E., & Hambleton, R. K. (1992). The effect of sample size on the functioning of the Mantel-Haenszel statistic. Educational and Psychological Measurement, 52(2), 443-451.
18. Mazor, K. M., Kanjee, A. and Clauser, B. E. (1995), Using Logistic Regression and the Mantel-Haenszel With Multiple Ability Estimates to Detect Differential Item Functioning. Journal of Educational Measurement, 32: 131–144. doi:10.1111/j.1745-3984.1995.tb00459.x
19. Magis, D., Beland, S., Tuerlinckx, F., De Boeck, P., (2010). A general framework and an R package for the detection of dichotomous differential item functioning. Behavior Research Methods, 42, 847-862.
20. Ministry of National Education of Turkey (2011) Retrieved from http://www.meb.gov.tr/duyurular/duyurular2011/EGITEK/sbs2011BasinBulteni/01_2011SBS_8SayisalBilgiler.pdf .
21. Rogers, H. J., & Swaminathan, H. (1993). A comparison of logistic regression and Mantel-Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17(2), 105-116.
22. Scott, N. W., Fayers, P. M., Aaronson, N. K., Bottomley, A., de Graeff, A., Groenvold, M., ... & EORTC Quality of Life Group. (2009). A simulation study provided sample size guidance for differential item functioning (DIF) studies using short scales. Journal of Clinical Epidemiology, 62(3), 288-295.
23. Suh, Y., & Bolt, D. M. (2011). A nested logit approach for investigating distractors as causes of differential item functioning. Journal of Educational Measurement, 48, 188–205.
24. Suh, Y., & Talley, A. E. (2015). An empirical comparison of DDF detection methods for understanding the causes of DIF in multiple-choice items. Applied Measurement in Education, 28, 48–67.
25. Swaminathan, H., Rogers, H.J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement 27, 361–370.
26. Terzi, R., & Suh, Y. (2015). An odds ratio approach for detecting DDF under the nested logit modeling framework. Journal of Educational Measurement, 52, 376–398. doi: 10.1111/jedm.12091
27. Thissen, D., & Steinberg, L. (1986). A taxonomy of item response models. Psychometrika, 51, 567–577.
28. Zieky, M. (1993). Practical questions in the use of DIF statistics in test development.
29. Zumbo, B. D., & Thomas, D. R. (1997). A measure of effect size for a model-based approach for studying DIF. Prince George, Canada: University of Northern British Columbia, Edgeworth Laboratory for Quantitative Behavioral Science.
30. Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF). Ottawa: National Defense Headquarters.
31. Zumbo, B. D. (2007). Three generations of DIF analyses: Considering where it has been, where it is now, and where it is going. Language assessment quarterly, 4, 223-233.