Research Article
Year 2018,
Volume: 9 Issue: 2, 136 - 149, 30.06.2018
Abstract
Testin adil olması başarı testi hazırlamadaki vazgeçilmez
unsurlardan biridir. Değişen madde fonksiyonu (DMF) ve değişen çeldirici
fonksiyonu (DÇF) analizleri test adilliğini değerlendirmede birbirlerini
tamamlayan rollere sahiptirler. Bu çalışma Türkiye’de yapılan geniş ölçekli bir
teste katılan öğrencilerin cinsiyetlerine göre doğru ve yanlış yanıtlarını ve
çeldirici seçimlerini DMF ve DÇF yöntemleri ile incelemeyi amaçlamaktadır. Bu
amaçla 2011 yılı Seviye Belirleme Sınavı Matematik bölümü DMF ve DÇF açısından
analiz edilmiştir. DMF analizleri için, Mantel-Haenszel ve Lojistik
Regresyon metotları kullanılmıştır. DÇF analizleri için, iki parametreli
lojistik kümelenmiş logit ve nominal yanıt modellerinin altındaki iki olasılık
oranı yaklaşımları kullanılmıştır. Sonuçlara göre 500 ve 1,000
örneklem büyüklüğünde DMF bulunmazken sadece 2,000 örneklem büyüklüğünde
anlamlı DMF sonuçları bulunmuştur. 20 maddelik testin beş maddesinde DMF ve bu
beş maddenin üçünde de DÇF görülmüştür.
References
- Abedi, J., Leon, S., & Kao, J. (2007). Examining differential distractor functioning in reading assessments for students with disabilities. Minneapolis, MN: University of Minnesota, Partnership for Accessible Reading Assessment.
- Arikan, Ç. A., Uğurlu, S., & Atar, B. (2016) Mimic, Sibtest, Lojistik Regresyon ve Mantel-Haenszel Yöntemleriyle Gerçekleştirilen DMF ve Yanlılık Çalışması . Hacettepe Eğitim Fakültesi Dergisi 31(1): 34-52.
- Camilli, G. (2006). Test fairness. Educational measurement, 4, 221-256.
- DeMars, C. E., & Lau, A. (2011). Differential item functioning detection with latent classes: How accurately can we detect who is responding differentially? Educational and Psychological Measurement, 71, 597–616.
- Doğan, N., & Öğretmen, T. (2010). Değişen Madde Fonksiyonunu Belirlemede Mantel‐Haenszel, Ki‐Kare ve Lojistik Regresyon Tekniklerinin Karşılaştırılması. Eğitim ve Bilim, 33(148), 100-112.
- Doornik, J.A. (2002), Object-Oriented Matrix Programming Using Ox, 3rd ed. London: Timberlake Consultants Press and Oxford: www.nuff.ox.ac.uk/Users/Doornik.
- Dorans, N. J., & Holland, P. W. (1993). DIF detection and description: Mantel-Haenszel and standardization. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 35–66). Hillsdale, NJ: Erlbaum.Penfield, R. D. (2008). An odds ratio approach for assessing differential distractor functioning effects under the nominal response model. Journal of Educational Measurement, 45, 247–269.
- Gierl, M., Khaliq, S. N., Bougthon, K. (1999). Gender differential item functioning in mathematics and science: Prevalence and policy implications. Paper presented at the symposium entitled “Improviming large – scale assessment in education” at the Annual Meeting of the Canadian Society for the Study of Education, Canada, June, 1999.
- Green, B. F., Crone, C. R., & Folk, V. G. (1989). A method for studying differential distractor functioning. Journal of Educational Measurement, 26, 147–160.
- Haladyna, T. M., & Downing, S. M. (2004). Construct-irrelevant variance in high-stakes testing Educational Measurement: Issues and Practice, 23, 17–27.
- Hidalgo, M. D., & LÓPez-Pina, J. A. (2004). Differential item functioning detection and effect size: A comparison between logistic regression and Mantel-Haenszel procedures. Educational and Psychological Measurement, 64(6), 903-915.
- Holland, P. W., & Thayer, D. T. (1986). Differential Item Functioning and the Mantel‐Haenszel Procedure. ETS Research Report Series, 1986(2).
- Kan, A., Sünbül, Ö., & Ömür, S., (2013). 6.-8. Sınıf seviye belirleme sınavları alt testlerinin çeşitli yöntemlere göre değişen madde fonksiyonlarının incelenmesi. Mersin Üniversitesi Eğitim Fakültesi Dergisi, 9(2).
- Karakaya, İ. (2012). Seviye Belirleme Sınavındaki Fen ve Teknoloji ile Matematik alt testlerinin madde yanlılığı açısından incelenmesi. Kuram ve Uygulamada Eğitim Bilimleri, 12(1), 222-229.
- Kelecioğlu, H., Karabay, B., & Karabay, E. (2014). Seviye belirleme sınavı’nın madde yanlılığı açısından incelenmesi. İlköğretim Online, 13(3). Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the national cancer institute, 22(4), 719-748.
- Marshall, S. P. (1983). Sex differences in mathematical errors: An analysis of distractor choices. Journal for Research in Mathematics Educations, 14, 325–336.
- Mazor, K. M., Clauser, B. E., & Hambleton, R. K. (1992). The effect of sample size on the functioning of the Mantel-Haenszel statistic. Educational and Psychological Measurement, 52(2), 443-451.
- Mazor, K. M., Kanjee, A. and Clauser, B. E. (1995), Using Logistic Regression and the Mantel-Haenszel With Multiple Ability Estimates to Detect Differential Item Functioning. Journal of Educational Measurement, 32: 131–144. doi:10.1111/j.1745-3984.1995.tb00459.x
- Magis, D., Beland, S., Tuerlinckx, F., De Boeck, P., (2010). A general framework and an R package for the detection of dichotomous differential item functioning. Behavior Research Methods, 42, 847-862.
- Ministry of National Education of Turkey (2011) Retrieved from http://www.meb.gov.tr/duyurular/duyurular2011/EGITEK/sbs2011BasinBulteni/01_2011SBS_8SayisalBilgiler.pdf .
- Rogers, H. J., & Swaminathan, H. (1993). A comparison of logistic regression and Mantel-Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17(2), 105-116.
- Scott, N. W., Fayers, P. M., Aaronson, N. K., Bottomley, A., de Graeff, A., Groenvold, M., ... & EORTC Quality of Life Group. (2009). A simulation study provided sample size guidance for differential item functioning (DIF) studies using short scales. Journal of Clinical Epidemiology, 62(3), 288-295.
- Suh, Y., & Bolt, D. M. (2011). A nested logit approach for investigating distractors as causes of differential item functioning. Journal of Educational Measurement, 48, 188–205.
- Suh, Y., & Talley, A. E. (2015). An empirical comparison of DDF detection methods for understanding the causes of DIF in multiple-choice items. Applied Measurement in Education, 28, 48–67.
- Swaminathan, H., Rogers, H.J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement 27, 361–370.
- Terzi, R., & Suh, Y. (2015). An odds ratio approach for detecting DDF under the nested logit modeling framework. Journal of Educational Measurement, 52, 376–398. doi: 10.1111/jedm.12091
- Thissen, D., & Steinberg, L. (1986). A taxonomy of item response models. Psychometrika, 51, 567–577.
- Zieky, M. (1993). Practical questions in the use of DIF statistics in test development.
- Zumbo, B. D., & Thomas, D. R. (1997). A measure of effect size for a model-based approach for studying DIF. Prince George, Canada: University of Northern British Columbia, Edgeworth Laboratory for Quantitative Behavioral Science.
- Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF). Ottawa: National Defense Headquarters.
- Zumbo, B. D. (2007). Three generations of DIF analyses: Considering where it has been, where it is now, and where it is going. Language assessment quarterly, 4, 223-233.
Differential Item and Differential Distractor Functioning Analyses on Turkish High School Entrance Exam
Abstract
Test fairness is one of the most critical elements in
creating assessments. Differential item functioning (DIF) and differential
distractor functioning (DDF) analyses play complementary roles in justifying
test fairness. This study aims to investigate the correct (i.e., DIF) and
incorrect (i.e., DDF) response choices of students based on gender in a
standardized high-stakes test administered in Turkey. Given the purpose of this
study, the Math section of 2011 Turkish High School Entrance Exam was
investigated. For DIF analyses, Mantel Haenszel and Logistic Regression methods
were used. For DDF analyses, two odds ratio approaches under the two-parameter
logistic-nested logit model and nominal response model were used. According to
the findings, in 500 and 1,000 sample sizes, DIF was not detected, however, only
a 2,000 sample size indicated significant DIF results. Five DIF items were
observed among 20 items, where three out of those five DIF items also showed
DDF.
References
- Abedi, J., Leon, S., & Kao, J. (2007). Examining differential distractor functioning in reading assessments for students with disabilities. Minneapolis, MN: University of Minnesota, Partnership for Accessible Reading Assessment.
- Arikan, Ç. A., Uğurlu, S., & Atar, B. (2016) Mimic, Sibtest, Lojistik Regresyon ve Mantel-Haenszel Yöntemleriyle Gerçekleştirilen DMF ve Yanlılık Çalışması . Hacettepe Eğitim Fakültesi Dergisi 31(1): 34-52.
- Camilli, G. (2006). Test fairness. Educational measurement, 4, 221-256.
- DeMars, C. E., & Lau, A. (2011). Differential item functioning detection with latent classes: How accurately can we detect who is responding differentially? Educational and Psychological Measurement, 71, 597–616.
- Doğan, N., & Öğretmen, T. (2010). Değişen Madde Fonksiyonunu Belirlemede Mantel‐Haenszel, Ki‐Kare ve Lojistik Regresyon Tekniklerinin Karşılaştırılması. Eğitim ve Bilim, 33(148), 100-112.
- Doornik, J.A. (2002), Object-Oriented Matrix Programming Using Ox, 3rd ed. London: Timberlake Consultants Press and Oxford: www.nuff.ox.ac.uk/Users/Doornik.
- Dorans, N. J., & Holland, P. W. (1993). DIF detection and description: Mantel-Haenszel and standardization. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 35–66). Hillsdale, NJ: Erlbaum.Penfield, R. D. (2008). An odds ratio approach for assessing differential distractor functioning effects under the nominal response model. Journal of Educational Measurement, 45, 247–269.
- Gierl, M., Khaliq, S. N., Bougthon, K. (1999). Gender differential item functioning in mathematics and science: Prevalence and policy implications. Paper presented at the symposium entitled “Improviming large – scale assessment in education” at the Annual Meeting of the Canadian Society for the Study of Education, Canada, June, 1999.
- Green, B. F., Crone, C. R., & Folk, V. G. (1989). A method for studying differential distractor functioning. Journal of Educational Measurement, 26, 147–160.
- Haladyna, T. M., & Downing, S. M. (2004). Construct-irrelevant variance in high-stakes testing Educational Measurement: Issues and Practice, 23, 17–27.
- Hidalgo, M. D., & LÓPez-Pina, J. A. (2004). Differential item functioning detection and effect size: A comparison between logistic regression and Mantel-Haenszel procedures. Educational and Psychological Measurement, 64(6), 903-915.
- Holland, P. W., & Thayer, D. T. (1986). Differential Item Functioning and the Mantel‐Haenszel Procedure. ETS Research Report Series, 1986(2).
- Kan, A., Sünbül, Ö., & Ömür, S., (2013). 6.-8. Sınıf seviye belirleme sınavları alt testlerinin çeşitli yöntemlere göre değişen madde fonksiyonlarının incelenmesi. Mersin Üniversitesi Eğitim Fakültesi Dergisi, 9(2).
- Karakaya, İ. (2012). Seviye Belirleme Sınavındaki Fen ve Teknoloji ile Matematik alt testlerinin madde yanlılığı açısından incelenmesi. Kuram ve Uygulamada Eğitim Bilimleri, 12(1), 222-229.
- Kelecioğlu, H., Karabay, B., & Karabay, E. (2014). Seviye belirleme sınavı’nın madde yanlılığı açısından incelenmesi. İlköğretim Online, 13(3). Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the national cancer institute, 22(4), 719-748.
- Marshall, S. P. (1983). Sex differences in mathematical errors: An analysis of distractor choices. Journal for Research in Mathematics Educations, 14, 325–336.
- Mazor, K. M., Clauser, B. E., & Hambleton, R. K. (1992). The effect of sample size on the functioning of the Mantel-Haenszel statistic. Educational and Psychological Measurement, 52(2), 443-451.
- Mazor, K. M., Kanjee, A. and Clauser, B. E. (1995), Using Logistic Regression and the Mantel-Haenszel With Multiple Ability Estimates to Detect Differential Item Functioning. Journal of Educational Measurement, 32: 131–144. doi:10.1111/j.1745-3984.1995.tb00459.x
- Magis, D., Beland, S., Tuerlinckx, F., De Boeck, P., (2010). A general framework and an R package for the detection of dichotomous differential item functioning. Behavior Research Methods, 42, 847-862.
- Ministry of National Education of Turkey (2011) Retrieved from http://www.meb.gov.tr/duyurular/duyurular2011/EGITEK/sbs2011BasinBulteni/01_2011SBS_8SayisalBilgiler.pdf .
- Rogers, H. J., & Swaminathan, H. (1993). A comparison of logistic regression and Mantel-Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17(2), 105-116.
- Scott, N. W., Fayers, P. M., Aaronson, N. K., Bottomley, A., de Graeff, A., Groenvold, M., ... & EORTC Quality of Life Group. (2009). A simulation study provided sample size guidance for differential item functioning (DIF) studies using short scales. Journal of Clinical Epidemiology, 62(3), 288-295.
- Suh, Y., & Bolt, D. M. (2011). A nested logit approach for investigating distractors as causes of differential item functioning. Journal of Educational Measurement, 48, 188–205.
- Suh, Y., & Talley, A. E. (2015). An empirical comparison of DDF detection methods for understanding the causes of DIF in multiple-choice items. Applied Measurement in Education, 28, 48–67.
- Swaminathan, H., Rogers, H.J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement 27, 361–370.
- Terzi, R., & Suh, Y. (2015). An odds ratio approach for detecting DDF under the nested logit modeling framework. Journal of Educational Measurement, 52, 376–398. doi: 10.1111/jedm.12091
- Thissen, D., & Steinberg, L. (1986). A taxonomy of item response models. Psychometrika, 51, 567–577.
- Zieky, M. (1993). Practical questions in the use of DIF statistics in test development.
- Zumbo, B. D., & Thomas, D. R. (1997). A measure of effect size for a model-based approach for studying DIF. Prince George, Canada: University of Northern British Columbia, Edgeworth Laboratory for Quantitative Behavioral Science.
- Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF). Ottawa: National Defense Headquarters.
- Zumbo, B. D. (2007). Three generations of DIF analyses: Considering where it has been, where it is now, and where it is going. Language assessment quarterly, 4, 223-233.
There are 31 citations in total.
Details
| Primary Language | English |
|---|---|
| Journal Section | Articles |
| Authors | |
| Publication Date | June 30, 2018 |
| Acceptance Date | March 26, 2018 |
| Published in Issue | Year 2018 Volume: 9 Issue: 2 |
