An Investigation of Item Bias in PISA Science Test in Terms of The Language and Culture

Nezaket Bilge Uzun; Selahattin Gelbal

EN TR

An Investigation of Item Bias in PISA Science Test in Terms of The Language and Culture

Abstract

In this study, differential item functioning (DIF) analyses of Science items of PISA 2006 tests were carried out between different samplings. Mantel Haenszel (MH), logistic regression(LR) and signed - unsigned area indexes methods were used. The research group of this study consists of the samples of Australia, Canada; England, Turkey. In order to investigate the sources of DIF field specialist opinions were consulted for released multiple choice items of science test. It is observed that as the linguistic and cultural differences increased between countries, the number of DIF items increased. The number of DIF items varied significantly according to the procedure used. There was not consistency according to DIF detecting method in DIF or nonDIF items. Generally; like other results of bias researchs this research indicated that the main possible reasons for DIF is due to differences in translation, curriculum,cultural relevance, linguistic differences across Turkish and English versions of the tests of released items.

Keywords

Differential Item Functioning,Mantel-Haenszel,Logistic Regression,SignedUnsigned Area Indexes,PISA

References

Abedalaziz N.,2011, Detecting DIF using Item Characteristic Curve Approaches.The International Journal of Educational and Psychological Assessment , Vol. 8(2)
Abedi J. (2008). Classification System for English Language Learners: Issues and Recommendations Educational Measurement: Issues and Practice; 27(3), 17-31.
Abedi, J. (2004). The No Child Left Behind Act and English language learners: Assessment and accountability issues. Educational Researcher, 33, 4-14.
Asil, M. (2010). Uluslararası Ögrenci Degerlendirme Programı (PISA) 2006 ögrenci anketinin kültürler arası esdegerliginin incelenmesi. Yayınlanmamıs doktora tezi, Hacettepe Üniversitesi, Ankara.
Asil, M. Ve Gelbal, S. (2012). PISA Öğrenci Anketinin Kültürler Arası Eşdeğerliği, Eğitim ve Bilim, 37, 166, 236-249.
Atalay K. (2010) PISA 2006 Öğrenci Anketinde Yer Alan Tutum Maddelerinin Değişen Madde Fonksiyonu Açısından İncelenmesi. Yüksek Lisans Tezi Eğitim Bilimleri Bölümü, Hacettepe Üniversitesi, Ankara.
Bakan Kalaycıoglu, D. ve Kelecioglu, H. (2011). Ögrenci Seçme Sınavı’nın Madde Yanlılıgı Açısından incelenmesi. Egitim ve Bilim, 36, 3-13.
Baker, Frank. (2001). The Basics of Item Response Theory. ERIC Clearinghouse on Assessment and Evaluation
Camilli, G. (2006). Test fairness. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 221-256). Westport: American Council on Education ve Praeger Publishers.
Camilli, G., Shepard, L.A. (1994). Methods for identifying biased test items. Thousand Oaks: Sage.

Deveci Ateşok, N. (2008) Üniversitelerarası Kurul Yabancı Dil Sınavının Madde Yanlılığı Bakımından İncelenmesi; Doktora Tezi, Ölçme ve Değerlendirme Anabilim Dalı, Ankara Üniversitesi
Deveci, N. (2008). Üniversitelerarası Kurul Yabancı Dil Sınavı’nın Madde Yanlılığı bakımından incelenmesi. Ankara Üniversitesi Eğitim Bilimleri Enstitüsü Eğitimde Psikolojik Hizmetler Anabilim Dalı. Yayınlanmamış Doktora Tezi, Ankara.
Ercikan, K., (1998), Translation Effects In International Assessments, International Journal of Educational Research, 29, 543-553.
Gierl, M., Jodoin, G. M. Ackerman, T. A. (2000). Performance of Mantel‐Haenszel, Simultaneous Item Bias Test, and Logistic Regression when the proportion of DIF items is Large. Paper Presented at the Annual Meeting of the American educational Research Association (AERA). New Orleans, Louisiana, USA.
Girl M.J., KhaliqS.N.. (2001) Identifying Sources of Differential Item and Bundle Functioning on Translated Achievement Tests: A Confirmatory Analysis Journal of Educational Measurement; Vol. 38, No. 2,; pp. 164-187
Gondal, M., B,(2001). Pakistan`daki 4. Sınıf Ve Urdu (Ulusal Dil) Başarı Test Maddelerinın Madde Yanlılığı Açısından İncelenmesi. Yayınlanmamış Doktora Tezi, Orta Doğu Teknik Üniversitesi, Eğitim Bilim Bölümü, Ankara.
Grisay, A. (2003). Translation Procedures in OECD/PISA 2000 International Assessment. Language Testing, 20(2), 225-240
Hambleton, R. K., Swaminathan, H. (1985) Item Response Theory: Principles and Applications. Boston: Kluwer-Nijhoff Publishing.
Hambleton, R.K. Rogers, H., J.( 1996). Developing an Item Bias Review Form. Which is Available Through ERIC/AE.
Lord, M.F., Novic, R. M. (1968). Statistical theories of mental test scores. New York: Addison-Wesley Publishing Company.
Millsap, R.E. (2005). Four unresolved problems in studies of factorial invariance. In A. Maydeu-Olivares & J.J. McArdle (Eds.) Contemporary Psychometrics. (pp. 153-171) Mahwah, NJ: Lawrence Erlbaum.
Millsap, R.E. Meredith, W. (1992). Component analysis in multivariate aging research. Experimental Aging Research, 18, 203-212
Osterlind, S.J. (1983). Test item bias. Beverly Hills: Sage
Öğretmen, T. (2006). Uluslararası Okuma Becerilerinde Gelişim Projesi (PIRLS) Testinin psikometrik Özelliklerinin İncelenmesi: Türkiye- Amerika Birleşik 2001 Devletleri Örneği. Doktora Tezi. Eğitim Bilimleri Bölümü, Hacettepe Üniversitesi.
Öğretmen, T., Doğan, N. (2004). OKÖSYS Matematik alt testine ait maddelerin yanlılık analizi. İnönü Üniversitesi Eğitim Fakültesi Dergisi. 8, 61-76.
PISA 2006 Uluslararası Öğrenci Başarılarını Değerlendirme Programı Ulusal Ön Rapor (2007) http://earged.meb.gov.tr/dosyalar adresinden indirilmiştir.
Raju, N. S. (1990). The area between two item characteristic curves. Psychometrika, 53, 495–502.
Raju, N. S., Ellis, B. B. (2002). Differential item and test functioning. In F. Drasgow & N. Schmitt (Eds.), Measuring and analyzing behavior in organizations (p. 156-188). San Francisco, CA: Jossey-Bass.
Yıldırım H, Berberoglu G. ;Judgmental and statistical DIF analyses of the PISA-2003 mathematics literacy items. International Journal of Testing, 2009; 9(2):108-121. ( PsycInfo )
Yıldırım, H.(2006). The Dıfferentıal Item Functıonıng (DIF) Analysis Of Mathematics İtems in The İnternational Assessment Programs. Yayımlanmamış Doktora Tezi. Orta Doğu Teknik Üniversitesi, Matematik Eğitim Bölümü. Ankara.
Zieky, M. (2003). A DIF primer. Princeton, NJ: Educational Testing Service.
Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores. Ottawa, Canada: Directorate of Human Resources Research and Evaluation, Department of National Defense

Details

Primary Language

English

Subjects

Studies on Education

Journal Section

Research Article

Authors

Nezaket Bilge Uzun This is me

Selahattin Gelbal This is me

Publication Date

November 15, 2017

Submission Date

November 14, 2017

Acceptance Date

March 27, 2017

Published in Issue

Year 2017 Volume: 25 Number: 6

IZ

https://izlik.org/JA27TX79CF

APA

Uzun, N. B., & Gelbal, S. (2017). An Investigation of Item Bias in PISA Science Test in Terms of The Language and Culture. Kastamonu Education Journal, 25(6), 2427-2446. https://izlik.org/JA27TX79CF

AMA

1.Uzun NB, Gelbal S. An Investigation of Item Bias in PISA Science Test in Terms of The Language and Culture. Kastamonu Education Journal. 2017;25(6):2427-2446. https://izlik.org/JA27TX79CF

Chicago

Uzun, Nezaket Bilge, and Selahattin Gelbal. 2017. “An Investigation of Item Bias in PISA Science Test in Terms of The Language and Culture”. Kastamonu Education Journal 25 (6): 2427-46. https://izlik.org/JA27TX79CF.

EndNote

Uzun NB, Gelbal S (November 1, 2017) An Investigation of Item Bias in PISA Science Test in Terms of The Language and Culture. Kastamonu Education Journal 25 6 2427–2446.

IEEE

[1]N. B. Uzun and S. Gelbal, “An Investigation of Item Bias in PISA Science Test in Terms of The Language and Culture”, Kastamonu Education Journal, vol. 25, no. 6, pp. 2427–2446, Nov. 2017, [Online]. Available: https://izlik.org/JA27TX79CF

ISNAD

Uzun, Nezaket Bilge - Gelbal, Selahattin. “An Investigation of Item Bias in PISA Science Test in Terms of The Language and Culture”. Kastamonu Education Journal 25/6 (November 1, 2017): 2427-2446. https://izlik.org/JA27TX79CF.

JAMA

1.Uzun NB, Gelbal S. An Investigation of Item Bias in PISA Science Test in Terms of The Language and Culture. Kastamonu Education Journal. 2017;25:2427–2446.

MLA

Uzun, Nezaket Bilge, and Selahattin Gelbal. “An Investigation of Item Bias in PISA Science Test in Terms of The Language and Culture”. Kastamonu Education Journal, vol. 25, no. 6, Nov. 2017, pp. 2427-46, https://izlik.org/JA27TX79CF.

Vancouver

1.Nezaket Bilge Uzun, Selahattin Gelbal. An Investigation of Item Bias in PISA Science Test in Terms of The Language and Culture. Kastamonu Education Journal [Internet]. 2017 Nov. 1;25(6):2427-46. Available from: https://izlik.org/JA27TX79CF

Kastamonu Education Journal is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license.

An Investigation of Item Bias in PISA Science Test in Terms of The Language and Culture

An Investigation of Item Bias in PISA Science Test in Terms of The Language and Culture

Abstract

Keywords

PISA Fen Başarı Testinin Madde Yanlılığının Kültür ve Dil Açısından İncelenmesi

Abstract

Keywords

References

Details

Primary Language

Subjects

Journal Section

Authors

Publication Date

Submission Date

Acceptance Date

Published in Issue

IZ

Cite