Examination of TIMSS Mathematics Data with Multilevel Measurement Models in Respect to Content, Cognitive and Topic Areas

Abstract

This research study aims to identify TIMMS 8th grade mathematics item groups and the specification of items in which Turkish 8th grade students have signıfıcantly lower level of correct responses compared to all other 8th grade participants. For this purpose, total 260 (82 from 1999, 88 from 2007, and 90 from 2011)  items released by International Association for the Evaluation of Educational Achievement (IEA) were grouped according to cognitive, content and sub-content domains. Then, mean correct responses of released items for each participant country were obtained from IEA’s yearly almanac. Finally, data were analyzed by using Multilevel Measurement Models and differences in achievement levels between Turkish 8th graders and their peers from other participating countries were predicted and tested in the context of item groups. Analysis of data showed that performance of Turkish students statistically significantly lower than performance of students from rest of the other participant countries in Number (Content Domain)-Fractions and Decimals (Topic Area)-Knowing (Cognitive Domain) item group. Detailed investigation revealed that students generally fail in procedures in fractions and conversions among fraction, decimal, and percent.

Keywords

• Two-level hierarchical linear measurement models,mathematics performance level,TIMSS

TIMSS Matematik Verilerinin Aşamalı Ölçme Modelleri ile İçerik, Bilişsel ve Konu Alanları Bakımından İncelenmesi

Öz

Bu çalışmanın amacı, TIMSS 8. sınıf matematik testlerine katılan öğrencilerimizin dünyanın başka yerlerinde yaşayan akranlarına göre doğru cevaplamada güçlük çektikleri madde gruplarını ve bu gruplarını oluşturan maddelerin özelliklerini ortaya çıkarmaktır. Bu amaç dahilinde, ilk olarak, Türkiye’nin 1999, 2007 ve 2011 yıllarında katıldığı TIMSS 8. sınıf matematik testlerinin International Association for the Evaluation of Educational Achievement (İEA) tarafından yayınlanan bütün maddeleri (toplam 260 adet), yine IEA tarafından yayınlanan değerlendirme çerçevesi dokümanları ve belirtke tabloları takip edilerek içerik, bilişsel alan ve konu alanı itibariyle gruplara ayrılmıştır. Ardından, IEA’nın bir yıllık olarak hazırladığı almanaklar kullanılarak bu maddelere ait ortalama doğru cevaplama oranları ve bu oranların TIMSS katılımcıları arası dağılımları çıkarılmıştır. Son olarak, elde edilen veriler aşamalı doğrusal ölçme modelleri ile analiz edilerek madde grupları bağlamında Türkiye ile diğer TIMSS katılımcıları arası başarı düzeyi farklılıkları tahmin edilip test edilmiştir. Bulgular öğrencilerimizin “Sayılar” içerik alanı altında bulunan “Kesirler ve Ondalıklı Sayılar” konusu ile ilgili olgular, kavramlar ve yöntemlere ait bilgi düzeylerinin diğer ülkelerde yaşayan akranlarına göre oldukça düşük olduğunu göstermekte ve bu farklılık istatistiksel olarak anlamlı bulunmaktadır.

Anahtar Kelimeler

• İki aşamalı doğrusal ölçme modelleri,matematik başarı düzeyi,TIMSS

References

1. Acar, T. (2011). Maddenin Farklı Fonksiyonlaşmasında Örneklem Büyüklüğü: Genelleştirilmiş Aşamalı Doğrusal Modelleme Uygulaması. Kuram ve Uygulamada Eğitim Bilimleri 11(1), 279-288.
2. Bauer D. J. (2003). Estimating Multilevel Linear Models as Structural Equation Models. Journal of Educational and Behavioral Statistics. Vol. 28, No. 2, pp. 135-167.
3. Behr, M. J., Harel, G., Post, T., & Lesh, R. (1992). Rational number, ratio, and proportion. In D. A.Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp. 296–333). New York: Macmillan.
4. Binici, S. (2007). Random-effects differential item functioning via hierarchical generalized linear model and generalized linear latent mixed model: A comparison of estimation methods. Unpublished doctoral dissertation, Florida State University.
5. Birgin, O. & Gürbüz, R. (2009). İlköğretim İkinci Kademe Öğrencilerinin Rasyonel Sayılar Konusundaki İşlemsel ve Kavramsal Bilgi Düzeylerinin İncelenmesi. Uludağ Üniversitesi Eğitim Fakültesi Dergisi, 22(2), 529-550.
6. Bulgar, S. (2003). Children’s sense-making of division of fractions. Journal of Mathematical Behavior, 22, 319-334.
7. Carpenter, T. (1988). Teaching as problem solving. In R. Charles, & E. Silver (Eds.), The teaching and assessing of mathematical problem solving (pp. 187–202). Reston, VA: National Council of Teachers of Mathematics.
8. Chaimongkol, S. (2005). Modeling differential item functioning (dif) using multilevel logistic regression models: A Bayesian Perspective. Unpublished doctoral dissertation. Florida State University.

9. Chu KL, Kamata A.(2005). Test equating in the presence of DIF items. Journal of Appl Meas. 6(3):342-54.
10. Davis, R. B., & Maher, C. A. (1990). What do we do when we ‘do mathematics’? In: N. Noddings (Ed.), Constructivist views of the teaching and learning of mathematics (Vol. Monograph No. 4). Reston, VA: National Council of Teachers of Mathematics.
11. De Corte, E., Greer, B., & Verschaffel, L. (1996). Mathematics teaching and learning. In D. C. Berliner,& R. C. Calfee (Eds.), Handbook of educational psychology (pp. 491–549). New York: Macmillan.
12. De Corte, E., Verschaffel, L., & Pauwels, A. (1990). Influence of the semantic structure of word problems on second graders’ eye movements. Journal of Educational Psychology, 82, 359–365.
13. Fischbein, E., Deri, M., Nello, M. S., & Marino, M. S. (1985). The role of implicit models in solving verbal problems in multiplication and division. Journal for Research in Mathematics Education, 1985(16), 3–17.
14. Greeno, J. G. (1991). Number sense as situated knowing in a conceptual domain. Journal for Research in Mathematics Education, 22(3), 170–218.
15. Kamata, A. (2001). Item Analysis by the Hierarchical Generalized Linear Model. Journal of Educational Measurement. Vol. 38, No. 1 (Spring, 2001) , pp. 79-93.
16. Kamata, A., Bauer, D.J. & Miyazaki, Y. (2008). Multilevel measurement modeling. In A.A. O'Connell & D.B. McCoach (Eds.) Multilevel Modeling of Educational Data (pp. 345-388). Charlotte, NC: Information Age Publishing.
17. Kamata, A. (2001). Item Analysis by the Hierarchical Generalized Linear Model. Journal of Educational Measurement, 38, 79-93.
18. Kamata, A., & Binici, S. (2003). Random Effect DIF Analysis via Hierarchical Generalized Linear Modeling. Paper presented at the biannual meeting of Psychometric Society, July, Sardinia, Italy.
19. Kamata, A., & Vaughn, B. K. (2004). An Introduction to Differential Item Functioning Analysis. Learning Disabilities: A Contemproary Journal, 2(2), 49-64.
20. Kamata, A., Chaimongkol, S., Genc, E., & Bilir, K. (2005). Random-Effect Differential Item Functioning Across Group Unites by the Hierarchical Generalized Linear Model. Paper presented at the annual meeting of the American Educational Research Association, April, Montreal, Canada.
21. Kim, W. (2003). Development of a differential item functioning (DIF) procedure using the hierarchical generalized model: a comparison study with logistic regression procedure. Unpublished doctoral dissertation. Pennsylvania State University.
22. Lamon, S. J. (1999). Teaching fractions and ratios for understanding. Mahwah, NJ, USA: Lawrence Erlbaum Associates, Inc.
23. Lehtinen, E., Merenluoto, K., & Kasanen, E. (1997). Conceptual change in mathematics: from rational to (un)real numbers. European Journal of Psychology of Education, XII(2), 131–145.
24. Luppescu, S. (2002). DIF detection in HLM. Paper presented at the annual meeting of the American Educational Research Association, New Orleans.
25. Maher, C. A., and Alston, A. (1989). Is meaning connected to symbols? An interview with Ling Chen. The Journal of Mathematical Behavior, 8(3), 241-248.
26. Miyazaki, Y.& Kenneth A. F. (2006). A Hierarchical Linear Model with Factor Analysis Structure at Level 2. Journal of Educational and Behavioral Statistics.Vol. 31, No. 2, pp. 125-156.
27. R Development Core Team (2012). R: A language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.
28. Raudenbush, S.W., Bryk, A.S, & Congdon, R. (2004). HLM 6 for Windows. Hierarchical linear and non-linear modeling. Lincolnwood, IL: Scientific Software International.
29. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. 2nd edition. Sage Publications Inc. Newbury, CA.
30. Resnick, L. B., & Ford, W. W. (1981). The psychology of mathematics for instruction. Hillsdale, NJ: Lawrence Erlbaum Associates.
31. Resnick, L. B., Nesher, P., Leonard, F., Magone, M., Omanson, S., & Peled, I. (1989). Conceptual bases of arithmetic errors: the case of decimal fractions. Journal of Research in Mathematics Education, 20(1), 8–27.
32. Resnick, L. B., & Omanson, S. (1987). Learning to understand arithmetic. In R. Glaser (Ed.), Advances in instructional psychology, Vol. 3. (pp. 41–95). Hillsdale, NJ: Erlbaum.
33. Skrondal, A. & Rabe-Hesketh, S. (2004). Generalized Latent Variable Modeling: Multilevel, Longitudinal, and Structural Equation Models. New York: Chapman & Hall/CRC.