Examination of TIMSS Mathematics Data with Multilevel Measurement Models in Respect to Content, Cognitive and Topic Areas

Year 2017, Volume: 7 Issue: 2, 221 - 240, 29.12.2017

Önder Köklü

https://doi.org/10.17984/adyuebd.307020

Abstract

This research study
aims to identify TIMMS 8th grade mathematics item groups and the specification
of items in which Turkish 8th grade students have signıfıcantly lower level of
correct responses compared to all other 8th grade participants. For this
purpose, total 260 (82 from 1999, 88 from 2007, and 90 from 2011)  items released by International Association
for the Evaluation of Educational Achievement (IEA) were grouped according to
cognitive, content and sub-content domains. Then, mean correct responses of
released items for each participant country were obtained from IEA’s yearly
almanac. Finally, data were analyzed by using Multilevel Measurement Models
and differences in achievement levels between Turkish 8th graders and their
peers from other participating countries were predicted and tested in the
context of item groups. Analysis of data showed that performance of Turkish
students statistically significantly lower than performance of students from
rest of the other participant countries in Number (Content Domain)-Fractions
and Decimals (Topic Area)-Knowing (Cognitive Domain) item group. Detailed
investigation revealed that students generally fail in procedures in fractions
and conversions among fraction, decimal, and percent.

Keywords

Two-level hierarchical linear measurement models, mathematics performance level, TIMSS

TIMSS Matematik Verilerinin Aşamalı Ölçme Modelleri ile İçerik, Bilişsel ve Konu Alanları Bakımından İncelenmesi

Abstract

Bu çalışmanın
amacı, TIMSS 8. sınıf matematik testlerine katılan öğrencilerimizin dünyanın
başka yerlerinde yaşayan akranlarına göre doğru cevaplamada güçlük çektikleri
madde gruplarını ve bu gruplarını oluşturan maddelerin özelliklerini ortaya
çıkarmaktır. Bu amaç dahilinde, ilk olarak, Türkiye’nin 1999, 2007 ve 2011
yıllarında katıldığı TIMSS 8. sınıf matematik testlerinin International
Association for the Evaluation of Educational Achievement (İEA) tarafından
yayınlanan bütün maddeleri (toplam 260 adet), yine IEA tarafından yayınlanan
değerlendirme çerçevesi dokümanları ve belirtke tabloları takip edilerek
içerik, bilişsel alan ve konu alanı itibariyle gruplara ayrılmıştır. Ardından,
IEA’nın bir yıllık olarak hazırladığı almanaklar kullanılarak bu maddelere ait
ortalama doğru cevaplama oranları ve bu oranların TIMSS katılımcıları arası
dağılımları çıkarılmıştır. Son olarak, elde edilen veriler aşamalı doğrusal
ölçme modelleri ile analiz edilerek madde grupları bağlamında Türkiye ile
diğer TIMSS katılımcıları arası başarı düzeyi farklılıkları tahmin edilip test
edilmiştir. Bulgular öğrencilerimizin “Sayılar” içerik alanı altında bulunan
“Kesirler ve Ondalıklı Sayılar” konusu ile ilgili olgular, kavramlar ve
yöntemlere ait bilgi düzeylerinin diğer ülkelerde yaşayan akranlarına göre
oldukça düşük olduğunu göstermekte ve bu farklılık istatistiksel olarak
anlamlı bulunmaktadır.

Keywords

İki aşamalı doğrusal ölçme modelleri, matematik başarı düzeyi, TIMSS

References

• Acar, T. (2011). Maddenin Farklı Fonksiyonlaşmasında Örneklem Büyüklüğü: Genelleştirilmiş Aşamalı Doğrusal Modelleme Uygulaması. Kuram ve Uygulamada Eğitim Bilimleri 11(1), 279-288.
• Bauer D. J. (2003). Estimating Multilevel Linear Models as Structural Equation Models. Journal of Educational and Behavioral Statistics. Vol. 28, No. 2, pp. 135-167.
• Behr, M. J., Harel, G., Post, T., & Lesh, R. (1992). Rational number, ratio, and proportion. In D. A.Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp. 296–333). New York: Macmillan.
• Binici, S. (2007). Random-effects differential item functioning via hierarchical generalized linear model and generalized linear latent mixed model: A comparison of estimation methods. Unpublished doctoral dissertation, Florida State University.
• Birgin, O. & Gürbüz, R. (2009). İlköğretim İkinci Kademe Öğrencilerinin Rasyonel Sayılar Konusundaki İşlemsel ve Kavramsal Bilgi Düzeylerinin İncelenmesi. Uludağ Üniversitesi Eğitim Fakültesi Dergisi, 22(2), 529-550.
• Bulgar, S. (2003). Children’s sense-making of division of fractions. Journal of Mathematical Behavior, 22, 319-334.
• Carpenter, T. (1988). Teaching as problem solving. In R. Charles, & E. Silver (Eds.), The teaching and assessing of mathematical problem solving (pp. 187–202). Reston, VA: National Council of Teachers of Mathematics.
• Chaimongkol, S. (2005). Modeling differential item functioning (dif) using multilevel logistic regression models: A Bayesian Perspective. Unpublished doctoral dissertation. Florida State University.
• Chu KL, Kamata A.(2005). Test equating in the presence of DIF items. Journal of Appl Meas. 6(3):342-54.
• Davis, R. B., & Maher, C. A. (1990). What do we do when we ‘do mathematics’? In: N. Noddings (Ed.), Constructivist views of the teaching and learning of mathematics (Vol. Monograph No. 4). Reston, VA: National Council of Teachers of Mathematics.
• De Corte, E., Greer, B., & Verschaffel, L. (1996). Mathematics teaching and learning. In D. C. Berliner,& R. C. Calfee (Eds.), Handbook of educational psychology (pp. 491–549). New York: Macmillan.
• De Corte, E., Verschaffel, L., & Pauwels, A. (1990). Influence of the semantic structure of word problems on second graders’ eye movements. Journal of Educational Psychology, 82, 359–365.
• Fischbein, E., Deri, M., Nello, M. S., & Marino, M. S. (1985). The role of implicit models in solving verbal problems in multiplication and division. Journal for Research in Mathematics Education, 1985(16), 3–17.
• Greeno, J. G. (1991). Number sense as situated knowing in a conceptual domain. Journal for Research in Mathematics Education, 22(3), 170–218.
• Kamata, A. (2001). Item Analysis by the Hierarchical Generalized Linear Model. Journal of Educational Measurement. Vol. 38, No. 1 (Spring, 2001) , pp. 79-93.
• Kamata, A., Bauer, D.J. & Miyazaki, Y. (2008). Multilevel measurement modeling. In A.A. O'Connell & D.B. McCoach (Eds.) Multilevel Modeling of Educational Data (pp. 345-388). Charlotte, NC: Information Age Publishing.
• Kamata, A. (2001). Item Analysis by the Hierarchical Generalized Linear Model. Journal of Educational Measurement, 38, 79-93.
• Kamata, A., & Binici, S. (2003). Random Effect DIF Analysis via Hierarchical Generalized Linear Modeling. Paper presented at the biannual meeting of Psychometric Society, July, Sardinia, Italy.
• Kamata, A., & Vaughn, B. K. (2004). An Introduction to Differential Item Functioning Analysis. Learning Disabilities: A Contemproary Journal, 2(2), 49-64.
• Kamata, A., Chaimongkol, S., Genc, E., & Bilir, K. (2005). Random-Effect Differential Item Functioning Across Group Unites by the Hierarchical Generalized Linear Model. Paper presented at the annual meeting of the American Educational Research Association, April, Montreal, Canada.
• Kim, W. (2003). Development of a differential item functioning (DIF) procedure using the hierarchical generalized model: a comparison study with logistic regression procedure. Unpublished doctoral dissertation. Pennsylvania State University.
• Lamon, S. J. (1999). Teaching fractions and ratios for understanding. Mahwah, NJ, USA: Lawrence Erlbaum Associates, Inc.
• Lehtinen, E., Merenluoto, K., & Kasanen, E. (1997). Conceptual change in mathematics: from rational to (un)real numbers. European Journal of Psychology of Education, XII(2), 131–145.
• Luppescu, S. (2002). DIF detection in HLM. Paper presented at the annual meeting of the American Educational Research Association, New Orleans.
• Maher, C. A., and Alston, A. (1989). Is meaning connected to symbols? An interview with Ling Chen. The Journal of Mathematical Behavior, 8(3), 241-248.
• Miyazaki, Y.& Kenneth A. F. (2006). A Hierarchical Linear Model with Factor Analysis Structure at Level 2. Journal of Educational and Behavioral Statistics.Vol. 31, No. 2, pp. 125-156.
• R Development Core Team (2012). R: A language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.
• Raudenbush, S.W., Bryk, A.S, & Congdon, R. (2004). HLM 6 for Windows. Hierarchical linear and non-linear modeling. Lincolnwood, IL: Scientific Software International.
• Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. 2nd edition. Sage Publications Inc. Newbury, CA.
• Resnick, L. B., & Ford, W. W. (1981). The psychology of mathematics for instruction. Hillsdale, NJ: Lawrence Erlbaum Associates.
• Resnick, L. B., Nesher, P., Leonard, F., Magone, M., Omanson, S., & Peled, I. (1989). Conceptual bases of arithmetic errors: the case of decimal fractions. Journal of Research in Mathematics Education, 20(1), 8–27.
• Resnick, L. B., & Omanson, S. (1987). Learning to understand arithmetic. In R. Glaser (Ed.), Advances in instructional psychology, Vol. 3. (pp. 41–95). Hillsdale, NJ: Erlbaum.
• Skrondal, A. & Rabe-Hesketh, S. (2004). Generalized Latent Variable Modeling: Multilevel, Longitudinal, and Structural Equation Models. New York: Chapman & Hall/CRC.