Assessment of Mathematics Performance with the Cognitive Diagnostic Model: TIMSS Türkiye and Singapore

Abstract

In this study, it is aimed to examine the weaknesses and strengths aspects of students in terms of knowledge and skills measured at the 8th grade mathematics level. In the study, the student response pattern given to the 8th mathematics tests in the TIMSS 2015 and the Q matrix developed by the field experts were used. This research consists of students who participated in the TIMSS 2015 from Türkiye and Singapore and who are in the eighth year of the formal education process. The research was conducted on the data of 435 Turkey, 436 Singapore students who received the 5th booklet. Data analysis was made with the DINA model, one of the cognitive diagnosis models, and the weak and strong aspects of the students in terms of knowledge and skills within the scope of the measured variables were handled independently of the total score and ranking of the countries. By interpreting the parameter values obtained at the end of the data analysis and the attribute prevalence, the weak and strong points of the students from the two countries in the measured skill and knowledge levels were interpreted. One of the most important findings is that Singapore, which has a high level of success and achieves high scores, has low values in terms of transformation geometry. On the other hand, it was determined that Turkish students had difficulties in items involving reasoning skills and open-ended items.

Keywords

• mathematics achievement
• cognitive skills
• cognitive diagnostic models
• DINA Model
• TIMSS

Matematik Performansının Bilişsel Tanı Modeli ile Değerlendirilmesi: TIMSS Türkiye ve Singapur Örneği

Abstract

Bu çalışmada, öğrencilerin 8.sınıf matematik dersinde bilgi ve beceriler bakımından zayıf ve güçlü yönlerinin incelenmesi amaçlanmaktadır. Araştırmada, TIMSS 2015 uygulamasında 8. sınıf matematik testlerine verilen öğrenci yanıt örüntüsü ile alan uzmanları tarafından geliştirilen Q matris kullanılmıştır. Bu araştırmanın evrenini TIMSS 2015 uygulamasına Türkiye ve Singapur’dan katılan ve formal eğitim sürecinin sekizinci yılında bulunan öğrenciler oluşturmaktadır. Araştırma 5. kitapçığı alan 435 Türkiye ve 436 Singapur öğrenci verisi üzerinden yürütülmüştür. Bilişsel tanı modellerinden DINA model ile veri analizi yapılarak öğrencilerin ölçülen değişkenler kapsamında bilgi ve beceri bakımından zayıf ve güçlü yönleri, ülkelerin toplam puan ve sıralamasından bağımsız bir şekilde ele alınmıştır. Veri analizi sonunda elde edilen parametre değerleri ve gözlenen niteliklerin sıklıkları yorumlanarak iki ülke öğrencilerinin ölçülen beceri ve bilgi düzeylerindeki zayıf ve güçlü noktaları yorumlanmıştır. En önemli bulgulardan biri başarı düzeyi üst sıralarda yer alan ve yüksek puanlar elde eden Singapur’un dönüşüm geometrisi konusunda düşük düzeyde değerler elde etmesidir. Diğer taraftan Türk öğrencilerin akıl yürütme düzeyinde ve açık uçlu maddelerde zorlandıkları belirlenmiştir.

Keywords

• matematik başarısı
• bilişsel beceriler
• bilişsel tanı modelleri
• DINA Model
• TIMSS.

References

1. Başokçu, T. O. (2011). Bağıl ve Mutlak Değerlendirme ile DINA Modele Göre Yapılan Sınıflamaların Geçerliğinin Karşılaştırılması [Yayınlanmamış Doktora Tezi]. Hacettepe Üniversitesi, Sosyal Bilimler Enstitüsü, Ankara.
2. Bayirli, A. (2020). Singapur Eğitim Sistemi ile Türk Eğitim Sisteminin Karşılaştırılması ve Türkiye İçin Çıkarımlar. USBAD Uluslararası Sosyal Bilimler Akademi Dergisi 2(4), 1104-1132.
3. Choi, K.M., Lee, Y. -S., and Park Y. S. (2014). What CDM can tell about what students have learned: an analysis of timss eighth grade mathematics. Eurasia Journal of Mathematics, Science & Technology Education, 11(6), 1563-1577.
4. de la Torre, J. (2008a). The generalized DINA model framework [Unpublished manuscript]. State University of New Jersey.
5. de la Torre, J. (2009a). A cognitive diagnosis model for cognitively-based multiple-choice options. Applied Psychological Measurement, 33, 163– 183.
6. de la Torre, J. (2009b). DINA Model and Parameter Estimation: A Didactic. Journal of Educational and Behavioral Statistics, 34 (1), 115–130.
7. Dogan, E., & Tatsuoka, K. (2008). An international comparison using a diagnostic testing model: Turkish students’ profile of mathematical skills on TIMSS-R. Educational Studies in Mathematics, 68(3), 263-272. Doornik, J. A. (2003). Object-oriented matrix programming using Ox (version 3.1) [Computer software]. Timberlake Consultants Press, London.
8. Henson, R. A., Roussos, L., and Templin, J. L. (2004). Cognitive diagnostic “fit” indices. ETS, NJ. Henson, R. A., Templin, J. L., and Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191 210.

9. IEA, (2017). TIMSS 2015 international database. http://timssandpirls.bc.edu/timss2015/international-database/
10. Jensen, B. (2012). Catching up: learning from the best school systems in east asia [Summary Report]. Grattan Institute.
11. Junker, B. W., and Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3), 258-272.
12. Karasar, N. (2008). Bilimsel araştırma yöntemi (18.Baskı). Nobel Yayın Dağıtım.
13. Lee, Y. -S., de la Torre J., and Park Y. S. (2012). Relationships between cognitive diagnosis, CTT, and IRT indices an empirical investigation. Asia Pacific Educ., 13, 333–345.
14. Lee, Y. -S., Johnson, M., Park, Y. J., Sachdeva, R., Zhang, J., and Waldman, M. (2013). An multidimensional scaling approach for investigating students' cognitive weakness and strength on the TIMSS 2007 Mathematics Assessment Annual Meeting of the American Educational Research Association in San Francisco. CA.
15. Martin, M. O., Mullis, I.V.S., and Hooper, M. (2016). Methods and procedures in TIMSS 2015. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College. http://timssandpirls.bc.edu/publications/timss/2015-methods.html
16. MEB, (2018). Talim terbiye kurulu başkanlığı 1-8. sınıflar matematik öğretim programları. MEB Yayınları.
17. Mislevy, R. J., Steinberg, L. S., and Almond, R. G. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1, 3-62.
18. Mullis, I.V.S., and Martin, M. O. (2013). TIMSS 2015 Assessment Frameworks. TIMSS and PIRLS In¬ternational Study Center. Chestnut Hill, MA: Lynch School of Education, Boston College.
19. Mullis, I. V.S., and Martin, M. O. (2017). TIMSS 2019 Assessment Frameworks. TIMSS and PIRLS [International Study Center]. Chestnut Hill, MA: Lynch School of Education, Boston College.
20. National Council of Teachers of Mathematics (NCTM, 2000). Principles and standards for school mathematics. VA: NCTM.
21. Rupp, A. A., Templin, J. L., and Henson, R. A. (2010). Diagnostic assessment: Theory, methods, and applications. Guilford Press.
22. Snow, R. E., and Lohman, D. F. (1993a). Cognitive psychology, new test design, and new test theory: an introduction. N. Fredriksen, R. J. Mislevy, & I. Bejar (Eds.), In Test Theory for a new generation of tests (1-17). NJ: Erlbaum.
23. Tatsuoka, K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345–354.
24. Tatsuoka, K. K. (1995). Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. P. D. Nichols, S. F. Chipman, & R. L. Brennan (Eds.), In Cognitively diagnostic assessment (327–359). Lawrence Erlbaum Associates.
25. Tatsuoka, K. K., Corter, J. E., and Tatsuoka, C. (2004). Patterns of diagnosed mathematical content and process skills in TIMSS-R across a sample of 20 countries. American Educational Research Journal, 41(4), 901-926.
26. Wellington, J. (2006). Educational research: contemporary issues and practical approaches. Continuum. Wenmin, Z. (2006). Detecting differential item functioning using the DINA model. [Yayımlanmamış doktora tezi]. The University of North Carolina at Greensboro.