A Diagnostic Comparison of Turkish and Korean Students’ Mathematics Performances on the TIMSS 2011 Assessment
Abstract
Although some researchers question the usefulness of international large-scale assessments (e.g., TIMSS), participating countries have continued to use the results from these large-scale assessments to improve their curricula and teaching methods. Despite the common reporting practice—single-score—in these large scale assessments gives useful insights about students’ overall performances, they still lack diagnostic information. Cognitive diagnosis models (CDMs) were developed to provide more feedback on students’ cognitive strengths and weaknesses. This study retrofitted the TIMSS 2011 eighth grade mathematics assessment by applying a specific CDM called the DINA (the deterministic, inputs, noisy, “and” gate) model to data from South Korea and Turkey. Results of the DINA model were used to make a detailed comparison between students of these two countries.
Keywords
References
- Choi, M. K., Lee, Y-S., & Park, Y. S. (in submission). What CDM can tell about what students have learned: An analysis of TIMSS eighth grade mathematics.
- Common Core State Standards Initiative (2010). The common core state standards for mathematics. Washington, D.C.: Author.
- Common Core State Standards Initiative (2014). Myths vs. facts. Retrieved from http://www.corestandards.org/about-the-standards/myths-vs-facts/
- de la Torre, J. (2008). An empirically-based method of Q-matrix validation for the DINA model: Development and applications. Journal of Educational Measurement, 45, 343–362.
- de la Torre, J., & Lee, Y. S. (2008, March). Relationships between cognitive diagnosis, CTT and IRT indices: An empirical investigation. Paper presented at the Annual Meeting of the National Council on Measurement in Education, New York.
- de la Torre, J. (2009). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34, 115–130.
- de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179–199.
- DiBello, L., Roussos, L. A., & Stout, W. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In C. V. Rao & S. Sinharay (Eds.), Handbook of Statistics (Vol. 26, Psychometrics) (pp. 979–1027). Amsterdam: Elsevier.
- Dogan, E., & Tatsuoka, K.K. (2008). An international comparison using a diagnostic testing model: Turkish students’ profile of mathematical skills on TIMSS-R. Educational Studies in Mathematics, 68(3), 263–272.
- Doornik, J. A. (2003). Object-oriented matrix programming using Ox (version 3.1) [Computer software]. London: Timberlake Consultants Press.
- Fu, J., & Li, Y. (2007, April). An integrated review of cognitively diagnostic psychometric models. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago, IL.
- Haertel, E. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 301–323.
- Hartz, S. M. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign, Urbana-Champaign, IL.
- Henson, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.
- Holliday, W. G., & Holliday, B. W. (2003). Why using international comparative math and science achievement data from TIMSS is not helpful. The Education Forum, 67, 250– 257.
- Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.
- Lee, Y.-S., Park, Y.S., & Taylan, D. (2011). A cognitive diagnostic modeling of attribute mastery in Massachusetts, Minnesota, and the U.S. national sample using the TIMSS 2007. International Journal of Testing.
- Leighton, J., & Gierl, M. (2007). Why cognitive diagnostic assessment? In J. P. Leighton &
- M. J. Gierl (Eds.), Cognitive diagnostic assessment for education (pp. 3–18). Cambridge:
- Cambridge University Press.
- Mullis, I.V.S., Martin, M.O., Foy, P., & Arora, A. (2012): TIMSS 2011 International Results in Mathematics. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.
- Nichols, P. D., Chipman, S. F., & Brennan, R. L. (Eds.). (2012). Cognitively diagnostic assessment. Hillsdale, NJ: Erlbaum.
- Roussos, L., DiBello, L. V., Stout, W., Hartz, S., Henson, R. A., & Templin, J. H. (2007). The fusion model skills diagnosis system. In J. P. Leighton, & Gierl, M. J. (Ed.), Cognitively diagnostic assessment for education: Theory and practice. (pp. 275–318). Thousand Oaks, CA: SAGE.
- Rupp, A. A., & Templin, J. L. (2008). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art. Measurement, 6(4), 219–262.
- Rupp, A. A., Templin, J. L., & Henson, R. A. (2010). Diagnostic assessment: Theory, methods, and applications. New York: Guilford Press.
- Tatsuoka, K. (1985). A probabilistic model for diagnosing misconceptions in the pattern classification approach. Journal of Educational Statistics, 12, 55-73.
- Templin, J., & Henson, R. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287–305.
- Toker, T., & Green, K. (2012, April). An application of cognitive diagnostic assessment on TIMMS-2007 8th Grade Mathematics items. Paper presented at the annual meeting of the American Educational Research Association, Vancouver, British Columbia, Canada.
- von Davier, M. (2005). A general diagnostic model applied to language testing data. ETS Research Report RR-05-16
- Wang, J. (2001). TIMSS primary and middle school data: Some technical concerns. Educational
- Researcher, 30, 17–21.
There are 31 citations in total.