2015-07-01

A Diagnostic Comparison of Turkish and Korean Students’ Mathematics Performances on the TIMSS 2011 Assessment

Sedat SEN , Muhammet ARICAN


Although some researchers question the usefulness of international large-scale assessments (e.g., TIMSS), participating countries have continued to use the results from these large-scale assessments to improve their curricula and teaching methods. Despite the common reporting practice—single-score—in these large scale assessments gives useful insights about students’ overall performances, they still lack diagnostic information. Cognitive diagnosis models (CDMs) were developed to provide more feedback on students’ cognitive strengths and weaknesses. This study retrofitted the TIMSS 2011 eighth grade mathematics assessment by applying a specific CDM called the DINA (the deterministic, inputs, noisy, “and” gate) model to data from South Korea and Turkey. Results of the DINA model were used to make a detailed comparison between students of these two countries.

  • Choi, M. K., Lee, Y-S., & Park, Y. S. (in submission). What CDM can tell about what students have learned: An analysis of TIMSS eighth grade mathematics.
  • Common Core State Standards Initiative (2010). The common core state standards for mathematics. Washington, D.C.: Author.
  • Common Core State Standards Initiative (2014). Myths vs. facts. Retrieved from http://www.corestandards.org/about-the-standards/myths-vs-facts/
  • de la Torre, J. (2008). An empirically-based method of Q-matrix validation for the DINA model: Development and applications. Journal of Educational Measurement, 45, 343–362.
  • de la Torre, J., & Lee, Y. S. (2008, March). Relationships between cognitive diagnosis, CTT and IRT indices: An empirical investigation. Paper presented at the Annual Meeting of the National Council on Measurement in Education, New York.
  • de la Torre, J. (2009). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34, 115–130.
  • de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179–199.
  • DiBello, L., Roussos, L. A., & Stout, W. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In C. V. Rao & S. Sinharay (Eds.), Handbook of Statistics (Vol. 26, Psychometrics) (pp. 979–1027). Amsterdam: Elsevier.
  • Dogan, E., & Tatsuoka, K.K. (2008). An international comparison using a diagnostic testing model: Turkish students’ profile of mathematical skills on TIMSS-R. Educational Studies in Mathematics, 68(3), 263–272.
  • Doornik, J. A. (2003). Object-oriented matrix programming using Ox (version 3.1) [Computer software]. London: Timberlake Consultants Press.
  • Fu, J., & Li, Y. (2007, April). An integrated review of cognitively diagnostic psychometric models. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago, IL.
  • Haertel, E. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 301–323.
  • Hartz, S. M. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign, Urbana-Champaign, IL.
  • Henson, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.
  • Holliday, W. G., & Holliday, B. W. (2003). Why using international comparative math and science achievement data from TIMSS is not helpful. The Education Forum, 67, 250– 257.
  • Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.
  • Lee, Y.-S., Park, Y.S., & Taylan, D. (2011). A cognitive diagnostic modeling of attribute mastery in Massachusetts, Minnesota, and the U.S. national sample using the TIMSS 2007. International Journal of Testing.
  • Leighton, J., & Gierl, M. (2007). Why cognitive diagnostic assessment? In J. P. Leighton &
  • M. J. Gierl (Eds.), Cognitive diagnostic assessment for education (pp. 3–18). Cambridge:
  • Cambridge University Press.
  • Mullis, I.V.S., Martin, M.O., Foy, P., & Arora, A. (2012): TIMSS 2011 International Results in Mathematics. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.
  • Nichols, P. D., Chipman, S. F., & Brennan, R. L. (Eds.). (2012). Cognitively diagnostic assessment. Hillsdale, NJ: Erlbaum.
  • Roussos, L., DiBello, L. V., Stout, W., Hartz, S., Henson, R. A., & Templin, J. H. (2007). The fusion model skills diagnosis system. In J. P. Leighton, & Gierl, M. J. (Ed.), Cognitively diagnostic assessment for education: Theory and practice. (pp. 275–318). Thousand Oaks, CA: SAGE.
  • Rupp, A. A., & Templin, J. L. (2008). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art. Measurement, 6(4), 219–262.
  • Rupp, A. A., Templin, J. L., & Henson, R. A. (2010). Diagnostic assessment: Theory, methods, and applications. New York: Guilford Press.
  • Tatsuoka, K. (1985). A probabilistic model for diagnosing misconceptions in the pattern classification approach. Journal of Educational Statistics, 12, 55-73.
  • Templin, J., & Henson, R. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287–305.
  • Toker, T., & Green, K. (2012, April). An application of cognitive diagnostic assessment on TIMMS-2007 8th Grade Mathematics items. Paper presented at the annual meeting of the American Educational Research Association, Vancouver, British Columbia, Canada.
  • von Davier, M. (2005). A general diagnostic model applied to language testing data. ETS Research Report RR-05-16
  • Wang, J. (2001). TIMSS primary and middle school data: Some technical concerns. Educational
  • Researcher, 30, 17–21.
Primary Language en
Journal Section Articles
Authors

Author: Sedat SEN

Author: Muhammet ARICAN

Dates

Publication Date : July 1, 2015

Bibtex @ { epod77298, journal = {Journal of Measurement and Evaluation in Education and Psychology}, issn = {1309-6575}, eissn = {1309-6575}, address = {}, publisher = {Eğitimde ve Psikolojide Ölçme ve Değerlendirme Derneği}, year = {2015}, pages = { - }, doi = {10.21031/epod.65266}, title = {A Diagnostic Comparison of Turkish and Korean Students’ Mathematics Performances on the TIMSS 2011 Assessment}, key = {cite}, author = {SEN, Sedat and ARICAN, Muhammet} }
APA SEN, S , ARICAN, M . (2015). A Diagnostic Comparison of Turkish and Korean Students’ Mathematics Performances on the TIMSS 2011 Assessment. Journal of Measurement and Evaluation in Education and Psychology , , . DOI: 10.21031/epod.65266
MLA SEN, S , ARICAN, M . "A Diagnostic Comparison of Turkish and Korean Students’ Mathematics Performances on the TIMSS 2011 Assessment". Journal of Measurement and Evaluation in Education and Psychology (2015 ): <https://dergipark.org.tr/en/pub/epod/article/77298>
Chicago SEN, S , ARICAN, M . "A Diagnostic Comparison of Turkish and Korean Students’ Mathematics Performances on the TIMSS 2011 Assessment". Journal of Measurement and Evaluation in Education and Psychology (2015 ):
RIS TY - JOUR T1 - A Diagnostic Comparison of Turkish and Korean Students’ Mathematics Performances on the TIMSS 2011 Assessment AU - Sedat SEN , Muhammet ARICAN Y1 - 2015 PY - 2015 N1 - doi: 10.21031/epod.65266 DO - 10.21031/epod.65266 T2 - Journal of Measurement and Evaluation in Education and Psychology JF - Journal JO - JOR SP - EP - SN - 1309-6575-1309-6575 M3 - doi: 10.21031/epod.65266 UR - https://doi.org/10.21031/epod.65266 Y2 - 2019 ER -
EndNote %0 Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi A Diagnostic Comparison of Turkish and Korean Students’ Mathematics Performances on the TIMSS 2011 Assessment %A Sedat SEN , Muhammet ARICAN %T A Diagnostic Comparison of Turkish and Korean Students’ Mathematics Performances on the TIMSS 2011 Assessment %D 2015 %J Journal of Measurement and Evaluation in Education and Psychology %P 1309-6575-1309-6575 %R doi: 10.21031/epod.65266 %U 10.21031/epod.65266
ISNAD SEN, Sedat , ARICAN, Muhammet . "A Diagnostic Comparison of Turkish and Korean Students’ Mathematics Performances on the TIMSS 2011 Assessment". Journal of Measurement and Evaluation in Education and Psychology (July 2015): - . https://doi.org/10.21031/epod.65266
AMA SEN S , ARICAN M . A Diagnostic Comparison of Turkish and Korean Students’ Mathematics Performances on the TIMSS 2011 Assessment. Journal of Measurement and Evaluation in Education and Psychology. 2015; -.
Vancouver SEN S , ARICAN M . A Diagnostic Comparison of Turkish and Korean Students’ Mathematics Performances on the TIMSS 2011 Assessment. Journal of Measurement and Evaluation in Education and Psychology. 2015; -.