The Cognitive Diagnostic Models for Estimating Students’ Ability and Their Applications

Abstract

Cognitive Diagnostic Models (CDM) are based on latent class analysis. Such analysis is a statistical method that determines subclasses by using multiple variable categorical data and making use of mutually related cases. CDM, on the other hand, is developed to calculate the structure of a certain knowledge or the development of a certain capacity by taking into account both strengths and weaknesses of the respondent in cognitive terms (Leighton ve Gierl, 2007). According to these models, replies of the students to the options of the test compose a vector of latent classes they belong. Thus, these models, through replies to the options, aim to determine the latent classes of students. To this end, various models are introduced. The aim of this paper is to study basic qualities, structures and applications of CDM

Keywords

• Cognitive Diagnostic Models, Multidimensional IRT, Latent Class Analyze, Q Matrix

ÖĞRENCİ YETENEĞİNİN KESTİRİMİNDE BİLİŞSEL TANI MODELLERİ VE UYGULAMALARI

Abstract

Bilişsel Tanı Modelleri (BTM), temelinde örtük sınıf analizi olan yaklaşımlardır. Örtük sınıf analizi, çok değişkenli kategorik bir veri kullanarak ve birbiriyle ilişkili durumlardan yararlanarak alt gruplar belirleyen istatistiksel bir yöntemdir. BTM ise cevaplayıcıda belirli bir bilginin yapısını ya da bir becerinin gelişimini, cevaplayıcının bilişsel düzeydeki güçlü ve zayıf yönlerini dikkate alarak hesaplamak amacıyla geliştirilmiştir (Leighton ve Gierl, 2007). Bu modellere göre öğrencilerin testteki maddelere verdikleri cevaplar, onların ait oldukları örtük sınıflarının bir vektörüdür. Bu nedenle modeller, maddelere verilen cevaplardan yola çıkarak öğrencilerin örtük sınıflarını belirlemeyi amaçlar. Bu amaç doğrultusunda geliştirilen farklı modeller vardır. Bu çalışmada BTM olarak adlandırılan bu modellerin temel özellikleri, yapıları ve kullanılan modellerin uygulamaları hakkında bilgiler verilecektir

Keywords

• Bilişsel Tanı Modelleri, Çok Boyutlu Madde Tepki Kuramı, Örtük Sınıf Analizi, Q Matrisi

References

1. Adams, R. J., Wilson, M. (1996). Formulating the Rasch model as a mixed coefficients multinomial logit. In: Engelhard, G., Wilson, M. (Ed), Objective Measurement III: Theory in Practice. Ablex, Norwood, NJ.
2. Adams, R. J., Wilson, M. & Wang, W.C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement 21, 1–23.
3. Almond, R. G., Steinberg, L. S. & Mislevy, R. J. (2003). A framework for reusing assessment components. In: Yanai, H., Okada, A., Shigemasu, K., Kano, Y., Meulman, J.J. (Ed.), New Developments in Psychometrics (s. 281–288). Tokyo: Springer.
4. Cheng Y.& Chang H. (2007). The modified maximum global discrimination index method for cognitive diagnostic computerized adaptive testing. Presented at the CAT and Cognitive Structure Paper Session, June 7.
5. de la Torre, J. (2009). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34, 115–130.
6. de la Torre, J. .& Douglas, J. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69 (3), 333-353.
7. DiBello, L. V., Stout, W. F., & Roussos, L. A. (1995). Unified cognitive/psychometric diagnostic assessment likelihood-based classification techniques. In P.D. Nichols, S. F. Chipman, and R. L. Brennan (Ed.), Cognitively Diagnostic Assessment (s. 327-361). Hillsdale, NJ: Lawrence Erlbaum Associates
8. Dibello, L. V. RoussosL. A. & Stout, W. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. Rao,C. Sinharay, S. (Ed.) Handbook of Statistics, Psychometrics, 26, Amsterdam: North- Holland.

9. Embretson, S. E. (1985). Multicomponent latent trait models for test design. In: Embretson, S.E. (Ed.), Test Design: Developments in Psychology and Psychometrics(s. 195–218). New York: Academic Press.
10. Embretson, S. E. (1997). Multicomponent response models. In: van der Linden,W.J., Hambleton, R.L. (Ed.), Handbook of Modern Item Response Theory (s.305–321). New York: Springer.
11. Fischer, G. H. ( 1976). 'Some probabilistic models for measuring change', in Advances in Psychological and Educational Measurement (s.107-112). D. De Gruijter & L. van der Kamp (Ed), Bern: Huber,.
12. Fischer, G. H. (1973). The linear logistic model as an instrument in educational research. Acta Psychologica 37, 359–374.
13. Fischer, G. H. (1983). Logistic latent trait models with linear constraints. Psychometrika, 48, .3–26.
14. Gierl, M. J. (2007). Making diagnostic inferences about cognitive attributes using Rule-Space model and attribute hierarchy method. Journal of Educational Measurement, 44, 325–340
15. Gitomer, D. H. & Yamamoto, K. (1991). Performance modeling that integrates latent trait and class theory. Journal of Educational Measurement 28, 173–189.
16. Haertel, E. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 333-352.
17. Haertel, E.H. (1984). An application of latent class models to assessment data. Applied Psychological Measurement, 8, 333–346.
18. Haertel, E. H. (1990). Continuous and discrete latent structure models of item response data. Psychometrika 55, 477–494.
19. Hartz, S. (2002). Skills diagnosis: Theory and practice. user manual for Arpeggio software. Princeton, NJ: ETS.
20. Hartz, S.M., Roussos, L.A. (2005). The Fusion Model for skills diagnosis: Blending theory with practice. ETS Research Report, Educational Testing Service, Princeton, NJ.
21. Henson, R. A., Roussos, L., Templin, J. L. (2004). Cognitive diagnostic “fit” indices. Yayımlanmamış ETS Proje Raporu, Princeton, NJ.
22. Leighton, J. P.& Gierl M. J. (2007). Why Cognitive Diagnostic Assessment? Leighton, J. P. Gierl M. J. (Ed). Cognitive Diagnostic Assessment for Education. New York: Cambridge University Press.
23. Li, F. (2008). A modified higher-order DINA model for detecting differential item functioning and differential attribute functioning. Yayımlanmamış Doktora Tezi, The University of Georgia,
24. Louis A. Roussos, L. V. D., William Stout,, & Sarah M. Hartz, R. A. H., Jonathan L. Templin. (2007). The
25. Fusion model skills diagnosis system. In J. Gierl (Ed.), Cognitive Diagnostic Assessment for Education Theory and Applications ( 275-318). New York: Cambridge University Press
26. Macready, G. B., Dayton, C. M. (1977). The use of probabilistic models in the assessment of mastery. Journal of Educational Statistics 2, 99–120.
27. Maris, E. (1999). Estimating multiple classification latent class models. Psychometrika, 64, 187-212.
28. Montero, D., Molfils, L., Wang, J., Yen, W., Julian, M. & Moody, M. (2003). Investigation of the application of cognitive diagnostic testing to an end-of- course- high school examination, Presented at the Annual Meeting of the National Council on Measurement in Education, Chicago, IL, April 24.
29. Nichols, P. D., Chipman, S. F., & Brennan, R. L. (Ed.). (1995). Cognitively diagnostic assessment. Hillsdale, New Jersey: Lawrence Erlbaum.
30. Reckase, M. D., McKinley, R.L. (1991). The discriminating power of items that measure more than one dimension. Applied Psychological Measurement, 15, 361–373.
31. Rupp, A. A. (2007). The answer is in the question: A guide for describing and investigating the conceptual foundations and statistical properties of cognitive psychometric models. International Journal of Testing. 7 (2), 95- 125
32. Rupp, A. A., & Mislevy, R. J. (2007). Cognitive psychology as it applies to diagnostic assessment. J. Leighton (Ed.), Cognitive diagnostic assessment in education: Theory and practice. Cambridge, UK: Cambridge University Press.
33. Spada, H. (1977). 'Logistic models of learning and thought', in structural models of thinking and learning. H. Spada & W. F. Kempf (eds), Vienna: Huber, , 227-62
34. Sympson, J. B. (1977). A model for testing with multidimensional items. In:Weiss, D.J. (Ed.), Proceedings of the 1977 Computerized Adaptive Testing Conference. University of Minnesota, Department of Psychology, Psychometric Methods Program, Minneapolis, 82–88.
35. Tatsuoka, K. K. & Tatsuoka, M. M. (1984). Bug distribution and pattern classification. Psychometrika 52(2), 193-206.
36. Tatsuoka, K. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345–354.
37. Tatsuoka, K. K. (1984). Caution indices based on item response theory. Psychometrika, 49(1), 95-110.
38. Tatsuoka, K. K. (1990). Toward an integration of item-response theory and cognitive error diagnoses. In: Frederiksen, N., Glaser, R.L., Lesgold, A.M., Shafto, M.G. (Ed.), Diagnostic Monitoring of Skill and Knowledge Acquisition. Erlbaum, Hillsdale, NJ.
39. Tatsuoka, K.K. (1995). Cognitive assessment, An introduction to the Rule Space Method, Routledge, New York: Taylor & Francis Group.
40. Tatsuoka, K. K. (1995). Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. In P.D. Nichols, S. F. Chipman, and R. L. Brennan (Ed.), Cognitively Diagnostic Assessment (p. 327-361). Hillsdale, NJ: Lawrence Erlbaum Associates.
41. Templin, J. L. & Henson, R.A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods. 11 (3), 287-305
42. von Davier, M. (2005). A general diagnostic model applied to language testing data. ETS Araştırma Raporu: RR-05-16, Educational Testing Service, Princeton, NJ.
43. von Davier, M., DiBello, L., Yamamoto, K. (2006). A shortlist of models for cognitive diagnosis. Klieme, E., Leutner, D. (Eds.), Assessment of Competencies in Educational Contexts. Bern: Hogrefe & Huber Publishers,.
44. von Davier, M., Yamamoto, K. (2004). A class of models for cognitive diagnosis. ETS Spearmann Konferansında bildiri olarak sunulmuştur, The Inn at Penn, Philadelphia, PA, Invitational Conference.
45. Whitely, S. E. (1980). Multicomponent latent trait models for ability tests. Psychometrika, 45, 479–494.