Validation of cognitive models for subtraction of time involving years and centuries

Abstract

Years and Centuries are the measurement units used to quantify a longer time duration, while subtraction is the operation required to determine the duration based on two given time points. However, subtraction of time is a difficult skill to be mastered by many elementary students. To identify the root cause of the student's failure in performing subtraction involving the unit of time, we developed and validated the three cognitive models related to this skill by conducting a descriptive study which involved 119 Grade Five students from three Malaysian elementary schools. The cognitive diagnostic assessment developed based on the three cognitive models was used to elicit the participants' responses. Then, Attribute Hierarchy Method and Classical Test Theory were employed to analyse the data. The findings indicated that the hierarchical structures of all cognitive models are supported by the student's responses. The three student-based cognitive models were also highly consistent with the corresponding expert-based cognitive models. The cognitive models developed could guide diagnostic assessment development and diagnostic inference making.

Keywords

• Cognitive models
• Cognitive diagnostic assessment
• Validation
• Attribute hierarchy method
• Learning of time

References

1. Akbay, L., Terzi, R., Kaplan, M., & Karaaslan, K.G. (2018). Expert-based attribute identification and validation: An application of cognitively diagnostic assessment. Journal on Mathematics Education, 9, 103-120.
2. Alves, C.B. (2012). Making diagnostic inferences about student performance on the Alberta education diagnostic mathematics project: An application of the Attribute Hierarchy Method. (Publication No. 919011661) [Doctoral Thesis, University of Alberta, Ann Arbor, Canada]. ProQuest Dissertations and Theses database.
3. Australian Curriculum Assessment and Reporting Authority [ACARA]. (2017). Numeracy learning progression and history. https://www.australiancurriculum.edu.au/media/3666/numeracy-history.pdf
4. Brace, N., Doran, C., Pembery, J., Fitzpatrick, E., & Herman, R. (2019). Assessing time knowledge in children aged 10 to 11 years. International Journal of Assessment Tools in Education, 6(4), 580-591.
5. Briggs, D.C., & Kizil, R.C. (2017). Challenges to the use of artificial neural networks for diagnostic classifications with student test data. International Journal of Testing, 17(4), 302-321.
6. Carpenter, T.P., Fennema, E., Franke, M.L., Levi, L., & Empson, S.B. (1999). Children’s mathematics: Cognitively guided instruction. Heinemann.
7. Chan, Y.L. (2017). Super skills modul aktiviti integrasi: Mathematics Year 5 KSSR. Sasbadi.
8. Chan, Y.L., Maun, R., & Krishnan, G. (2017). Dual language programme mathematics Year 5 textbook. Dewan Bahasa dan Pustaka.

9. Chen, F., Yan, Y., & Xin, T. (2017). Developing a learning progression for number sense based on the rule space model in China. Educational Psychology, 37(2), 128-144.
10. Chin, H., Chew, C.M., & Lim, H.L. (2021). Development and validation of online cognitive diagnostic assessment with ordered multiple-choice items for ‘Multiplication of Time’. Journal of Computers in Education, 8(2), 289-316.
11. Chin, H., Chew, C.M., & Lim, H.L. (2021b). Development and validation of online cognitive diagnostic assessment with ordered multiple-choice items for ‘Multiplication of Time’. Journal of Computers in Education, 8(2), 289-316.
12. Chin, H., Chew, C.M., Lim, H.L., & Thien, L.M. (2022). Development and validation of a cognitive diagnostic assessment with ordered multiple-choice items for addition of time. International Journal of Science and Mathematics Education, 20(4), 817-837.
13. Clements, D.H., Sarama, J., Baroody, A.J., & Joswick, C. (2020). Efficacy of a learning trajectory approach compared to a teach-to-target approach for addition and subtraction. ZDM Mathematics Education, 52, 637–648.
14. Cui, Y., & Leighton, J.P. (2009). The Hierarchy Consistency Index: Evaluating person fit for cognitive diagnostic assessment. Journal of Educational Measurement, 46(4), 429-449.
15. Cui, Y., Gierl, M., & Guo, Q. (2016). Statistical classification for cognitive diagnostic assessment: An artificial neural network approach. Educational Psychology, 36(6), 1065-1082.
16. Earnest, D. (2015). When "half an hour" is not "thirty minutes": Elementary students solving elapsed time problem. In T.G. Bartell, K.N. Bieda, R.T. Putnam, K. Bradfield, & H. Dominguez (Eds.), Proceedings of the 37th annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (pp. 285-291). Michigan State University.
17. Earnest, D. (2021). About time: Syntactically-guided reasoning with analog and digital clocks. Mathematical Thinking and Learning. [Advance Online Publication].
18. Fuson, K.C. (1990). Conceptual structures for multiunit numbers: Implications for learning and teaching multidigit addition, subtraction, and place value. Cognition and Instruction, 7(4), 343-403.
19. Gierl, M.J., Alves, C., & Taylor-Majeau, R. (2010). Using the Attribute Hierarchy Method to make diagnostic inferences about examinees’ knowledge and skills in mathematics: An operational implementation of cognitive diagnostic assessment. International Journal of Testing, 10(4), 318-341.
20. Gierl, M.J., Leighton, J.P., & Hunka, S.M. (2000). An NCME instructional module on exploring the logic of Tatsuoka's Rule‐Space Model for test development and analysis. Educational Measurement: Issues and Practice, 19(3), 34-44.
21. Gierl, M.J., Leighton, J.P., Wang, C., Zhou, J., Gokiert, R., & Tan, A. (2009a). Validating cognitive models of task performance in algebra on the SAT (College Board Research 2009-3). The College Board.
22. Gierl, M.J., Roberts, M.P.R., Alves, C., & Gotzmann, A. (April, 2009b). Using judgments from content specialists to develop cognitive models for diagnostic assessments. Paper presented at the Annual Meeting of National Council on Measurement in Education, San Diego, CA.
23. Gierl, M.J., Wang, C., & Zhou, J. (2008). Using the Attribute Hierarchy Method to make diagnostic inferences about examinees' cognitive skills in algebra on the SAT. Journal of Technology, Learning, and Assessment, 6(6), 1-49.
24. Gorin, J.S. (2006). Test design with cognition in mind. Educational Measurement: Issues and Practice, 25(4), 21-35.
25. Graf, E.A., Peters, S., Fife, J.H., Van Rijn, P.W., Arieli‐Attali, M., & Marquez, E. (2019). A Preliminary Validity Evaluation of a Learning Progression for the Concept of Function (Report No: ETS RR–19-21). Wiley.
26. Hadenfeldt, J.C., Neumann, K., Bernholt, S., Liu, X., & Parchmann, I. (2016). Students’ progression in understanding the matter concept. Journal of Research in Science Teaching, 53(5), 683-708.
27. Harris, S. (2008). It's about time: Difficulties in developing time concepts. Australian Primary Mathematics Classroom, 13(1), 28-31.
28. Iuculano, T., Padmanabhan, A., & Menon, V. (2018). Systems neuroscience of mathematical cognition and learning: Basic organization and neural sources of heterogeneity in typical and atypical development. In A. Henik & W. Fias (Eds.), Heterogeneity of function in numerical cognition (pp. 287-336). Academic Press.
29. Jin, H., Shin, H.J., Hokayem, H., Qureshi, F., & Jenkins, T. (2019). Secondary students’ understanding of ecosystems: A learning progression approach. International Journal of Science and Mathematics Education, 17(2), 217-235.
30. Kamii, C., & Russell, K.A. (2012). Elapsed time: Why is it so difficult to teach? Journal for Research in Mathematics Education, 43(3), 296-315.
31. Kane, M.T., & Bejar, I. I. (2014). Cognitive frameworks for assessment, teaching, and learning: A validity perspective. Psicología Educativa, 20(2), 117-123.
32. Keehner, M., Gorin, J.S., Feng, G., & Katz, I.R. (2017). Developing and validating cognitive models in assessment. In A. Rupp & J.P. Leighton (Eds.), The handbook of cognition and assessment: Frameworks, methodologies, and applications (1st ed., pp. 75-101). Wiley Blackwell.
33. Ketterlin-Geller, L.R., & Yovanoff, P. (2009). Diagnostic assessments in mathematics to support instructional decision making. Practical Assessment, Research & Evaluation, 14(16), 1-11.
34. Lambert, K., Wortha, S.M., & Moeller, K. (2020). Time reading in middle and secondary school students: The influence of basic-numerical abilities. The Journal of Genetic Psychology, 181(4), 255-277.
35. Langenfeld, T., Thomas, J., Zhu, R., & Morris, C.A. (2020). Integrating Multiple Sources of Validity Evidence for an Assessment‐Based Cognitive Model. Journal of Educational Measurement, 57(2), 159-184.
36. Leighton, J.P., & Gierl, M.J. (2007). Defining and evaluating models of cognition used in educational measurement to make inferences about examinees' thinking processes. Educational Measurement: Issues and Practice, 26(2), 3-16.
37. Leighton, J.P., Cui, Y., & Cor, M.K. (2009). Testing expert-based and student-based cognitive models: An application of the Attribute Hierarchy Method and Hierarchy Consistency Index. Applied Measurement in Education, 22(3), 229-254.
38. Leighton, J.P., Gierl, M.J., & Hunka, S.M. (2004). The Attribute Hierarchy Method for cognitive assessment: A variation on Tatsuoka's Rule‐Space Approach. Journal of Educational Measurement, 41(3), 205-237.
39. Levin, I. (1989). Principles underlying time measurement: The development of children's constraints on counting time. In I. Levin and D. Zakay (Eds.), Advances in psychology (Vol. 59, pp. 145-183). Elsevier.
40. Li, H., & Suen, H.K. (2013). Constructing and validating a Q-matrix for cognitive diagnostic analyses of a reading test. Educational Assessment, 18(1), 1-25.
41. Linacre, J. (1994). Sample size and item calibration stability. Rasch Measurement Transactions, 7(4), 328.
42. Morrison, K.M., & Embretson, S.E. (2014). Using cognitive complexity to measure the psychometric properties of mathematics assessment items. Multivariate Behavioral Research, 49(3), 292-293.
43. Multon, K.D., & Coleman, J.S.M. (2010). Coefficient alpha. In N. Salkind (Ed), Encyclopedia of research design (pp. 159–162). Sage Publication.
44. Murata, A., & Kattubadi, S. (2012). Grade 3 students’ mathematization through modeling: Situation models and solution models with mutli-digit subtraction problem solving. The Journal of Mathematical Behavior, 31(1), 15-28.
45. National Council of Teachers of Mathematics [NCTM] (2000). Principles and standards for school mathematics. NCTM.
46. Nichols, P.D. (1994). A framework for developing cognitively diagnostic assessments. Review of Educational Research, 64(4), 575-603.
47. Nichols, P.D., Kobrin, J.L., Lai, E., & Koepfler, J.D. (2017). The role of theories of learning and cognition in assessment design and development. In A.A. Rupp & J.P. Leighton (Eds.), The handbook of cognition and assessment: Frameworks, methodologies, and applications (1st ed., pp. 41–74). Wiley Blackwell.
48. Nuerk, H.C., Moeller, K., & Willmes, K. (2015). Multi-digit number processing: Overview, conceptual clarifications, and language influences. Oxford University Press.
49. Ojose, B. (2015). Common misconceptions in mathematics: Strategies to correct them. University Press of America.
50. Pallant, J. (2016). SPSS survival manual: A step by step guide to data analysis using SPSS program (6th ed.). McGraw-Hill Education.
51. Pellegrino, J.W., & Chudowsky, N. (2003). Focus article: The foundations of assessment. Measurement: Interdisciplinary Research and Perspectives, 1(2), 103-148.
52. Pellegrino, J.W., Chudowsky, N., & Glaser, R. (2001). Knowing what students know: The science and design of educational assessment. National Academy Press.
53. Pelton, T., Milford, T., & Pelton, L.F. (2018). Developing Mastery of Time Concepts by Integrating Lessons and Apps. In N. Calder, K. Larkin & N. Sinclair (Eds.), Using Mobile Technologies in the Teaching and Learning of Mathematics (pp. 153-166). Springer.
54. Polit, D.F., & Beck, C.T. (2006). The Content Validity Index: Are you sure you know what's being reported? Critique and recommendations. Research in Nursing and Health, 29(5), 489-497.
55. Rittle-Johnson, B., & Schneider, M. (2015). Developing conceptual and procedural knowledge in mathematics. In R. Cohen Kadosh & A. Dowker (Eds.), Oxford handbook of numerical cognition (pp. 1102 1118). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199642342.013.014
56. Russell, M., & Masters, J. (2009, April 13-17). Formative Diagnostic Assessment in Algebra and Geometry. Paper presented at the Annual Meeting of the American Education Research Association, San Diego, CA.
57. Salkind, N. (2010) Convenience sampling. In N. Salkind (Ed.), Encyclopedia of research design (p. 254). Sage publications.
58. Schultz, M., Lawrie, G.A., Bailey, C.H., Bedford, S.B., Dargaville, T.R., O'Brien, G., ... & Wright, A.H. (2017). Evaluation of diagnostic tools that tertiary teachers can apply to profile their students’ conceptions. International Journal of Science Education, 39(5), 565-586.
59. Sia, C.J.L. (2017). Development and validation of Cognitive Diagnostic Assessment (CDA) for primary mathematics learning of time [Unpublished master's thesis]. Universiti Sains.
60. Sia, C.J.L., & Lim, C.S. (2018). Cognitive diagnostic assessment: An alternative mode of assessment for learning. In D.R. Thompson, M. Burton, A. Cusi, & D. Wright (Eds.), Classroom assessment in mathematics (pp. 123-137). Springer.
61. Sia, C.J.L., Lim, C.S., Chew, C.M., & Kor, L.K. (2019). Expert-based cognitive model and student-based cognitive model in the learning of “Time”: Match or mismatch? International Journal of Science and Mathematics Education, 17(6), 1–19.
62. Tan, P.L., Kor, L.K. & Lim, C.S. (2019). Abstracting common errors in the learning of time intervals via cognitive diagnostic assessment. Creative Practices in Language Learning and Teaching (CPLT) Special Issue: Generating New Knowledge through Best Practices in Computing and Mathematical Sciences, 7(1), 3-10.
63. Tan, P.L., Lim, C.S., & Kor, L.K. (2017). Diagnosing primary pupils' learning of the concept of" after" in the topic" time" through knowledge states by using cognitive diagnostic assessment. Malaysian Journal of Learning and Instruction, 14(2), 145-175.
64. Tatsuoka, K.K. (1986). Toward an integration of Item-Response Theory and cognitive error diagnosis. In N. Frederiksen, R. Glaser, A. Lesgold, & M.G. Shafto (Eds.), Diagnostic monitoring of skill and knowledge acquisition (pp. 453-488). Lawrence Erlbaum Associates.
65. Tatsuoka, K.K. (1991). Boolean algebra applied to determination of universal set of knowledge states (Research Report No: RR-91-44-0NR). Educational Testing Service.
66. Tatsuoka, K.K. (2009). Cognitive assessment: An introduction to the Rule Space Method. Routledge.
67. Van de Walle, J.A., Karp, K.S., & Bay-Williams, J.M. (2012). Elementary and secondary school mathematics: Teaching with developmental approach. Pearson.
68. Van Steenbrugge, H., Valcke, M., & Desoete, A. (2010). Mathematics learning difficulties in primary education: Teachers’ professional knowledge and the use of commercially available learning packages. Educational Studies, 36(1), 59-71.
69. Wang, C., & Gierl, M.J. (2011). Using the Attribute Hierarchy Method to make diagnostic inferences about examinees’ cognitive skills in critical reading. Journal of Educational Measurement, 48(2), 165-187.
70. Wurpts, I.C., & Geiser, C. (2014). Is adding more indicators to a latent class analysis beneficial or detrimental? Results of a Monte-Carlo study. Frontiers in Psychology, 5(920), 1-15.