Latent Class Analysis and DIF Testing in Mathematics Achievement: A Comparative Study of Korea and Türkiye Using MIMIC Modeling

Abstract

This study examines the latent classes of mathematics achievement and investigates measurement invariance (differential item functioning) between Korea and Turkey. Moreover, it explores the influence of the country on the latent classes of mathematics achievement. To achieve this, data from eighth-grade students in TIMSS 2019 were analyzed using Latent Class MIMIC Modeling. The findings uncovered diverse latent classes of math achievement and detected both uniform and non-uniform DIF between Korea and Turkey. Furthermore, the country was found to significantly affect the latent class membership of math achievement. This study highlights the necessity of verifying measurement invariance of indicator variables in latent class analysis (LCA). It also sheds light on areas where students performed favorably or unfavorably in mathematics achievement tests across these countries by investigating DIF. These findings have important implications for mathematics education in Korea and Turkey.

Keywords

• Mathematics Achievement
• Latent Class Analysis (LCA)
• Multiple Indicator Multiple Cause (MIMIC) Modeling
• Measurement Invariance
• Differential Item Functioning (DIF)
• TIMSS 2019

Ethical Statement

The authors confirm that there are no conflicts of interest related to this research, and that no unethical behavior occurred during the process of data collection, analysis, and reporting of the results.

References

1. Arıkan, S., Van de Vijver, F., & Yagmur, K. (2016). Factors contributing to mathematics achievement differences of Turkish and Australian students in TIMSS 2007 and 2011. EURASIA Journal of Mathematics, Science and Technology Education, 12(8), 2039-2059. https://doi.org/10.12973/eurasia.2016.1268a
2. Asparouhov, T., & Muthén, B. (2014). Auxiliary variables in mixture modeling: Three-step approaches using M plus. Structural equation modeling: A multidisciplinary Journal, 21(3), 329-341. https://doi.org/10.1080/10705511.2014.915181
3. Badri, M. (2019). School Emphasis on Academic Success and TIMSS Science/Math Achievements. International Journal of Research in Education and Science, 5(1), 176-189. https://eric.ed.gov/?id=EJ1197994
4. Clark, S. L., & Muthén, B. (2009). Relating latent class analysis results to variables not included in the analysis. https://www.statmodel.com/download/relatinglca.pdf
5. Demirus, K. B., & Pektas, S. (2022). Investigation of Timss 2015 science test items in terms of differential item functioning according to language and culture. International Journal of Education Technology & Scientific Researches, 7(18), 1166-1178. http://dx.doi.org/10.35826/ijetsar.499
6. De Ayala, R. J., Kim, S. H., Stapleton, L. M., & Dayton, C. M. (2002). Differential item functioning: A mixture distribution conceptualization. International Journal of Testing, 2(3-4), 243-276. https://doi.org/10.1080/15305058.2002.9669495
7. Dittrich, K., & Neuhaus, D. A. (2023). Korea’s ‘education fever’ from the late nineteenth to the early twenty-first century. History of Education, 52(4), 539-552. https://doi.org/10.1080/0046760X.2022.2098391
8. Dorans, N. J., & Holland, P. W. (1992). DIF detection and description: Mantel‐Haenszel and standardization 1, 2. ETS Research Report Series, 1992(1), i-40. https://doi.org/10.1002/j.2333-8504.1992.tb01440.x

9. Geesa, R. L., Izci, B., Song, H., & Chen, S. (2019). Exploring Factors of Home Resources and Attitudes Towards Mathematics in Mathematics Achievement in South Korea, Turkey, and the United States. Eurasia Journal of Mathematics, Science and Technology Education, 15(9), eM_1751. https://doi.org/10.29333/ejmste/108487
10. Geesa, R. L., Izci, B., Chen, S., & Song, H. S. (2020). The Role of Gender and Attitudes toward Science in Fourth and Eighth Graders' Science Achievement in South Korea, Turkey, and the United States. Journal of Research in Education, 29(2), 54-87. https://eric.ed.gov/?id=EJ1274027
11. Guhl, P. (2019). The impact of early math and numeracy skills on academic achievement in elementary school.
12. Hambleton, R. K. (2001). The next generation of the ITC test translation and adaptation guidelines. European journal of psychological assessment, 17(3), 164-172. https://doi.org/10.1027/1015-5759.17.3.164
13. Huang, J. (2020). Assessing robustness of the Rasch mixture model to detect differential item functioning: A Monte Carlo simulation study (Doctoral dissertation, University of Denver). https://digitalcommons.du.edu/etd/1784/
14. Hwang, H. J., & Hwang, H. K. (2008). An Effect of the Constructivist Discussion on Learning Attitude in Mathematics and Children's Mathematics Achievement. Education of Primary School Mathematics, 11(1), 59-74. https://koreascience.kr/article/JAKO200801440607130.page
15. Im, S., & Park, H. J. (2010). A comparison of US and Korean students' mathematics skills using a cognitive diagnostic testing method: linkage to instruction. Educational Research and Evaluation, 16(3), 287-301. https://doi.org/10.1080/13803611.2010.523294
16. Jedidi, K., Jagpal, H. S., & DeSarbo, W. S. (1997). Finite-mixture structural equation models for response-based segmentation and unobserved heterogeneity, Marketing Science 16(1), 39-59. https://doi.org/10.1287/mksc.16.1.39
17. Kalaycioğlu, D. B., & Berberoğlu, G. (2011). Differential item functioning analysis of the science and mathematics items in the university entrance examinations in Turkey. Journal of Psychoeducational Assessment, 29(5), 467-478. https://doi.org/10.1177/0734282910391623
18. Klieme, E., & Baumert, J. (2001). Identifying national cultures of mathematics education: Analysis of cognitive demands and differential item functioning in TIMSS. European journal of psychology of education, 16, 385-402. https://link.springer.com/article/10.1007/BF03173189
19. Ko, D. H., & Jung, H. S. (2020). Analysis on the Effect of Mathematics Class Characteristics and Mathematical Confidence on Mathematical Academic Achievement: Applying Hierarchical Linear Model. School Mathematics, 22(2), 313-332. https://doi.org/10.29275/sm.2020.06.22.2.313
20. Lee, J., & Stankov, L. (2018). Non-cognitive predictors of academic achievement: Evidence from TIMSS and PISA. Learning and Individual Differences, 65, 50-64. https://doi.org/10.1016/j.lindif.2018.05.009
21. Lubinski, D., Benbow, C. P., & Kell, H. J. (2014). Life paths and accomplishments of mathematically precocious males and females four decades later. Psychological Science, 25(12), 2217-2232. https://doi.org/10.1177/0956797614551371
22. Lyons-Thomas, J., Sandilands, D., & Ercikan, K. (2014). Gender Differential Item Functioning in Mathematics in Four International Jurisdictions. Education & Science/Egitim ve Bilim, 39, 20-32.
23. Masyn, K. E. (2017). Measurement invariance and differential item functioning in latent class analysis with stepwise multiple indicator multiple cause modeling. Structural Equation Modeling: A Multidisciplinary Journal, 24(2), 180-197. https://doi.org/10.1080/10705511.2016.1254049
24. Mullis, I. V., & Martin, M. O. (2017). TIMSS 2019 Assessment Frameworks. International Association for the Evaluation of Educational Achievement. https://eric.ed.gov/?id=ed596167
25. Mullis, I. V., Martin, M. O., Foy, P., Kelly, D. L., & Fishbein, B. (2020). TIMSS 2019 international results in mathematics and science. https://www.skolporten.se/app/uploads/2020/12/timss-2019-highlights-1.pdf
26. Nylund-Gibson, K., & Choi, A. Y. (2018). Ten frequently asked questions about latent class analysis. Translational Issues in Psychological Science, 4(4), 440-461. https://doi.org/10.1037/tps0000176
27. Nylund-Gibson, K., & Masyn, K. E. (2016). Covariates and mixture modeling: Results of a simulation study exploring the impact of misspecified effects on class enumeration. Structural Equation Modeling: A Multidisciplinary Journal, 23(6), 782-797. https://doi.org/10.1080/10705511.2016.1221313
28. Ram, N., & Grimm, K. J. (2009). Methods and measures: Growth mixture modeling: A method for identifying differences in longitudinal change among unobserved groups. International journal of behavioral development, 33(6), 565-576. https://doi.org/10.1177/0165025409343765
29. Saatcioglu, F. M. (2022). Differential item functioning across gender with MIMIC modeling: PISA 2018 financial literacy items. International Journal of Assessment Tools in Education, 9(3), 631-653. https://doi.org/10.21449/ijate.1076464
30. Samuelsen, K. M. (2008). Examining differential item functioning from a latent mixture perspective. Advances in latent variable mixture models, 177-197.
31. Şen, S., & Arıkan, M. (2015). A diagnostic comparison of Turkish and Korean students’ mathematics performances on the TIMSS 2011 assessment. Journal of Measurement and Evaluation in Education and Psychology, 6(2). https://doi.org/10.21031/epod.65266
32. Shin, K., Jahng, K. E., & Kim, D. (2019). Stories of South Korean mothers’ education fever for their children’s education. Asia Pacific Journal of Education, 39(3), 338-356. https://doi.org/10.1080/02188791.2019.1607720
33. Sohn, W. S. (2010). Exploring Potential Sources of DIF for PISA 2006 Mathematics Literacy Items: Application of Logistic Regression Analysis. Journal of Educational Evaluation, 23(2), 371-390. https://scholar-kyobobook-co-kr-ssl.ca.skku.edu/article/detail/4010023071385
34. Song, H. S., Kim, H. C., & Jung, H. S. (2023). The Effect of Participation in Private Education on Math Affective Attitudes: Measurement Invariance and Differential Item Functioning in Latent Class MIMIC Model. Journal of Educational Evaluation, 36(4), 687-709. http://dx.doi.org/10.31158/JEEV.2023.36.4.687
35. Tsaousis, I., Sideridis, G. D., & AlGhamdi, H. M. (2020). Measurement invariance and differential item functioning across gender within a latent class analysis framework: Evidence from a high-stakes test for university admission in Saudi Arabia. Frontiers in Psychology, 11, 622. https://doi.org/10.3389/fpsyg.2020.00622
36. Vermunt, J. K. (2010). Latent class modeling with covariates: Two improved three-step approaches. Political analysis, 18(4), 450-469. https://doi.org/10.1093/pan/mpq025
37. Wang, X. S., Perry, L. B., Malpique, A., & Ide, T. (2023). Factors predicting mathematics achievement in PISA: a systematic review. Large-scale Assessments in Education, 11(1), 24. https://doi.org/10.1186/s40536-023-00174-8
38. Wiberg, M. (2019). The relationship between TIMSS mathematics achievements, grades, and national test scores. Education Inquiry, 10(4), 328-343. https://doi.org/10.1080/20004508.2019.1579626
39. Woo, H., & Hodges, N. N. (2015). Education fever: Exploring private education consumption motivations among Korean parents of preschool children. Family and Consumer Sciences Research Journal, 44(2), 127-142. https://doi.org/10.1111/fcsr.12131
40. Yildirim, H. H. (2006). The DIF analysis of mathematics items in the international assessment programs. https://www.proquest.com/openview/37bc178a79e82e2c13ca84983403a39a/1?cbl=2026366&diss=y&pq-origsite=gscholar
41. Yoon, J. Y., & Lee, Y. S. (2013). A Study of DIF Analyses using TIMSS (2007) Mathematics Test Across South Korea, U. S., and Singapore. Journal of Educational Evaluation, 26(2), 415-439. https://scholar-kyobobook-co-kr-ssl.ca.skku.edu/article/detail/4010023595562