An Application of Latent Class Analysis for TIMSS 2015 Data: Detecting Heterogeneous Subgroups

Abstract

This study aimed to investigate the heterogeneity of the TIMSS 2015 data from Turkey and the USA 8th grade math. Latent Class Analysis (LCA) was used to determine the latent classes that cause heterogeneity in the data by using categorical observed variables. As a result of the LCA, supporting absolute and relative model fit indices through AvePP and entropy values, it was concluded that the data obtained from both countries fit the three-class model. The latent class probabilities and conditional response probabilities were examined for homogeneity and degree of segregation of the classes from each other. Based on the findings, it is recommended that the assumption of homogeneity in international evaluations be evaluated empirically with LCA. With this article, an example of the application of LCA is provided, and it is believed to be useful for researchers in the context of education and psychological evaluation.

Keywords

• Latent class analysis
• TIMSS 2015
• heterogeneity

References

1. Agresti, A. (1990). Categorical data analysis. New York: Wiley.
2. Akaike, H. (1987). Factor analysis and AIC. Psychometrika, 52, 317-332.
3. Baghaei, P., & Carstensen, C. H. (2013). Fitting the mixture Rasch model to a reading comprehension test: Identifying reader types. Practical Assessment, Rese, 18(5), 1–13. doi: 10.7275/n191-pt86
4. Bolt, D. M., Cohen, A. S., & Wollack, J. A. (2002). Item parameter estimation under conditions of test speededness: Application of a mixture Rasch model with ordinal constraints. Journal of Educational Measurement, 39, 331-348. doi: 10.1111/j.1745-3984.2002.tb01146.x
5. Butera, N. M., Lanza, S. T., & Coffman, D. L. (2014). A framework for estimating causal effects in latent class analysis: Is there a causal link between early sex and subsequent profiles of delinquency? Prevention Science, 15(3), 397– 407. doi: 10.1007/s11121-013-0417-3
6. Chung, H., Park, Y., & Lanza, S. T. (2005). Latent transition analysis with covariates: Pubertal timing and substance use behaviours in adolescent females. Statistics in Medicine, 24, 2895-2910. doi: 10.1002/sim.2148
7. Clark, S. L. (2010). Mixture modeling with behavioral data (Unpublished doctoral dissertation). University of California, Los Angeles.
8. Eid, M., Langeheine, R., & Diener, E. (2003). Comparing typological structures across cultures by multigroup latent class analysis: A primer. Journal of Cross-Cultural Psychology, 34(2), 195-210. doi: 10.1177/0022022102250427

9. Clark, S. L., & Muthén, B. O. (2009). Relating latent class analysis results to variables not included in the analysis. 2009 Manuscript submitted for publication. Retrieved from http://www.statmodel.com/download/relatinglca.pdf
10. Collins, L. M., & Lanza, S. T. (2010). Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences. New Jersey: Wiley.
11. De Ayala, R. J., & Santiago, S. Y. (2017). An introduction to mixture item response theory models. Journal of School Psychology, 60, 25-40. doi: 10.1016/j.jsp.2016.01.002
12. DeMars, C. E., & Lau, A. (2011). Differential item functioning detection with latent classes: How accurately detect who is responding differentially? Educational and Psychological Measurement, 71(4), 597–616. doi:10.1177/0013164411404221
13. Embretson, S. E. (2007). Mixture Rasch models for measurement in cognitive psychology. In M. von Davier & C. H. Carstensen (Eds.), Multivariate and mixture distribution Rasch models: Extensions and applications (pp. 235-253). New York: Springer Verlag.
14. Glück, J., & Spiel, C. (2007). Studying development via Item Response Model: A wide range of potential uses. In M. von Davier, & C. H. Carstensen (Eds.), Multivariate and mixture distribution Rasch models: Extensions and applications (pp. 281-292). New York: Springer Verlag.
15. Goodman, L. A. (2002). Latent class analysis: The empirical study of latent types, latent variables, and latent structures. In J. A. Hagenaars & A. L. McCutcheon (Eds.), Applied latent class analysis. New York: Cambridge University Press.
16. Güngör Culha, D. & Korkmaz, M. (2011). Örtük sınıf analizi ile bir örnek uygulama. Eğitimde ve Psikolojide Ölçme Değerlendirme Dergisi, 2(2), 191-199.
17. Güngör, Culha, D., Korkmaz, M. & Somer, O. (2013). Çoklu-grup örtük sınıf analizi ve ölçme eşdeğerliği. Türk Psikoloji Dergisi, 28(72), 48-57.
18. Hagenaars, J. A., & McCutcheon, A. (2002). Applied latent class analysis. Cambridge, UK: Cambridge University Press.
19. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of Item Response Theory. Newbury Park, CA: Sage.
20. Jeon, M. (2018). A constrained confirmatory mixture IRT model: Extensions and estimation of the Saltus model using Mplus. The Quantitative Methods for Psychology, 14, 120–136. doi: 10.20982/tqmp.14.2.p120
21. Jung, T., & Wickrama, K. A. S. (2008). An introduction to latent class growth analysis and growth mixture modeling. Social and Personality Psychology Compass, 2(1), 302–317. doi: 10.1111/j.1751-9004.2007.00054.x
22. Kankaraš, M., Vermunt, J. K., & Moors, G. (2011). Measurement equivalence of ordinal items: A comparison of factor analytic, Item Response Theory, and latent class approaches. Sociological Methods & Research, 40, 279–310. doi: 10.1177/0049124111405301
23. Kreiner, S., & Christensen, K. B. (2007). Validity and objectivity in health-related scales: Analysis by graphical loglinear Rasch models. In M. von Davier & C. H. Carstensen (Eds.). Multivariate and mixture distribution Rasch models: Extensions and applications (pp. 329-346). New York: Springer Verlag. doi:10.1007/s11336-013-9347-z
24. Lanza, S. T., Flaherty, B. P., & Collins, L. M. (2003). Latent class and latent transition analysis. In J. A. Schinka & W. F. Velicer (Eds.), Handbook of psychology: Vol. 2, research methods in psychology (pp. 663-685). Hoboken, NJ: Wiley.
25. Lazarsfeld, P. F., & Henry, N. W. (1968). Latent structure analysis. Boston: Houghton Mifflin.
26. Leech, R. M., McNaughton, S. A., & Timperio, A. (2014). The clustering of diet, physical activity and sedentary behavior in children and adolescents: A review. International Journal of Behavioral Nutrition and Physical Activity, 11, 4. doi: 10.1186/1479-5868-11-4
27. Lo, Y., Mendell, N. R., & Rubin, D. B. (2001). Testing the number of components in a normal mixture. Biometrika, 88(3), 767-778. doi: 10.1093/biomet/88.3.767
28. Lubke, G. H., & Muthén, B. (2005). Investigating population heterogeneity with factor mixture models. Psychological Methods, 10, 21–39. doi: 10.1037/1082-989X.10.1.21
29. Lubke, G., & Neale, M. C. (2006). Distinguishing between latent classes and continuous factors: Resolution by maximum likelihood? Multivariate Behavioral Research, 41, 499–532. doi: 10.1207/s15327906mbr4104_4
30. Magidson, J., & Vermunt, J. K. (2002). Latent class models for clustering: A comparison with K-means. Canadian Journal of Marketing, 20(1), 36–43.
31. Martin, M. O., Mullis, I. V. S., & Hooper, M. (2016). Methods and procedures in TIMSS 2015. Chestnut Hill, MA: Boston College, TIMSS & PIRLS International Study Center. Zugriff am (Vol. 21).
32. Masyn, K. (2017). Measurement invariance and differential item functioning in latent class analysis with stepwise multiple indicator multiple cause modeling. Structural Equation Modeling, 24, 180-197. doi: 10.1080/10705511.2016.1254049
33. McLachlan, G. J., & Peel, D. (2004). Finite mixture models. New York: Wiley.
34. Morin, A. J., Meyer, J. P., Creusier, J., & Biétry, F. (2016). Multiple-group analysis of similarity in latent profile solutions. Organizational Research Methods, 19(2), 231–254. doi: 10.1177/1094428115621148
35. Mullis, I. V., Martin, M. O., Foy, P., & Hooper, M. (2016). TIMSS 2015 international results in mathematics. Amsterdam, The Netherlands: International Association for the Evaluation of Educational Achievement.
36. Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23, 13–23. doi: 10.3102/0013189X023002013
37. Mislevy, R., & Huang, C. W. (2007). Measurement models as narrative structures. In M. von Davier & C. H. Carstensen (Eds.), Multivariate and mixture distribution Rasch models: Extensions and applications. New York: Springer Verlag.
38. Muthén, L. K., & Muthén, B. O. (2017). Mplus user’s guide (8th Edition). Los Angeles, CA: Muthén & Muthén.
39. Nagin, D. (2005). Group-based modeling of development. London: Harvard University.
40. Nylund, K. L., Asparouhov, T., & Muthén, B. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling, 14(4), 535-569. doi: 10.1080/10705510701575396
41. Oliveri, M. E., Ercikan, K., Zumbo, B. D., & Lawless, R. (2014). Uncovering substantive patterns in student responses in international large-scale assessments—Comparing a latent class to a manifest DIF approach. International Journal of Testing, 14(3), 265-287. doi: 10.1080/15305058.2014.891223
42. Oliveri, M. E., & von Davier, M. (2011). Investigation of model fit and score scale comparability in international assessments. Psychological Test and Assessment Modeling, 53(3), 315–333.
43. Olson J. S., Hummer A. K., & Harris K. M. (2017). Gender and health behavior clustering among U.S. Young Adults, Biodemography and Social Biology, 63(1), 3-20. doi: 10.1080/19485565.2016.1262238
44. Park, Y. S., Lee, Y.-S., & Xing, K. (2016). Investigating the impact of item parameter drift for item response theory models with mixture distributions. Frontiers in Psychology, 7, 255. doi: 10.3389/fpsyg.2016.00255
45. Rindskopf, D. (2003). Mixture or homogeneous? Comment on Bauer and Curran (2003). Psychological Methods, 8(3), 364–368. doi: 10.1037/1082-989X.8.3.364
46. Rutkowski, L., & Rutkowski, D. (2018). Improving the comparability and local usefulness of international assessments: A look back and a way forward. Scandinavian Journal of Educational Research, 62(3), 354–367. doi: 10.1080/00313831.2016.1261044
47. Samuelsen, K. (2005). Examining differential item functioning from a latent class perspective. Doctoral Dissertation. Available from ProQuest Dissertations and Theses database. (UMI No. 3175148).
48. Schwarz, G. (1978). Estimating the dimension of a model. The Annuals of Statistics, 6(2), 461-464.
49. Sen, S. (2016) Applying the Mixture Rasch Model to the Runco Ideational Behavior Scale. Creativity Research Journal, 28(4), 426-434. doi: 10.1080/10400419.2016.1229985
50. Thomson, S., Wernert, N., O’Grady, E., & Rodrigues, S. (2017). TIMSS 2015: Reporting Australia’s results. Melbourne, Australia: Australian Council for Educational Research.
51. Toker, T. (2016). A comparison latent class analysis and the mixture Rasch model: A cross-cultural comparison of 8th grade mathematics achievement in the fourth international mathematics and science study (TIMSS-2011). Doctoral Dissertation, The Faculty of the Morgridge College of Education University of Denver, USA.
52. Uyar, Ş. (2015). Gözlenen gruplara ve örtük sınıflara göre belirlenen değişen madde fonksiyonunun karşılaştırılması. Yayınlanmamış Doktora Tezi. Hacettepe Üniversitesi, Eğitim Bilimleri Ensitüsü. Ankara.
53. Vermunt, J. K., & Magidson, J. (2004). Latent class analysis. The Sage Encyclopedia of Social Sciences Research Methods (pp. 549-553).
54. Vermunt, J. K., & Magidson, J. (2020). How to perform three-step latent class analysis in the presence of measurement non-invariance or differential item functioning. Structural Equation Modeling: A Multidisciplinary Journal. doi: 10.1080/10705511.2020.1818084
55. Wang, J., & Wang, X. (2012). Structural equation modeling: Applications using Mplus. Hoboken, NJ: John Wiley & Sons.
56. Yandı, A., Köse, İ. A. & Uysal, Ö. (2017). Farklı yöntemlerle ölçme değişmezliğinin incelenmesi: PISA 2012 örneği. Mersin Üniversitesi Eğitim Fakültesi Dergisi, 13(1), 243-253. doi: 10.17860/mersinefd.305952