Investigation of Cross-Cultural Measurement Invariance through Multi-Group Confirmatory Factor Analysis and Multi-Group Latent Class Analysis

Year 2025, Volume: 33 Issue: 4, 816 - 828, 11.10.2025

Fatma Coşkun , Hamide Deniz Gülleroğlu

https://doi.org/10.24106/kefdergi.1795829

Abstract

Purpose: This study aims to examine cross-cultural measurement invariance using multi-group confirmatory factor analysis (MG-CFA) and multi-group latent class analysis (MG-LCA).
Design/Methodology/Approach: For this purpose, data obtained from the 'Students Like Learning Mathematics Scale' in the TIMSS 2019 study were used. The sample of the research was determined using the maximum variation sampling method. Measurement invariance analyses were conducted on 15 comparison groups formed by Singapore, Hong Kong, Japan, Norway, Turkey, and South Africa.
Findings: The MG-CFA results showed that strict measurement invariance was achieved only between Singapore-Hong Kong and Hong Kong-Norway. Between South Africa and Turkey, measurement invariance was observed at the level of structural invariance, while in other groups, it was achieved at the level of metric invariance. According to the MG-LCA results, measurement invariance was established at the partially homogeneous model level for some groups and at the heterogeneous model level for others.
Highlights: In MG-LCA, it was determined that for the cross-cultural comparisons where measurement invariance was achieved at the partially homogeneous model level, measurement invariance was largely achieved at the metric invariance level according to MG-CFA. The MG-CFA and MG-LCA approaches address measurement invariance from different perspectives. MG-CFA is a sample- and data-oriented approach, whereas MG-LCA is a person-centered approach. Using these two methods together provides more comprehensive insights into why measurement invariance could not be achieved.

Keywords

Measurement invariance , Confirmatory factor analysis , Latent class analysis , TIMSS 2019 , Student surveys

Kültürlerarası Ölçme Değişmezliğinin Çok Gruplu Doğrulayıcı Faktör Analizi ve Çok Gruplu Örtük Sınıf Analiziyle İncelenmesi

Abstract

Çalışmanın amacı: Bu çalışmada, kültürlerarası ölçme değişmezliğinin çok gruplu doğrulayıcı faktör analizi (ÇG-DFA) ve çok gruplu örtük sınıf analizi (ÇG-ÖSA) ile incelenmesi amaçlanmıştır.
Materyal ve Yöntem: Bu amaç doğrultusunda, TIMSS 2019 uygulamasında “Matematik Öğrenmeyi Sevme Ölçeği” ile elde edilen veriler kullanılmıştır. Araştırmanın örneklemi, maksimum çeşitlilik yöntemi ile belirlenmiştir. Ölçme değişmezliği analizleri Singapur, Hong Kong, Japonya, Norveç, Türkiye ve Güney Afrika Cumhuriyeti’nin oluşturduğu 15 karşılaştırma grubu üzerinde gerçekleştirilmiştir.
Bulgular: ÇG-DFA sonuçları, katı ölçme değişmezliğinin yalnızca Singapur-Hong Kong ve Hong Kong-Norveç arasında sağlandığını göstermiştir. Güney Afrika Cumhuriyeti ve Türkiye arasında ölçme değişmezliği yapısal değişmezlik düzeyinde kalırken; diğer gruplarda arasında ölçme değişmezliğinin metrik değişmezlik düzeyinde sağlandığı görülmüştür. ÇG-ÖSA sonuçlarına göre, bazı gruplar arasında kısmi homojen model, diğerleri arasında ise heterojen model düzeyinde ölçme değişmezliği sağlanmıştır.
Önemli Vurgular: ÇG-ÖSA’ da ölçme değişmezliğinin kısmi homojen model düzeyinde sağlandığı kültürlerarasında ÇG-DFA’ya göre ölçme değişmezliğinin büyük oranda metrik değişmezlik düzeyinde sağlandığı belirlenmiştir. ÇG-DFA ve ÇG-ÖSA yaklaşımları ölçme değişmezliğini farklı boyutlardan ele almaktadır. ÇG-DFA, örneklem ve veri odaklı bir yaklaşımken; ÇG-ÖSA birey merkezli bir yaklaşımdır. Bu iki yöntemin birlikte kullanılması, ölçme değişmezliğinin neden sağlanamadığı hakkında daha kapsamlı bilgiler sağlamaktadır.

Keywords

Ölçme değişmezliği , Doğrulayıcı faktör analizi , Örtük sınıf analizi , TIMSS 2019 , Öğrenci anketleri

References

• Altıntaş, Ö. (2016). Ankara Üniversitesi yabancı uyruklu öğrenci seçme testinin ölçme değişmezliğinin örtük sınıf ve Rash Modeline göre incelenmesi (Yayınlanmamış Doktora Tezi), Ankara Üniversitesi, Ankara.
• Arıcıgil-Çilan, Ç. (2015). Uygulamalı gizli sınıf analizi. İstanbul: Çağlayan Kitabevi.
• Bergman, L.R., & Magnusson, D. (1997). A person-oriented approach in research on developmental psychopathology. Development and psychopathology, 9(2), 291-319.
• Bergman, L.R., & Wangby, M. (2014). The person-oriented approach: A short theoretical and practical guide. Estonian Journal of Education, 2(1), 29-49. doi: http://dx.doi.org/10.12697/eha.2014.2.1.02b
• Borsboom, D. (2006). When does measurement invariance matter? Medical care, 44(11), 176-181.
• Brown, T. A., & Moore, M. T. (2012). Confirmatory factor analysis. In R. H. Hoyle (Ed.), Handbook of structural equation modeling (pp. 361-379). New York, NY: Guilford Press.
• Cheung, G.W., & Rensvold, R.B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233-255. doi: http://dx.doi.org/10.1207/S15328007SEM0902_5
• Cieciuch, J., Davidov, E., Schmidt, P., & Algesheimer, R. (2019). How to obtain comparable measures for cross-national comparisons. Kölner Zeitschrift für Soziologie und Sozialpsychologie, 71(S1), 157-186. https://doi.org/10.5167/uzh-172539
• Clogg, C.C., & Goodman, L.A. (1985). Simultaneous latent structure analysis in several groups. Sociological Methodology, 15(1), 81-110.
• Collins, L. M., & Lanza, S. T. (2009). Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences. Hoboken, NJ: Wiley. https://doi.org/10.1002/9780470567333
• Coşkun, F. (2023). PISA 2018 verisi üzerinden öğrencilerin kültürlerarası iletişim algısının örtük sınıf modelleri ile incelenmesi: türkiye örneklemi. Gazi Eğitim Bilimleri Dergisi, 9(1), 1-20.
• Coşkun, F., & Gülleroğlu, H. D. (2023). Öğrencileri eğitimde internet kullanımına yönlendirmeye ilişkin öğretmen profillerinin PISA 2018 uygulaması kapsamında karşılaştırması. Journal of History School, 16(LXV), 1530-1556. https://doi.org/10.29228/joh.69740
• Çüm, S., Demir, E. K., Akın Arıkan, Ç., & Şahin, M. D. (2020). Okuma Zevki Ve Okuma Çeşitliliğinin Örtük Sınıf Analizi: Türkiye Ve Çin İncelemesi. Gazi Üniversitesi Gazi Eğitim Fakültesi Dergisi, 40(3), 943-977.
• Davidov, E., Meuleman, B., Cieciuch, J., Schmidt, P., and Billiet, J. (2014). Measurement equivalence in cross-national research. Annual Review of Sociology, 40(1), 55-75. doi: http://dx.doi.org/10.1146/annurev-soc-071913-043137
• Demir, E. (2019). R diliyle istatistik uygulamaları. Ankara: Pegem Akademi.
• DeMars, C.E. (2016). Partially compensatory multidimensional item response theory models: Two alternate model forms. Educational and Psychological Measurement, 76(2), 1-27. doi: http://dx.doi.org/10.1177/0013164415589595
• Eid, M., Langeheine, R., & Diener, E. (2003). Comparing typological structures across cultures by multigroup latent class analysis: A primer. Journal of Cross-Cultural Psychology, 34(2), 195-210. https://doi.org/10.1177/0022022102250427
• Ercikan, K., Por, H.-H., & Guo, H. (2023). Cross-cultural validity and comparability in assessments of complex constructs. Educational Testing Service. https://oecd-ilibrary.org
• Foy, P. & LaRoche, S. (2020). Estimating standard errors in the TIMSS 2019 results. In M.O. Martin, M. von Davier, and I.V.S. Mullis, (Eds.) TIMSS 2019 Technical report. https://timssandpirls.bc.edu/timss2019/methods
• Fraenkel, J.R., & Wallen, N. E. (2009). How to design and evaluate research in education (7th ed.). Boston: McGraw Hill Higher Education.
• Gliner, J.A., Morgan, G.A., & Leech, N.L. (2017). Research methods in applied settings: An integrated approach to design and analysis (Third Edition). New York, London: Routledge Taylor & Francis Group
• Green B.F. (1952). Latent structure analysis and its relation to factor analysis. Journal of the American Statistical Association, 47(257), 71-76.
• Güngör, D., Korkmaz, M., & Somer, O. (2013). Çoklu-grup örtük sınıf analizi ve ölçme eşdeğerliği. Türk Psikoloji Dergisi, 28(72), 48-57.
• Güngör-Culha, D. (2012). Örtük sınıf analizlerinde ölçme eşdeğerliğinin incelenmesi (Yayınlanmamış doktora tezi). Ege Üniversitesi, İzmir.
• Hambleton, R. K., & Zenisky, A. L. (2011). Translating and adapting tests for cross-cultural assessments. In R. K. Hambleton, P. F. Merenda, & C. D. Spielberger (Eds.), Adapting educational and psychological tests for cross-cultural assessment (pp. 3-38). Mahwah, NJ: Erlbaum.
• Hambleton, R.K. (2004). Issues, designs, and technical guidelines for adapting tests into multiple languages and cultures. R.K. Hambleton, P.F. Merenda and C.D. Spielberger (Eds). Adapting educational and psychological tests for cross-cultural assessment (pp. 3-40). London: Psychology Press.
• Hernández-Torrano, D., & Courtney, M. G. R. (2021). Modern international large-scale assessment in education: An integrative review and mapping of the literature. Large-scale Assessments in Education, 9(17). https://doi.org/10.1186/s40536-021-00109-1
• Janousch, C., Sinha, M., & Springer, M. (2022). Resilience profiles across context: A latent profile analysis in a German, Greek, and Swiss sample of adolescents. PLOS ONE, 17(1), e0262039. https://doi.org/10.1371/journal.pone.0262039
• Johansson, S. (2016). International large-scale assessments: What uses, what consequences? Educational Research, 58(2), 139-148. https://doi.org/10.1080/00131881.2016.1165559
• Johnson, T. (1998). Approaches to equivalence in cross-cultural and cross-national survey research. In J. A. Harkness (Ed.), Zuma-Nachrichten Spezial Volume 3: Cross-Cultural Survey Equivalence (pp. 1–40). Mannheim: Zuma.
• Kankaraš M., Moors G., and Vermunt J.K. (2011). Testing for measurement invariance with latent class analysis. In E. Davidov, P. Schmidt, J. Billiet (Eds.), Cross-cultural analysis: Methods and applications (pp. 359-384). New York: Routledge.
• Kankaras, M. (2010). Essays on measurement equivalence in cross-cultural survey research: A latent class approach (Unpublished PhD thesis). Tilburg University, Netherlands.
• Kankaras, M., & Vermunt, J.K. (2014). Simultaneous latent class analysis across groups. A.C. Michalos (Ed.) Encyclopedia of quality of life and well-being research (pp. 5969-5974). Heidelberh: Springer.
• Kankaraš, M., Moors, G., & Vermunt, J. K. (2018). Testing for measurement invariance with latent class analysis. In E. Davidov, P. Schmidt, & J. Billiet (Eds.), Cross-cultural analysis (2nd ed., pp. 361-384). Routledge. https://doi.org/10.4324/9781315537078
• Kankaras, M., Vermunt, J. K., & Moors, G. (2011). Measurement equivalence of ordinal items: A comparison of factor analytic, item response theory, and latent class approaches. Sociological Methods & Research, 40(2), 279-310. https://doi.org/10.1177/0049124111405301
• Kim, E. S., Cao, C., Wang, Y., & Nguyen, D. T. (2017). Measurement invariance testing with many groups: A comparison of five approaches. Structural Equation Modeling: A Multidisciplinary Journal, 24(4), 524-544. https://doi.org/10.1080/10705511.2017.1304822
• Kline, R.B. (2016). Principles and practice of structural equation modeling (Fourth Edition). New York, London: The Guilford Press.
• Kirsch, I., & Braun, H. (2020). Changing times, changing needs: Enhancing the utility of international large-scale assessments. Large-scale Assessments in Education, 8(10). https://doi.org/10.1186/s40536-020-00088-9
• Küçükoğlu, M., & Ercan, H. (2019). Norveç’te refah devletinin ortaya çıkışı ve gelişimi. Uluslararası Toplum Araştırmaları Dergisi, 11(18), 2275-2308. https://doi.org/10.26466/opus.501680
• Lin, H. T. (2006). A comparison of model selection indices for nested latent class models. Monte Carlo Methods and Applications, 12(3), 239-259. https://doi.org/10.1515/156939606778705164
• Lubke, G., & Neale, M. (2008). Distinguishing between latent classes and continuous factors with categorical outcomes: Class invariance of parameters of factor mixture models. Multivariate Behavioral Research, 43(4), 592-620. https://doi.org/10.1080/00273170802490673
• Maden-Kalkan, Ç., & Yılmaz-Şaşmaz, A. (2021). Covid-19 Salgınının Kentsel Yaşam Kalitesi Açısından Potansiyel Etkileri Çin Örneği. Kent Akademisi, 14(4), 1283-1298.
• Magidson, J., & Vermunt, J.K. (2004). Latent class models. In D. Kaplan (Ed.), Handbook of quantitative methodology for the social sciences (pp. 345–368). Newbury Park, CA: Sage Publications.
• Magidson, J., Vermunt, J.K., & Madura, J. P. (2020). Latent class analysis. London: SAGE Publications Limited.
• Meade, A. W., Johnson, E. C., & Braddy, P. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of applied psychology, 93(3), 568-592. https://doi.org/10.1037/0021-9010.93.3.568
• Meitinger, K., Davidov, E., Schmidt, P., & Braun, M. (2020). Measurement invariance: Testing for it and explaining why it is absent. Survey Research Methods, 14(4), 345–349. https://doi.org/10.5167/uzh-192239
• Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58(4), 525-543.
• Mullis, I.V.S., Martin, M.O., Foy, P., Kelly, D. L., & Fishbein, B. (2020). TIMSS 2019 International Results in Mathematics and Science. Retrieved from Boston College, TIMSS & PIRLS International Study Center website: https://timssandpirls.bc.edu/timss2019/international-results/
• Oberski, D. (2016). Mixture models: Latent profile and latent class analysis. In J. Robertson and M. Kaptein (Eds.), Modern Statistical Methods for HCI. Cham, Switzerlans: Springer International Publishing.
• Özçelik-Tezel, C. (2007). Ulusal kimliğin oluşumunda müze ve toplum ilişkisi: Singapur. Elektronik Sosyal Bilimler Dergisi, 6(20), 133-155.
• Steenkamp, J.B.E., & Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. Journal of consumer research, 25(1), 78-90.
• Tabachnick, B.G., & Fidell, L.S. (2019). Using multivariate statistics (Seventh Editon). Boston, MA: Pearson.
• Thissen, D. (2001). IRTLRDIF v. 2.0 b: Software for the computation of the statistics involved in item response theory likelihood-ratio tests for differential item functioning. Chapel Hill, NC: LL Thurstone Psychometric Laboratory.
• Triandis, H. C. (1972). The analysis of subjective culture. New York, NY: Wiley.
• Van De Schoot, R., Schmidt, P., De Beuckelaer, A., Lek, K., and Zondervan-Zwijnenburg, M. (2015). Measurement invariance. Frontiers in psychology, 6(1064). doi: https://10.3389/fpsyg.2015.01064
• Van de Vijver, F.J., & Leung, K. (2021). Methods and data analysis for cross-cultural research (Second Education). United Kingdom: Cambridge University Press.
• Van de Vijver, F.J.R. (1998). Towards a theory of bias and equivalence. J.A. Harkness (Ed.), Zuma-Nachrichten Spezial Volume 3. Cross-Cultural Survey Equivalence (pp. 41-65). Mannheim: Zuma.
• Van de Vijver, F.J.R., & Leung, K. (2011). Equivalence and bias: A review of concepts, models, and data analytic procedures. D. Matsumoto and F.J.R. Van de Vijver (Eds.). Cross-cultural research methods in psychology (pp. 17–45). New York, NY: Cambridge University Press.
• Vandenberg, R.J., and Lance, C.E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational research methods, 3(1), 4-70.
• Wendt, H., Kasper, D., & Trendtel, M. (2017). Assuming measurement invariance of background indicators in international comparative educational achievement studies: A challenge for the interpretation of achievement differences. Large-scale Assessments in Education, 5(10). https://doi.org/10.1186/s40536-017-0043-9
• Wu, A.D., Li, Z., and Zumbo, B.D. (2007). Decoding the meaning of factorial invariance and updating the practice of multi-group confirmatory factor analysis: A demonstration with TIMSS data. Practical Assessment, Research, and Evaluation, 12(3), 1-26. https://doi.org/10.7275/mhqa-cd89
• Yandı, A., Köse, İ.A., ve Uysal, Ö. (2017). Farklı yöntemlerle ölçme değişmezliğinin incelenmesi: PISA 2012 örneği. Mersin Üniversitesi Eğitim Fakültesi Dergisi, 13(1), 243-253.
• Yin, L., and Fishbein, B. (2020). Creating and interpreting the TIMSS 2019 context questionnaire scales. In M.O. Martin, M. von Davier, & I.V.S. Mullis (Eds.), Methods and Procedures: TIMSS 2019 Technical Report (pp. 16.1-16.331). Boston College, TIMSS & PIRLS International Study Center.
• Zhao, M., & Jin, R. (2023). Advancing a cross-cultural understanding of teacher perceptions of school climate: A latent class analysis using 2018 TALIS data. Frontiers in Psychology, 14, 1129306. https://doi.org/10.3389/fpsyg.2023.1129306