Examining the Invariance of a Measurement Model by Using the Covariance Structure Approach

Abstract

The primary aim of the present study is to examine the measurement invariance of the structural equating model constructed on the numerical and verbal abilities test for sixth grade students across gender, amount of weekly pocket money and students’ perceptions of the sufficiency of their pocket money. The secondary aim is to illustrate the use of the IBM AMOS-24 software package step by step with examples to address invariance using the covariance structural analysis approach. The research data were collected from 2304 sixth grade students enrolled in public schools within the Keçiören and Pursaklar suburbs in Ankara. The covariance structures analysis approach was employed during the examination of the measurement model invariance. The study revealed that invariance was achieved in terms of configural, measurement (in both measurement weights and measurement residuals) and structural invariance with respect to all subgroups.

Keywords

• Ability
• configural invariance
• measurement invariance
• structural invariance
• structural equation modeling

References

1. Arana, F. G., Rice, K. G., & Ashby, J. S. (2018). Perfectionism in Argentina and the United States: Measurement structure, invariance, and implications for depression. Journal of Personality Assessment, 100(2), 219-230. https://doi.org/10.1080/00223891.2017.1296845
2. Atkinson, R., Atkinson, R., & Hilgard, E. (1995). Psikolojiye giriş II [Introduction to Psychology II]. İstanbul: Sosyal.
3. Bailey, S. K., Neigel, A. R., Dhanani, L. Y., & Sims, V. K. (2018). Establishing measurement equivalence across computer-and paper-based tests of spatial cognition. Human factors, 60(3), 340-350.https://doi.org/10.1177/0018720817747731
4. Başusta, N. B., & Gelbal, S. (2015). Gruplararası karşılaştırmalarda ölçme değişmezliğinin test edilmesi: PISA öğrenci anketi örneği [Testing measurement invariance in intergroup comparisons: PISA student survey sample]. Hacettepe University Journal of Education, 30(4), 80-90.
5. Benson, N. (2008). Cattell-Horn-Carroll cognitive abilities and reading achievement. Journal of Psychoeducational Assessment, 26(1), 27-41. https://doi.org/10.1177/0734282907301424
6. Berkowitz, M., & Stern, E. (2018). Which cognitive abilities make the difference? Predicting academic achievements in advanced STEM studies. Journal of Intelligence, 6(4), 48. https://doi.org/10.3390/jintelligence6040048
7. Blankson, A. N., & McArdle, J. J. (2013). Measurement invariance of cognitive abilities across ethnicity, gender, and time among older Americans. Journals of Gerontology Series B: Psychological Sciences and Social Sciences, 70(3), 386-397. https://doi.org/10.1093/geronb/gbt106
8. Borghans, L., Golsteyn B. H., Heckman J. J., & Humphries J. E. (2016). What grades and achievement tests measure. Proceedings of the National Academy of Sciences, 113 (47), 13354–13359. https://doi.org/10.1073/pnas.1601135113

9. Byrne, B. M. (2013). Structural equation modeling with LISREL, PRELIS, and SIMPLIS: Basic concepts, applications, and programming. Psychology Press.
10. Byrne, Barbara M. (2016) Structural equation modeling with AMOS: Basic concepts, applications, and programming. Routledge.
11. Camerota, M., Willoughby, M. T., Kuhn, L. J., & Blair, C. B. (2018). The childhood executive functioning inventory (CHEXI): Factor structure, measurement invariance, and correlates in US preschoolers. Child Neuropsychology, 24(3), 322-337. https://doi.org/10.1080/09297049.2016.1247795
12. Camilli, G. (2006). Test fairness. In R. L. Brennan’s (Ed.), Educational measurement (pp. 221–256). Westport: Praeger.
13. Campbell, H., Barry, C. L., Joe, N. J., & Finney, J. S. (2008). Configural, metric and scalar invariance of the modified achievement goal questionnaire across African American and White university students. Educational and Psychological Measurement, 68(6), 988-1007. https://doi.org/10.1177/0013164408318766
14. Chavez, L. M., Shrout, P. E., García, P., Forno, E., & Celedón, J. C. (2018). Measurement invariance of the adolescent quality of life-mental health scale (AQOL-MHS) across gender, Age and Treatment Context. Journal of Child and Family Studies, 27(10), 3176-3184. https://doi.org/10.1007/s10826-018-1158-5
15. Cheung, G.W. & Rensvold, R.B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5
16. Chung, H., Kim, J., Park, R., Bamer, A. M., Bocell, F. D., & Amtmann, D. (2016). Testing the measurement invariance of the University of Washington Self-Efficacy Scale short form across four diagnostic subgroups. Quality of life research, 25(10), 2559-2564. https://doi.org/10.1007/s11136-016-1300-z
17. Drasgow, F., & Kanfer, R. (1985). Equivalence of psychological measurement in heterogeneous populations. Journal of Applied Psychology, 70(4), 662-680. http://dx.doi.org/10.1037/0021-9010.70.4.662
18. Floyd, R. G., Keith, T. Z., Taub, G. E., & McGrew, K. S. (2007). Cattell-Horn-Carroll cognitive abilities and their effects on reading decoding skills: g has indirect effects, more specific abilities have direct effects. School Psychology Quarterly, 22(2), 200-233. https://doi.org/10.1037/1045-3830.22.2.200
19. Gaddy, M. A., Casner, H. G., & Rosinski, J. (2018). Factor structure and measurement invariance of the Brief Addiction Monitor. Journal of substance abuse treatment, 90(7), 29-37. https://doi.org/10.1016/j.jsat.2018.04.010
20. Ghanizadeh, A. (2017). The interplay between reflective thinking, critical thinking, self-monitoring, and academic achievement in higher education. Higher Education, 74(1), 101-114. https://doi.org/10.1007/s10734-016-0031-y
21. Kaufman, A. S. (2018). Contemporary intellectual assessment: Theories, tests, and issues. Guilford Publications.
22. Kyllonen, P., & Kell, H. (2018). Ability Tests Measure Personality, Personality Tests Measure Ability: Disentangling Construct and Method in Evaluating the Relationship between Personality and Ability. Journal of Intelligence, 6(3), 32. https://doi.org/10.3390/jintelligence6030032
23. McGrew, K. S. (2005). The Cattell-Horn-Carroll Theory of Cognitive Abilities: Past, Present, and Future. In D. P. Flanagan & P. L. Harrison (Eds.), Contemporary Intellectual Assessment: Theories, Tests, and Issues (p. 136–181). The Guilford Press.
24. Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58(4), 525-543. http://dx.doi.org/10.1007/BF02294825
25. Millfont, T. L., & Fischer, R. (2010). Testing measurement invariance across groups: Applications in cross-cultural research. International Journal of Psychological Research, 3(1), 111-121.
26. Millsap, R., & Tein, J. Y. (2004). Model specification and identification in multiple-group factor analysis of ordered-categorical measures. Multivariate Behavioral Research, 39(3), 479-515. DOI: 10.1207/ S15327906MBR3903_4
27. Özgüven, İ. E. (2007). Psikolojik testler [Psychological tests]. Ankara: Nobel.
28. Pektaş, S. (2018). The effects of differential item functioning determination methods on test parameters estimates, decision studies, g and phi coefficients. (PhD Theses). Ankara: Gazi University Graduate School of Educational Sciences Department of Measurement and Evaluation in Education.
29. Sak, U. (2014). Üstün zekâlılar: Özellikleri tanılanmaları eğitimleri [The gifted: Traits identification training]. Ankara: Vize.
30. Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of psychological research online, 8(2), 23-74.
31. Türk Dil Kurumu (2016). Yetenek [Talent]. It is obtained on 26.10.2019 at http://www.tdk.gov.tr/index.php?option=com_gts&arama=gts&guid=TDK.GTS.5c27e8237eb1a0.18983524 .
32. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70. doi:10.1177/109442810031002
33. Watson, A. M., Thompson, L. F., & Meade, A. W. (2007, April). Measurement invariance of the job satisfaction survey across work contexts. paper presented at the 22nd annual meeting of the society for industrial and organizational psychology, New York. It is obtained at http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.583.5341&rep=rep1&type=pdf .
34. Wicherts, J. M. (2016). The importance of measurement invariance in neurocognitive ability testing. The Clinical Neuropsychologist, 30(7), 1006-1016. https://doi.org/10.1080/13854046.2016.1205136
35. Wicherts, J. M., & Dolan, C. V. (2010). Measurement invariance in confirmatory factor analysis: An illustration using IQ test performance of minorities. Educational Measurement: Issues and Practice, 29(3), 39-47. https://doi.org/10.1111/j.1745-3992.2010.00182.x
36. Wu, D. A., Li, Z. & Zumbo, B. D. (2007). Decoding the meaning of factorial invariance and updating the practice of multi-group confirmatory factor analysis: A demonstration with TIMSS data. Practical Assessment, Research & Evaluation, 12(3), 1-26.
37. Xu, H., & Tracey, T. J. G. (2017). Use of multi-group confirmatory factor analysis in examining measurement invariance in counseling psychology research. The European Journal of Counselling Psychology, 6(1). 75-82. http://dx.doi.org/10.23668/psycharchives.2041
38. Yeşilyaprak, B. (2007). Eğitimde rehberlik hizmetleri [Counseling services in education]. Ankara: PegemA.