How and When to Use Which Fit Indices? A Practical and Critical Review of the Methodology

Abstract

With the help of statistical software programs, such as AMOS, Lisrel, R, Matlab, and many equivalents, most of the complicated research models have become more computable and easily understandable. Even the most complicated and complex models with various relationships can be easily computed with the help of software. Although with slight differences, outputs are consistent, and tables are mostly comprehensible. However, with the increasing curiosity and amount of knowledge about the research methodology, these simple looking outputs start to become more complicated and deeper. Even though aforementioned statements seem contradictory, what we imply here is very sound to a midlevel researcher because, as knowledge and understanding of statistics deepens, questions and doubts about from where, how, and why these numbers are calculated increase. Curiosity about the fit indices, chi-square and degrees of freedom, modification indices, covariances, and residuals begin to arouse. In this review and commentary, we focus on the infamous CMIN (or chi-square), different model definitions, and calculation of fit indices by the help of these models while avoiding statistical jargon as much as possible. With the aim of putting an end to a decade long debate, when and how to use which fit indices, what they really indicate, and which numbers refer to good or bad fit is also discussed.

Keywords

• Fit indices
• Confirmatory Factor Analysis
• SEM
• Chi-Square
• Discrepancy Functions
• Modification Indices
• Maximum Likelihood

References

1. Akaike, H. (1987). Factor analysis and AIC. In Selected papers of Hirotugu Akaike (pp. 371-386). Springer, New York, NY.
2. Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. Testing structural equation models, 154, 136.
3. Burnham, K. P., & Anderson, D. R. (2001). Kullback-Leibler information as a basis for strong inference in ecological studies. Wildlife research, 28(2), 111-119.
4. Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: the issue of partial measurement invariance. Psychological bulletin, 105(3), 456.
5. Cochran, W. G. (1952). The χ2 test of goodness of fit. The Annals of Mathematical Statistics, 315-345.
6. Gulliksen, H., & Tukey, J. W. (1958). Reliability for the law of comparative judgment. Psychometrika, 23(2), 95-110.
7. James, L. R., Mulaik, S. A., & Brett, J. (1982). Causalanalysis: Models, assumptions and data. Beverly Hills, CA: Sage
8. Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34(2), 200. Kenny, D. A. (2015). Measuring model fit. (http://davidakenny.net/cm/fit.htm)

9. Mulaik, S.A., James, L.R., Van Alstine, J., Bennet, N., Lind, S., and Stilwell, C.D. (1989), "Evaluation of Goodness-of-Fit Indices for Structural Equation Models," Psychological Bulletin, 105 (3), 430-45.
10. Newsom, J. T. (2018). Minimum sample size recommendations (Psy 523/623 structural equation modeling, Spring 2018). Manuscript Retrieved from upa.pdx.edu/IOA/newsom/semrefs.htm.
11. Psutka, J. V., & Psutka, J. (2015, September). Sample size for maximum likelihood estimates of Gaussian model. In International Conference on Computer Analysis of Images and Patterns (pp. 462-469). Springer, Cham.
12. Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of psychological research online, 8(2), 23-74.
13. Schwab, A., & Starbuck, W. H. (2013). Why Baseline Modelling is Better than Null-Hypothesis Testing: Examples from International Business Research. Philosophy of Science and Meta-Knowledge in International Business and Management, 171.
14. Steiger, J. H., & Lind, J. (1980). Paper presented at the annual meeting of the Psychometric Society. Statistically-based tests for the number of common factors.
15. Steiger, J. H., Shapiro, A., & Browne, M. W. (1985). On the multivariate asymptotic distribution of sequential chi-square statistics. Psychometrika, 50(3), 253-263.
16. Templin, J. (2015). Maximum Likelihood Estimation; Robust Maximum Likelihood; Missing Data with Maximum Likelihood [PowerPoint slides]. Retrieved from https://jonathantemplin.com/files/sem/sem15pre906/sem15pre906_lecture03.pdf on 9-10-2019.
17. Wheaton, B., Muthen, B., Alwin, D. F., & Summers, G. F. (1977). Assessing reliability and stability in panel models. Sociological methodology, 8, 84-136.