Factor extraction in exploratory factor analysis for ordinal indicators: Is principal component analysis the best option?

Abstract

Researchers continue to choose PCA in scale development and adaptation studies because it is the default setting and overestimates measurement quality. When PCA is utilized in investigations, the explained variance and factor loadings can be exaggerated. PCA, in contrast to the models given in the literature, should be investigated in categorical/ordered, severely skewed data, and multidimensional structures. The purpose of this study is to compare the relative bias and percent correct estimation of PCA, PAF, and MINRES techniques with Monte Carlo simulations. In Monte Carlo simulations sample size, level of skewness, number of categories, average factor loadings, number of factors, level of inter-factor correlation and test length were manipulated. The results show that PCA overestimates most models with lower average factor loadings, but PAF and MINRES provide unbiased results even with low factor loadings. PAF and MINRES produce more accurate and impartial results, and it is projected that PCA will lead researchers to believe that the items in scale development or adaptation studies are of "high quality."

Keywords

• PCA
• Factor extraction
• Monte Carlo simulation

References

1. Beauducel, A., & Herzberg, P.Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling: A Multidisciplinary Journal, 13(2), 186 203. https://doi.org/10.1207/s15328007sem1302_2
2. Brown, T.A. (2006). Confirmatory factor analysis for applied research. Guilford Press.
3. Collins, L.M., Schafer, J.L., & Kam, C.-M. (2001). A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods, 6(4), 330–351. https://doi.org/10.1037/1082-989X.6.4.330
4. Comrey, A.L., & Lee, H.B. (1992). A first course in factor analysis (2nd ed.). L. Erlbaum Associates.
5. Costello, A.B., & Osborne, J. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research, and Evaluation, 10(1), 1–9. https://doi.org/10.7275/JYJ1-4868
6. Coughlin, K. (2013). An analysis of factor extraction strategies: A comparison of the relative strengths of principal axis, ordinary least squares, and maximum likelihood in research contexts that include both categorical and continuous variables. https://www.semanticscholar.org/paper/7bbfc6050a24dce49454a56d9af2ad2e6fc40ad2
7. De Winter, J.C.F., & Dodou, D. (2016). Common factor analysis versus principal component analysis: A comparison of loadings by means of simulations. Communications in Statistics - Simulation and Computation, 45(1), 299 321. https://doi.org/10.1080/03610918.2013.862274
8. Fabrigar, L.R., & Wegener, D.T. (2012). Exploratory factor analysis. Oxford University Press. https://doi.org/10.1093/acprof:osobl/9780199734177.001.0001

9. Fabrigar, L.R., Wegener, D.T., Maccallum, R., & Strahan, E. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272–299. https://doi.org/10.1037/1082-989X.4.3.272
10. Flora, D.B., & Curran, P.J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9(4), 466–491. https://doi.org/10.1037/1082-989X.9.4.466
11. Forero, C.G., Maydeu-Olivares, A., & Gallardo-Pujol, D. (2009). Factor analysis with ordinal indicators: A Monte Carlo study comparing DWLS and ULS estimation. Structural Equation Modeling: A Multidisciplinary Journal, 16(4), 625 641. https://doi.org/10.1080/10705510903203573
12. Garson, G.D. (2023). Factor analysis and dimension reduction in R: A social scientist’s toolkit. Routledge, Taylor & Francis Group.
13. Ho, A.D., & Yu, C.C. (2015). Descriptive statistics for modern test score distributions: Skewness, kurtosis, discreteness, and ceiling effects. Educational and Psychological Measurement, 75(3), 365–388. https://doi.org/10.1177/0013164414548576
14. Howard, M.C. (2016). A review of exploratory factor analysis decisions and overview of current practices: What we are doing and how can we improve? International Journal of Human Computer Interaction, 32, 51 62. https://doi.org/10.1080/10447318.2015.1087664
15. Jöreskog, K.G. (2003). Factor analysis by MINRES. https://www.ssicentral.com/wp content/uploads/2021/04/lis_minres.pdf
16. Kaplan, D. (Ed.). (2004). The SAGE handbook of quantitative methodology for the social sciences. SAGE.
17. Kılıç, A.F., & Doğan, N. (2021). Comparison of confirmatory factor analysis estimation methods on mixed-format data. International Journal of Assessment Tools in Education, 8(1), 21–37. https://doi.org/10.21449/ijate.782351
18. Koyuncu, İ., & Kılıç, A.F. (2019). The use of exploratory and confirmatory factor analyses: A document analysis. Eğitim ve Bilim, 44(198), 361 388. https://doi.org/10.15390/EB.2019.7665
19. Li, C.-H. (2016). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods, 48(3), 936–949. https://doi.org/10.3758/s13428-015-0619-7
20. Lozano, L.M., García-Cueto, E., & Muñiz, J. (2008). Effect of the number of response categories on the reliability and validity of rating scales. Methodology, 4(2), 73–79. https://doi.org/10.1027/1614-2241.4.2.73
21. Mabel, O.A., & Olayemi, O.S. (2020). A comparison of principal component analysis, maximum likelihood and the principal axis in factor analysis. American Journal of Mathematics and Statistics, 2(10), 44–54.
22. MacCallum, R.C., & Tucker, L.R. (1991). Representing sources of error in the common-factor model: Implications for theory and practice. Psychological Bulletin, 109(3), Article 3. https://doi.org/10.1037/0033-2909.109.3.502
23. Oranje, A. (2003, April 21). Comparison of estimation methods in factor analysis with categorized variables: Applications to NEAP data [Paper presentation]. Annual Meeting of the National Council on Measurement in Education.
24. Revelle, W. (2024). psych: Procedures for psychological, psychometric, and personality research (Version 2.4.3) [Computer software]. Northwestern University. https://cran.r-project.org/package=psych
25. Rhemtulla, M., Brosseau-Liard, P.É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354–373. https://doi.org/10.1037/a0029315
26. Sigal, M.J., & Chalmers, R.P. (2016). Play it again: Teaching statistics with Monte Carlo simulation. Journal of Statistics Education, 24(3), 136 156. https://doi.org/10.1080/10691898.2016.1246953
27. Snook, S., & Gorsuch, R. (1989). Component analysis versus common factor analysis: A Monte Carlo study. Psychological Bulletin, 106, 148 154. https://doi.org/10.1037/0033 2909.106.1.148
28. Tabachnick, B.G., & Fidell, L.S. (2013). Using multivariate statistics (6th ed., international ed.). Pearson.
29. Watkins, M.W. (2018). Exploratory factor analysis: A guide to best practice. Journal of Black Psychology, 44(3), 219–246. https://doi.org/10.1177/0095798418771807
30. West, S.G., Finch, J.F., & Curran, P.J. (1995). Structural equation models with non-normal variables: Problems and remedies. In R. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 56–75). Sage.
31. Widaman, K. (1993). Common factor analysis versus principal component analysis: Differential bias in representing model parameters? Multivariate Behavioral Research, 28(3), 263–311. https://doi.org/10.1207/s15327906mbr2803_1
32. Zygmont, C., & Smith,M.R. (2014). Robust factor analysis in the presence of normality violations, missing data, and outliers: Empirical questions and possible solutions. The Quantitative Methods for Psychology, 10(1), 40 55. https://doi.org/10.20982/tqmp.10.1.p040