ÇARPIK DAĞILIMLARDA FAKTÖR SAYISI BELİRLEME YÖNTEMLERİNİN PERFORMANSLARININ İNCELENMESİ

Year 2023, Volume: 5, 288 - 312, 20.10.2023

Gül Güler Abdullah Faruk Kılıç

https://doi.org/10.48166/ejaes.1357828

Abstract

Bu araştırmanın amacı faktör sayısı belirleme yöntemlerinin çeşitli simülasyon koşulları altında performanslarını değerlendirmektir. Bu amaç doğrultusunda boyutluluk belirleme yöntemlerinden optimal paralel analiz, MAP, HULL, EGA (TMFG) kestirimi, EGA (Glasso) kestirimi ve comparison data forest yöntemi karşılaştırılmıştır. Çalışmada simülasyon koşulları olarak dağılımın türü, örneklem büyüklüğü, faktör başına düşen madde sayısı, kategori sayısı ve ölçme modeli belirlenmiştir. Çalışmada her bir koşul için 100 replikasyon yapılmıştır. Çalışmada tamamen çaprazlanmış simülasyon deseni kullanılmıştır. Çarpık dağılımlarda faktör sayısı belirleme yöntemlerinin performanslarının incelendiği bu çalışma sonucunda tüm koşulların accuracy değerlerinin ortalaması dikkate alındığında en yüksek ortalamaya HULL yönteminin sahip olduğu görülmüştür. Aynı zamanda en düşük RB ortalaması da HULL yöntemindedir. Ancak tüm koşullarda yeterli performansı gösteren bir yöntemin olmadığı söylenebilir. Diğer bir deyişle her koşulda doğru sonucu verecek bir yöntem bulunmamaktadır. Bu çalışmada tek faktörlü, faktörler arası korelasyonu 0.00 ve 0.30 olan iki faktörlü yapılar incelenmiştir. Eğitimde ve psikolojide ikiden fazla faktör sayısına sahip yapılar göz önünde bulundurulduğunda gelecekteki araştırmalarda çarpık dağılım gösteren verilerde daha fazla faktör ve madde sayısıyla çalışılarak yöntemlerin performansları karşılaştırılabilir.

Keywords

Faktör sayısı belirleme, MAP, HULL, karşılaştırmalı faktör forest, EGA

PERFORMANCE OF FACTOR RETENTION METHODS IN SKEWED DISTRIBUTIONS

Abstract

This research aims to evaluate the performance of dimensionality determination methods under various simulation conditions. Therefore, dimensionality determination methods were compared, including optimal parallel analysis, MAP, HULL, EGA (TMFG) estimation, EGA (glasso) estimation, and comparison data forest method. The type of distribution, sample size, number of items per factor, number of categories, and measurement model were specified as simulation conditions in the study. For each condition, 100 replications were conducted. A fully crossed simulation design was employed in the study. The results of this study, which examined the performance of factor determination methods under skewed distributions, indicated that the HULL method had the highest average considering the average accuracy values of all conditions. Meanwhile, the HULL method had the lowest RB average. However, no method demonstrated adequate performance under all conditions. This study examined one-factor and two-factor structures with interfactor correlations of 0.00 and 0.30. Considering structures with more than two factors in education and psychology, future research could focus on working with data exhibiting skewed distributions involving more factors and items to compare the performance of methods.

Keywords

factor retention, MAP, HULL, comparison factor forest, EGA

References

• Beauducel, A., & Herzberg, P. Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling: A Multidisciplinary Journal, 13(2), 186–203. https://doi.org/10.1207/s15328007sem1302_2
• Cho, S.-J., Li, F., & Bandalos, D. L. (2009). Accuracy of the parallel analysis procedure with polychoric correlations. Educational and Psychological Measurement, 69(5), 748–759. https://doi.org/10.1177/0013164409332229
• Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10(7), 27–29. https://doi.org/10.7275/jyj1-4868
• Curran, P. J., West, S. G., & Finch, J. F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1(1), 16–29. https://doi.org/10.1037/1082-989X.1.1.16
• Finch, W. H. (2020). Using Fit Statistic Differences to Determine the Optimal Number of Factors to Retain in an Exploratory Factor Analysis. Educational and Psychological Measurement, 80(2), 217–241. https://doi.org/10.1177/0013164419865769.
• Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9(4), 466–491. https://doi.org/10.1037/1082-989X.9.4.466
• Foldnes, N., & Grønneberg, S. (2017). The asymptotic covariance matrix and its use in simulation studies. Structural Equation Modeling: A Multidisciplinary Journal, 24(6), 881–896. https://doi.org/10.1080/10705511.2017.1341320
• Forero, C. G., Maydeu-Olivares, A., & Gallardo-Pujol, D. (2009). Factor analysis with ordinal indicators: A monte carlo study comparing DWLS and ULS estimation. Structural Equation Modeling: A Multidisciplinary Journal, 16(4), 625–641. https://doi.org/10.1080/10705510903203573
• Garrido, L. E., Abad, F. J., & Ponsoda, V. (2011). Performance of Velicer’s minimum average partial factor retention method with categorical variables. Educational and Psychological Measurement, 71(3), 551–570. https://doi.org/10.1177/0013164410389489
• Golino, H. F., & Christensen, A. P. (2020). EGAnet: Exploratory Graph Analysis – A framework for estimating the number of dimensions in multivariate data using network psychometrics.
• Golino, H. F., & Epskamp, S. (2017). Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research. PLOS ONE, 12(6), 1–26. https://doi.org/10.1371/journal.pone.0174035.
• Goretzko, D., & Bühner, M. (2020). One model to rule them all? Using machine learning algorithms to determine the number of factors in exploratory factor analysis. Psychological Methods, 25(6), 776–786.
• Goretzko, D., & Bühner, M. (2022). Factor retention using machine learning with ordinal data. Applied Psychological Measurement, 46(5), 406–421. https://doi.org/10.1177/01466216221089345
• Goretzko, D., Pham, T. T. H., & Bühner, M. (2021). Exploratory factor analysis: Current use, methodological developments and recommendations for good practice. Current Psychology, 40(7), 3510–3521. https://doi.org/10.1007/s12144-019-00300-2
• Goretzko, D., & Ruscio, J. (2023). The comparison data forest: A new comparison data approach to determine the number of factors in exploratory factor analysis. Behavior Research Methods. https://doi.org/10.3758/s13428-023-02122-4
• Gorsuch, R. L. (1974). Factor analysis. W. B. Saunders.
• Haslbeck, J. M. B., & van Bork, R. (2022). Estimating the number of factors in exploratory factor analysis via out-of-sample prediction errors. Psychological methods, 10.1037/met0000528. Advance online publication. https://doi.org/10.1037/met0000528.
• Henson, R., & Roberts, J. (2006). Use of exploratory factor analysis in published research: Common errors and some comment on improved practice. Educational and Psychological Measurement, 66(3), 393–416.
• Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179–185. https://doi.org/10.1007/BF02289447
• Kılıç, A. F., & Uysal, İ. (2019). Comparison of factor retention methods on binary data: A simulation study. Turkish Journal of Education, 8(3), 160–179. https://doi.org/10.19128/turje.518636
• Koyuncu, İ., & Kılıç, A. F. (2019). The use of exploratory and confirmatory factor analyses: A document analysis. Eğitim ve Bilim, 44(198), 361–388. https://doi.org/10.15390/EB.2019.7665
• Kılıç, A. F., & Uysal, İ. (2021). Faktör çıkarma yöntemlerinin paralel analiz sonuçlarına etkisi. Trakya Eğitim Dergisi, 11(2), 926–942. https://doi.org/10.24315/tred.747075.
• Montoya, A. K., & Edwards, M. C. (2021). The poor fit of model fit for selecting number of factors in exploratory factor analysis for scale evaluation. Educational and Psychological Measurement, 81(3), 413–440. https://doi.org/10.1177/0013164420942899.
• Ledesma, R. D., Valero-Mora, P., & Macbeth, G. (2015). The Scree Test and the Number of Factors: a Dynamic Graphics Approach. The Spanish journal of psychology, 18, E11. https://doi.org/10.1017/sjp.2015.13.
• Lee, C., Park Y., & Cho B. (2023). Use of exploratory graph analysis in inspecting the dimensionality of the revised University of California Los Angeles (R-UCLA) Loneliness Scale Among Older Adults. Res Gerontol Nurs.16(1):15-20. https://doi.org/10.3928/19404921-20230104-03.
• Li, C.-H. (2016). The performance of ML, DWLS, and ULS estimation with robust corrections in structural equation models with ordinal variables. Psychological Methods, 21(3), 369–387. https://doi.org/10.1037/met0000093
• Li, Y., Wen, Z., Hau, K.-T., Yuan, K.-H., & Peng, Y. (2020). Effects of cross-loadings on determining the number of factors to retain. Structural Equation Modeling: A Multidisciplinary Journal, 27(6), 841–863. https://doi.org/10.1080/10705511.2020.1745075
• Lorenzo-Seva, U., Timmerman, M. E., & Kiers, H. A. L. (2011). The Hull method for selecting the number of common factors. Multivariate Behavioral Research, 46(2), 340–364. https://doi.org/10.1080/00273171.2011.56452
• Lozano, L. M., García-Cueto, E., & Muñiz, J. (2008). Effect of the number of response categories on the reliability and validity of rating scales. Methodology, 4(2), 73–79. https://doi.org/10.1027/1614-2241.4.2.73
• Montoya, A. K., & Edwards, M. C. (2021). The Poor Fit of Model Fit for Selecting Number of Factors in Exploratory Factor Analysis for Scale Evaluation. Educational and Psychological Measurement, 81(3), 413–440. https://doi.org/10.1177/0013164420942899
• Muthén, B. O., & Kaplan, D. (1985). A comparison of some methodologies for the factor analysis of non-normal Likert variables. British Journal of Mathematical and Statistical Psychology, 38(2), 171–189. https://doi.org/10.1111/j.2044-8317.1985.tb00832.x
• Navarro-Gonzalez, D., & Lorenzo-Seva, U. (2021). EFA.MRFA: Dimensionality assessment using minimum rank factor analysis. https://cran.r-project.org/package=EFA.MRFA
• O’Connor, B. P. (2022). EFA.dimensions: Exploratory factor analysis functions for assessing dimensionality. https://cran.r-project.org/package=EFA.dimensions
• R Core Team. (2022). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. https://www.r-project.org
• Reio, T. G., & Shuck, B. (2015). Exploratory Factor Analysis: Implications for Theory, Research, and Practice. Advances in Developing Human Resources, 17(1), 12–25. https://doi.org/10.1177/1523422314559804
• Rosseel, Y. (2012). lavaan: An R Package for Structural Equation Modeling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02
• Ruscio, J., & Roche, B. (2012). Determining the number of factors to retain in an exploratory factor analysis using comparison data of known factorial structure. Psychological Assessment, 24(2), 282–292. https://doi.org/10.1037/a0025697
• Schmitt, T. A., & Sass, D. A. (2011). Rotation Criteria and Hypothesis Testing for Exploratory Factor Analysis: Implications for Factor Pattern Loadings and Interfactor Correlations. Educational and Psychological Measurement, 71(1), 95–113. https://doi.org/10.1177/0013164410387348
• Svetina, D. (2011). Assessing dimensionality in complex data structures: A performance comparison of DETECT and NOHARM procedures. Yayınlanmamış doktora tezi, Arizona State Üniversitesi, Arizona.
• Tabachnick, B. G., & Fidell, L. S. (2019). Using multivariate statistics (7th ed.). Pearson.
• Timmerman, M. E., & Lorenzo-Seva, U. (2011). Dimensionality assessment of ordered polytomous items with parallel analysis. Psychological Methods, 16(2), 209–220. https://doi.org/10.1037/a0023353
• West, S. G., Finch, J. F., & Curran, P. J. (1995). Structural equation models with non-normal variables: Problems and remedies. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications. Sage.
• Yang, Y., & Xia, Y. (2015). On the number of factors to retain in exploratory factor analysis for ordered categorical data. Behavior Research Methods, 47(3), 756–772. https://doi.org/10.3758/s13428-014-0499-2