Investigating the Performance of the Exploratory Graph Analysis When the Data Are Unidimensional and Polytomous

Abstract

The question of how observable variables should be associated with latent structures has been at the center of the area of psychometrics. A recently proposed alternative model to the traditional factor retention methods is called Exploratory Graph Analysis (EGA). This method belongs to the broader family of network psychometrics which assumes that the associations between observed variables are caused by a system in which variables have direct and potentially causal interaction. This method approaches the psychological data in an exploratory manner and enables the visualization of the relationships between variables and allocation of variables to the dimensions in a deterministic manner. In this regard, the aim of this study was set as comparing the EGA with traditional factor retention methods when the data is unidimensional and items are constructed with polytomous response format. For this investigation, simulated data sets were used and three different conditions were manipulated: the sample size (250, 500, 1000 and 3000), the number of items (5, 10, 20) and internal consistency of the scale (α = 0.7 and α = 0.9). The results revealed that EGA is a robust method especially when used with graphical least absolute shrinkage and selection operator (GLASSO) algorithm and provides better performance in the retention of a true number of dimension than Kaiser's rule and yields comparable results with the other traditional factor retention methods (optimal coordinates, acceleration factor and Horn's parallel analysis) under some conditions. These results were discussed based on the existing literature and some suggestions were given for future studies.

Keywords

• exploratory graph analysis
• factor analysis
• network psychometrics

References

1. Barabási, A.-L., & Pósfai, M. (2016). Network science. Cambridge: Cambridge University.
2. Beard, C., Millner, A. J., Forgeard, M. J., Fried, E. I., Hsu, K. J., Treadway, M. T., Leonard, C. V., Kertz, S. J., & Björgvinsson, T. (2016). Network analysis of depression and anxiety symptom relationships in a psychiatric sample. Psychological medicine, 46(16), 3359–3369. doi: 10.1017/S0033291716002300
3. Borsboom, D., & Cramer, A. O. J. (2013). Network analysis: An integrative approach to the structure of psychopathology. Annual Review of Clinical Psychology, 10(9), 91-121. doi: 10.1146/annurev-clinpsy-050212-185608
4. Borsboom, D., & Molenaar, D. (2015). Psychometrics. In J. Wright (Ed.). International encyclopedia of the social & behavioral sciences (Second ed.. Vol. 19. pp. 418-422). Amsterdam: Elsevier.
5. Cartwright, D., & Harary, F. (1956). Structural balance: A generalization of Heider's theory. Psychological Review, 63(5), 277–293. doi: 10.1037/h0046049
6. Cattell R. B. (1966). The scree test for the number of factors. Multivariate behavioral research, 1(2), 245–276. doi: 10.1207/s15327906mbr0102_10
7. Cattell, R. B. (1978). The scientific use of factor analysis. New York: Plenum.
8. Chalmers, R.P. (2012). mirt: A Multidimensional Item Response Theory Package for the R Environment. Journal of Statistical Software, 48(6), 1-29. doi: 10.18637/jss.v048.i06

9. Chen, J., & Chen, Z. (2008). Extended bayesian information criteria for model selection with large model spaces. Biometrika, 95(3), 759-771. doi: 10.1093/biomet/asn034
10. Cliff, N. (1988). The eigenvalue-greater-than-one rule and the reliability of components. Psychological Bulletin, 103(2), 276-279.
11. Cohen J, (1988). Statistical power analysis for the behavior science. Lawrance Eribaum Association.
12. de Nooy, W., Mrvar, A., & Batagelj, V. (2011). Exploratory social network analysis with Pajek. Cambridge: Cambridge University.
13. DeVellis, R. F. (2017). Scale development: Theory and applications. Thousand Oaks. CA: SAGE Publications.
14. Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The Satisfaction with Life Scale. Journal of Personality Assessment, 49(1), 71-75. doi: 10.1207/s15327752jpa4901_13
15. DiFranza, J. R., Savageau, J. A., Rigotti, N. A., Fletcher, K., Ockene, J. K., McNeill, A. D., Coleman, M., & Wood, C. (2002). Development of symptoms of tobacco dependence in youths: 30 month follow up data from the DANDY study. Tobacco control, 11(3), 228–235. doi: 10.1136/tc.11.3.228
16. Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of α estimation. British Journal of Psychology, 105(3), 399-412. doi: 10.1111/bjop.12046
17. Eaton, N. R. (2015). Latent variable and network models of comorbidity: toward an empirically derived nosology. Social Psychiatry and Psychiatric Epidemiology, 50(6), 845-849. doi: 10.1007/s00127-015-1012-7
18. Edwards, J.R., & Bagozzi, R.P. (2000). On the nature and direction of relationships between constructs and measures. Psychological Methods, 5(2), 155-174. doi: 10.1037/1082-989X.5.2.155.
19. Epskamp, S., & Fried, E. I. (2016). A primer on estimating regularized psychological networks arXiv preprint Stat-Ap/1607.01367. Available at: http://arxiv.org/abs/1607.013677
20. Epskamp, S., Borsboom, D., & Fried, E. I. (2018). Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods, 50(1), 195-212. doi: 10.3758/s13428-017-0862-1
21. Epskamp, S., Maris, G., Waldorp, L.J., & Borsboom, D. (2015). Network Psychometrics. In Irwing. P., Hughes, D. and Booth, T. (Eds.). Handbook of Psychometrics. New York: Wiley.
22. Fisher, A. J., Reeves, J. W., Lawyer, G., Medaglia, J. D., & Rubel, J. A. (2017). Exploring the idiographic dynamics of mood and anxiety via network analysis. Journal of abnormal psychology, 126(8), 1044–1056. doi: 10.1037/abn0000311
23. Foygel, R. and Drton, M. (2010). Extended Bayesian information criteria for Gaussian graphical models. Advances in Neural Information Processing Systems, 23, 2020-2028.
24. Friedman, J., Hastie, T., & Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics (Oxford, England), 9(3), 432-441. doi: 10.1093/biostatistics/kxm045
25. Garrido, L. E., Abad, F. J., & Ponsoda, V. (2016). Are fit indices really fit to estimate the number of factors with categorical variables? Some cautionary findings via Monte Carlo simulation. Psychological Methods, 21(1). 93–111. doi: 10.1037/met0000064
26. Golino, H. F., & Epskamp, S. (2017). Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research. PloS One, 12(6), e0174035. doi: 10.1371/journal.pone.0174035
27. Golino, H., Shi. D., Christensen, A. P., Garrido, L. E.. Nieto, M. D., Sadana, R., Thiyagarajan. J. A., & Martinez-Molina, A. (2020). Investigating the performance of exploratory graph analysis and traditional techniques to identify the number of latent factors: A simulation and tutorial. Psychological Methods. Advance online publication. doi: 10.1037/met0000255
28. Golino, H.. & Christensen, A. P. (2020). EGAnet: Exploratory Graph Analysis -- A framework for estimating the number of dimensions in multivariate data using network psychometrics. R package version 0.9.4.
29. Gorsuch R.L. (1988) Exploratory Factor Analysis. In: Nesselroade J.R., Cattell R.B. (eds) Handbook of Multivariate Experimental Psychology. Perspectives on Individual Differences. Springer, Boston, MA. https://doi.org/10.1007/978-1-4613-0893-5_6
30. Guttman, L. (1954). Some necessary conditions for common-factor analysis. Psychometrika, 19(2), 149-161. doi: 10.1007/BF02289162
31. Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179-185. doi: 10.1007/BF02289447
32. Kline, P. (2014). An easy guide to factor analysis. Routledge.
33. Lance, C. E., Butts, M. M., & Michels, L. C. (2006). The sources of four commonly reported cutoff criteria: What did they really say? Organizational Research Methods, 9(2), 202–220. doi: 10.1177/1094428105284919
34. Lauritzen, S. L. (1996). Graphical Models. Oxford Statistical Science Series. volume 17. New York. NY: Oxford University Press.
35. Massara, G. P., Di Matteo, T., & Aste, T. (2016). Network filtering for big data: Triangulated Maximally Filtered Graph. Journal of Complex Networks, 5, 161–178. doi: 10.1093/comnet/cnw015
36. Nunnally. J. C. (1978). Psychometric theory. New York: McGraw-Hill.
37. Pearl, J. (2000). Causality: Models, reasoning, and inference. New York: Cambridge University.
38. Pons, P.. & Latapy, M. (2006). Computing communities in large networks using random walks. Journal of Graph Algorithms Applications, 10(2), 191-218. doi: 10.1007/11569596_31
39. R Core Team (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing. Vienna. Austria. URL https://www.R-project.org/.
40. Radloff, L.S. (1977). The CES–D Scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1(3), 385-401. doi: 10.1177/014662167700100306
41. Raiche, G. (2010). nFactors: An R package for parallel analysis and non graphical solutions to the Cattell's scree test. R package version 2.3.3.
42. Raiche, G., Riopel. M. and Blais, J.-G. (2006). Non graphical solutions for the Cattell’s scree test. Paper presented at the International Annual Meeting of the Psychometric Society, Montreal.
43. Revelle, W., & Rocklin, T. (1979). Very simple structure: An alternative procedure for estimating the optimal number of interpretable factors. Multivariate Behavioral Research, 14(4), 403-414. doi: 10.1207/s15327906mbr1404_2
44. Ruscio, J., & Roche, B. (2012). Determining the number of factors to retain in an exploratory factor analysis using comparison data of known factorial structure. Psychological Assessment, 24(2), 282-292. doi: 10.1037/a0025697
45. Schmittmann, V. D., Cramer, A. O. J., Waldorp, L. J., Epskamp, S., Kievit, R. A., & Borsboom, D. (2013). Deconstructing the construct: A network perspective on psychological phenomena. New Ideas in Psychology, 31(1), 43-53. doi: 10.1016/j.newideapsych.2011.02.007.
46. Velicer, W. F. (1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 41(3), 321-327. doi: doi.org/10.1007/BF02293557
47. Velicer, W. F., Eaton, C. A., & Fava, J. L. (2000). Construct explication through factor or component analysis: A review and evaluation of alternative procedures for determining the number of factors or components. In R. D. Goffin & E. Helmes (Eds.). Problems and solutions in human assessment: Honoring Douglas N. Jackson at seventy (p. 41-71). Kluwer Academic/Plenum Publishers. doi: 10.1007/978-1-4615-4397-8_3