Alternative Confirmatory Factor Analytic Models for Examining Preservice Teachers' Non-Cognitive Skills

Abstract

This study provides a comparative analysis of the inferences that can be reached about preservice teachers' emotional awareness and personality traits when several alternative factor analytic models are tested simultaneously. An empirical illustration is provided using two datasets collected for a research study aiming to profile the social-emotional behavior repertoire of preservice teachers. Four alternative factor analytic models were considered for both datasets: the unidimensional model, the correlated model, the higher- (or second-) order model, and the bifactor model. Results indicate that the higher-order factor model for emotional awareness (n = 670) and the bifactor model for personality traits (n = 670) were the preferred models that better represent the underlying factor structures. Multidimensionality and practical decisions concerning score reporting are discussed.

Keywords

• non-cognitive skills
• teacher training
• factor analytic models
• multidimensionality

References

1. Brown, T. A. (2015). Confirmatory factor analysis for applied research. Guilford.
2. Brunner, M., Nagy, G., & Wilhelm, O. (2012). A tutorial on hierarchically structured constructs. Journal of Personality, 80, 796-846. https://doi.org/10.1111/j.1467-6494.2011.00749.x
3. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105. https://doi.org/10.1037/h0046016
4. Chen, F. F., West, S. G., & Sousa, K. H. (2006). A comparison of Bifactor and Second-order models of quality of life. Multivariate Behavioral Research, 41(2), 189-225. https://doi.org/10.1207/s15327906mbr4102_5
5. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334. https://doi.org/10.1007/BF02310555
6. Dunn, K. J., & McCary, G. (2020). The place of the bifactor model in confirmatory factor analysis investigations into construct dimensionality in Language Testing. Front. Psychol., 1357. https://doi.org/10.3389/fpsyg.2020.01357
7. Eid, M., & Diener, E. (2006). Introduction: The need for multimethod measurement in psychology. In M. E. Diener, Handbook of multimethod measurement in psychology (pp. 3-9). American Psychological Association.
8. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates. Goldberg, L. R. (1992). The development of markers for the Big-Five factor structure. Psychological Assessment, 4(1), 26-42. https://doi.org/10.1037/1040-3590.4.1.26

9. Gustafsson, J. E., & Åberg-Bengtsson, L. (2010). Unidimensionality and the interpretability of psychological instruments. In S. Embretson, Measuring psychological constructs: Advances in model-based approaches (pp. 97-121). American Psychological Association. https://doi.org/10.1037/12074-005
10. Holzinger, K. J., & Swineford, F. (1937). The bi-factor method. Psychometrika, 2, 41-54. https://doi.org/10.1007/BF02287965
11. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55. https://doi.org/10.1080/10705519909540118
12. Inceman-Kara, F., & Yuksel, G. (2022). Focusing on Adolescents' Emotional Awareness: Instrument Validation and Evaluation of a Training Program. Participatory Educational Research, 9(4), 289-306. http://dx.doi.org/10.17275/per.22.91.9.4
13. Jennings, P. A., & Greenberg, M. T. (2009). The prosocial classroom: Teacher social and emotional competence in relation to student and classroom outcomes. Review of Educational Research, 79, 491-525. http://dx.doi.org/10.3102/0034654308325693
14. Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1-73. https://doi.org/10.1111/jedm.12000
15. Markon, K. E. (2019). Bifactor and hierarchical models: specification, inference, and interpretation. Annual Review of Clinical Psychology, 15, 51-69. https://doi.org/10.1146/annurevclinpsy-050718-095522
16. McCarthy, D. (2021). Adding social emotional awareness to teacher education field experiences. The Teacher Educator, 56(3), 287-304. https://doi.org/10.1080/08878730.2021.1890291
17. McDonald, R. P. (1999). Test theory: A unified approach. Erlbaum.
18. Messick, S. (1995). Validity of psychological assessment: validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741-749. https://doi.org/10.1037/0003-066X.50.9.741
19. Morsunbul, (2014). The validity and reliability study of the Turkish version of quick big five personality test. Dusunen Adam, 27(4), 316-322. https://doi.org/10.5350/DAJPN2014270405
20. Muthén, L. K., & Muthén, B. O. (1998-2017). Mplus user’s guide (8th Edition). Muthén & Muthén. Nyugen, N. T., & Biderman, M. (2013). Predicting Counterproductive Work Behavior from a Bi-factor Model of Big Five Personality. Academy of Management Annual Meeting Proceedings. https://doi.org/10.5465/ambpp.2013.13719abstract
21. Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47, 667-696. http://dx.doi.org/101080/00273171.2012.715555
22. Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95, 129-140. http://dx.doi.org/10.1080/00223891.2012.725437
23. Rieffe, C., Oosterveld, P., Miers, A. C., Terwogt, M. M., & & Ly, V. (2008). Emotion awareness and internalising symptoms in children and adolescents: The Emotion Awareness Questionnaire revised. Personality and Individual Differences, 45(8), 756-761. https://doi.org/10.1016/j.paid.2008.08.001
24. Rijmen, F. (2010). Formal Relations and an Empirical Comparison among the Bi-Factor, the Testlet, and a Second-Order Multi-dimensional IRT Model. Journal of Educational Measurement, 47(3), 361-372. https://doi.org/10.1111/j.1745-3984.2010.00118.x
25. Rodriguez, A., Reise, S. P., & Haviland, M. G. (2015). Evaluating bifactor models: calculating and interpreting statistical indices. Psychological Methods, 21(2), 137-150. http://dx.doi.org/10.1037/met0000045
26. Rodriquez, A., Reise, S. P., & Haviland, M. G. (2016). Applying bifactorsStatistical indices in the evaluation of psychological measures. Journal of Personality Assessment, 98(3), 223-237. http://dx.doi.org/10.1080/00223891.2015.1089249
27. Sinharay, S., Puhan, G., & Haberman, S. J. (2011). An NCME instructional module on subscores. Educational Measurement: Issues and Practice, 30, 29-40. https://doi.org/10.1111/j.1745-3992.2011.00208.x
28. Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics. Pearson. Thurstone, L. (1944). Second-order factors. Psychometrika, 9(2), 71-100. http://dx.doi.org/10.1007/bf02288715
29. Ulloa, M., Evans, I., & Jones, L. (2016). The effects of emotional awareness training on teachers' ability to manage the emotions of preschool children: An experimental study. Escritos de Psicología, 9(1), 1-14. http://dx.doi.org/10.5231/psy.writ.2015.1711