Comparison of cronbach’s alpha and McDonald’s omega for ordinal data: Are they different?

Year 2023, Volume: 10 Issue: 4, 709 - 722, 23.12.2023

Fatih Orçan

https://doi.org/10.21449/ijate.1271693

Cited By: 4

Abstract

Among all, Cronbach’s Alpha and McDonald’s Omega are commonly used for reliability estimations. The alpha uses inter-item correlations while omega is based on a factor analysis result. This study uses simulated ordinal data sets to test whether the alpha and omega produce different estimates. Their performances were compared according to the sample size, number of items, and deviance from tau equivalence. Based on the result, the alpha and omega had similar results, except for the small sample size, the smaller number of items, and the low factor loading values. When there were 5 or more items in the scale and factor analysis which the omega was calculated from showed fit to the data set, using omega over alpha could be preferred. Also, as the number of items exceeds 5, the alpha and omega differences disappear. Since calculating the alpha is easier compared to the omega (omega requires fitting a factor model first) using alpha over omega can also be suggested. However, when the number of items and the correlations among the items were small, omega performed worse than alpha. Therefore, alpha should be used for the reliability estimations.

Keywords

Cronbach’s Alpha, McDonald’s Omega, Reliability, MonteCarloSEM, Simulation

References

• Bandalos, D.L. (2002). The effects of item parceling on goodness-of-fit and parameter estimate bias in structural equation modeling. Structural Equation Modeling, 9(1), 78–102. https://doi.org/10.1207/S15328007SEM0901_5
• Bernardi, R.A. (1994). Validating research results when cronbach’s alpha is below .70: A methodological procedure. Educational and Psychological Measurement, 54(3), 766–775. https://doi.org/10.1177/0013164494054003023
• Cortina, J.M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98–104. https://doi.org/10.1037/0021-9010.78.1.98
• Edwards, A.A., Joyner, K.J., & Schatschneider, C. (2021). A simulation study on the performance of different reliability estimation methods. Educational and Psychological Measurement, 81(6), 1089 1117. https://doi.org/10.1177/0013164421994184
• Ercan, I., Yazici, B., Sigirli, D., Ediz, B., & Kan, I. (2007). Examining cronbach alpha, theta, omega reliability coefficients according to sample size. Journal of Modern Applied Statistical Methods, 6(1), 291-303. https://doi.org/10.22237/jmasm/1177993560
• Gagne, P., & Hancock, G.R. (2006). Measurement model quality, sample size, and solution propriety in confirmatory factor models. Multivariate Behavioral Research, 41(1), 65-83. https://doi.org/10.1207/s15327906mbr4101_5
• Goodboy, A.K., & Martin, M.M. (2020). Omega over alpha for reliability estimation of unidimensional communication measures. Annals of the International Communication Association, 44(4), 422-439. https://doi.org/10.1080/23808985.2020.1846135
• Henson, R.K., Kogan, L.R., & Vacha-Haase, T. (2001). A reliability generalization study of the teacher efficacy scale and related instruments. Educational and Psychological Measurement, 61(3), 404-420. https://doi.org/10.1177/00131640121971284
• Hu, L., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. https://doi.org/10.1080/10705519909540118
• Kalkbrenner, M.T. (2023). Alpha, omega, and h internal consistency reliability estimates: Reviewing these options and when to use them. Counseling Outcome Research and Evaluation, 14(1), 77-88. https://doi.org/10.1080/21501378.2021.1940118
• McDonald, R.P. (2011). Test theory: A unified treatment. Routlege.
• Orçan, F. (2021). MonteCarloSEM: An R package to simulate data for SEM. International Journal of Assessment Tools in Education, 8(3), 704 713. https://dergipark.org.tr/en/pub/ijate/issue/62753/804203
• R Core Team. (2014). R: A language and environment for statistical computing [Computer software manual]. http://www.R-project.org/
• Raykov, T., & Marcoulides, G.A. (2015). A direct latent variable modeling based method for point and interval estimation of coefficient alpha. Educational and Psychological Measurement, 75(1), 146–156. https://doi.org/10.1177/0013164414526039
• Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1-36. https://doi.org/10.18637/jss.v048.i02
• Signorell, A. (2023). DescTools: Tools for descriptive statistics. R package version 0.99.48. https://CRAN.R-project.org/package=DescTools
• Streiner, D.L. (2003). Starting at the Beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80(1), 99 103. https://doi.org/10.1207/S15327752JPA8001_18
• Vaske, J.J., Beaman, J., & Sponarski, C.C. (2017). Rethinking internal consistency in cronbach's alpha. Leisure Sciences, 39(2), 163 173. https://doi.org/10.1080/01490400.2015.1127189
• Viladrich, C., Angulo-Brunet, A., & Doval, E. (2017). A journey around alpha and omega to estimate internal consistency reliability. Anales de Psicología / Annals of Psychology, 33(3), 755–782. https://doi.org/10.6018/analesps.33.3.268401
• Yurdagül, H., (2008). Minimum sample size for cronbach’s coefficient alpha: A Monte-Carlo Study. H.U. Journal of Education, 35, 397 405. http://www.efdergi.hacettepe.edu.tr/shw_artcl-571.html