Estimating Internal Consistency: Did We Choose the Right Coefficient?

Abstract

Internal consistency is a concept extensively used in academic discourse, yet its definition remains debated. In the context of validation studies, it is noteworthy that, although internal consistency is commonly assessed, some studies could benefit from employing more accurate estimators that are better suited to the underlying factorial structure. These editorial addresses the various recommended estimators for calculating internal consistency based on the characteristics of the studied model. We explore one-dimensional measures, identifying when estimators such as α are suitable, particularly for tau-equivalent models. For congeneric measurement models, coefficient ω is recommended. We also discuss complex models incorporating multidimensional structures, including essential unidimensionality, scales with multiple correlated or uncorrelated factors, and higher-order models. Researchers should avoid reporting the total internal consistency of the instrument unless unidimensionality or a higher-order factor structure has been demonstrated. When data are approximately unidimensional, measures are congeneric with moderate factor loadings, and sample sizes are large, it is reasonable to report both α and ω.

Keywords

• Internal Consistency
• Reliability
• Psychometrics

İç Tutarlılığın Tahmini: Doğru Katsayıyı Seçtik mi?

Öz

İç tutarlılık, akademik söylemde yaygın olarak kullanılan bir kavramdır ancak tanımı hâlâ tartışmalıdır. Geçerlilik çalışmaları bağlamında, iç tutarlılığın yaygın olarak değerlendirildiği, ancak bazı çalışmaların altta yatan faktör yapısına daha uygun ve daha doğru tahmin edicilerden faydalanabileceği dikkate değerdir. Bu editoryal, incelenen modelin özelliklerine göre iç tutarlılığın hesaplanması için önerilen çeşitli tahmin edicileri ele almaktadır. Tek boyutlu ölçümleri inceliyor, α gibi tahmin edicilerin özellikle tau-eşdeğer modeller için ne zaman uygun olduğunu belirtiyoruz. Kongenerik ölçüm modelleri için ω katsayısı önerilmektedir. Ayrıca, zorunlu tek boyutluluk, birden fazla ilişkili veya ilişkisiz faktöre sahip ölçekler ve üst düzey modeller dahil olmak üzere çok boyutlu yapıları içeren karmaşık modelleri de tartışıyoruz. Araştırmacılar, tek boyutluluk veya üst düzey faktör yapısı gösterilmedikçe aracın toplam iç tutarlılığını raporlamaktan kaçınmalıdır. Veriler yaklaşık olarak tek boyutlu olduğunda, ölçümler orta düzey faktör yüklerine sahip kongenerik ise ve örneklem büyüklüğü büyükse, hem α hem de ω’nun raporlanması makuldür.

Anahtar Kelimeler

• İç Tutarlılık
• Güvenilirlik
• Psikometri

Kaynakça

1. Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238-246. https://doi.org/10.1037/0033-2909.107.2.238
2. Bentler, P. M. (2021). Alpha, FACTT, and beyond. Psychometrika, 86(4), 861-868. https://doi.org/10.1007/s11336-021-09797-8
3. Berge, J. M. F. T. (2014). Tau‐equivalent and congeneric measurements. Wiley StatsRef: Statistics Reference Online. https://doi.org/10.1002/9781118445112.stat06393
4. Bollen, K. A. (1980). Issues in the comparative measurement of political democracy. American Sociological Review, 45(3), 370. https://doi.org/10.2307/2095172
5. Cho, E., & Béland, S. (2025). Reliability in unidimensional ordinal data: A comparison of continuous and ordinal estimators. Psychological Methods. https://doi.org/10.1037/met0000739
6. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334. https://doi.org/10.1007/bf02310555
7. Crutzen, R., & Peters, G. Y. (2015). Scale quality: Alpha is an inadequate estimate and factor-analytic evidence is needed first of all. Health Psychology Review, 11(3), 242-247. https://doi.org/10.1080/17437199.2015.1124240
8. Doval, E., Viladrich, C., & Angulo-Brunet, A. (2023). Coefficient alpha: The resistance of a classic. Psicothema, 35(1), 5-20. https://doi.org/10.7334/psicothema2022.321

9. Flora, D. B. (2020). Your coefficient alpha is probably wrong, but which coefficient omega is right? A tutorial on using r to obtain better reliability estimates. Advances in Methods and Practices in Psychological Science, 3(4), 484-501. https://doi.org/10.1177/2515245920951747
10. Gadermann, A. M., Guhn, M., & Zumbo, B. D. (2012). Estimating ordinal reliability for likert-type and ordinal item response data: A conceptual, empirical, and practical guide. Practical Assessment, Research & Evaluation, 17(3), 1-13. https://doi.org/10.7275/n560-j767
11. Graham, J. M. (2006). Congeneric and (essentially) tau-equivalent estimates of score reliability. Educational and Psychological Measurement, 66(6), 930-944. https://doi.org/10.1177/0013164406288165
12. Green, S. B., & Hershberger, S. L. (2000). Correlated errors in true score models and their effect on coefficient alpha. Structural Equation Modeling A Multidisciplinary Journal, 7(2), 251-270. https://doi.org/10.1207/s15328007sem0702_6
13. Green, S. B., & Yang, Y. (2015). Evaluation of dimensionality in the assessment of internal consistency reliability: Coefficient alpha and omega coefficients. Educational Measurement Issues And Practice, 34(4), 14-20. https://doi.org/10.1111/emip.12100
14. Green, S. B., Lissitz, R. W., & Mulaik, S. A. (1977). Limitations of coefficient alpha as an index of test unidimensionality1. Educational And Psychological Measurement, 37(4), 827-838. https://doi.org/10.1177/001316447703700403
15. Gu, F., Little, T. D., & Kingston, N. M. (2012). Misestimation of reliability using coefficient alpha and structural equation modeling when assumptions of tau-equivalence and uncorrelated errors are violated. Methodology, 9(1), 30-40. https://doi.org/10.1027/1614-2241/a000052
16. Jarvis, C. B., MacKenzie, S. B., & Podsakoff, P. M. (2003). A critical review of construct indicators and measurement model misspecification in marketing and consumer research. Journal of Consumer Research, 30(2), 199-218. https://doi.org/10.1086/376806
17. Jöreskog, K. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36(2), 109-133. https://doi.org/10.1007/bf02291393
18. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Addison-Wesley.
19. McDonald, R. P. (1981). The dimensionality of tests and items. British Journal of Mathematical and Statistical Psychology, 34(1), 100-117. https://doi.org/10.1111/j.2044-8317.1981.tb00621.x
20. Miller, M. B. (1995). Coefficient alpha: A basic introduction from the perspectives of classical test theory and structural equation modeling. Structural Equation Modeling A Multidisciplinary Journal, 2(3), 255-273. https://doi.org/10.1080/10705519509540013
21. Mokkink, L. B., Elsman, E. B., & Terwee, C. B. (2024). COSMIN guideline for systematic reviews of patient-reported outcome measures version 2.0. Quality of Life Research, 33(11), 2929-2939. https://doi.org/10.1007/s11136-024-03761-6
22. Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., Bouter, L. M., & de Vet, H. C. (2010). The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. Journal of Clinical Epidemiology, 63(7), 737-745. https://doi.org/10.1016/j.jclinepi.2010.02.006
23. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
24. Prinsen, C. A. C., Mokkink, L. B., Bouter, L. M., Alonso, J., Patrick, D. L., De Vet, H. C. W., & Terwee, C. B. (2018). COSMIN guideline for systematic reviews of patient-reported outcome measures. Quality of Life Research, 27(5), 1147-1157. https://doi.org/10.1007/s11136-018-1798-3
25. Raykov, T. (1997). Estimation of composite reliability for congeneric measures. Applied Psychological Measurement, 21(2), 173-184. https://doi.org/10.1177/01466216970212006
26. Raykov, T. (2004). Point and interval estimation of reliability for multiple-component measuring instruments via linear constraint covariance structure modeling. Structural Equation Modeling A Multidisciplinary Journal, 11(3), 342-356. https://doi.org/10.1207/s15328007sem1103_3
27. Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2012a). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95(2), 129-140. https://doi.org/10.1080/00223891.2012.725437
28. Reise, S. P. (2012b). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667-696. https://doi.org/10.1080/00273171.2012.715555
29. Revelle, W. (1979). Hierarchical cluster analysis and the internal structure of tests. Multivariate Behavioral Research, 14(1), 57-74. https://doi.org/10.1207/s15327906mbr1401_4
30. Revelle, W., & Condon, D. M. (2019). Reliability from α to ω: A tutorial. Psychological Assessment, 31(12), 1395-1411. https://doi.org/10.1037/pas0000754
31. Rodriguez, A., Reise, S. P., & Haviland, M. G. (2015). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137-150. https://doi.org/10.1037/met0000045
32. Sijtsma, K. (2008). On the use, the misuse, and the very limited usefulness of cronbach’s alpha. Psychometrika, 74(1), 107-120. https://doi.org/10.1007/s11336-008-9101-0
33. Sijtsma, K., & Pfadt, J. M. (2021). Part II: On the use, the misuse, and the very limited usefulness of cronbach’s alpha: Discussing lower bounds and correlated errors. Psychometrika, 86(4), 843-860. https://doi.org/10.1007/s11336-021-09789-8
34. Stout, W. (1987). A nonparametric approach for assessing latent trait unidimensionality. Psychometrika, 52(4), 589-617. https://doi.org/10.1007/bf02294821
35. Streiner, D. L. (2003). Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80(1), 99-103. https://doi.org/10.1207/s15327752jpa8001_18
36. Tang, W., Cui, Y., & Babenko, O. (2014). Internal consistency: Do we really know what it is and how to assess it? Journal of Psychology & Behavioral Science, 2(2), 205–220. https://jpbs.thebhprj.org/journals/jpbs/Vol_2_No_2_June_2014/13.pdf
37. Thorndike, R. M. (1995). Book review: Psychometric Theory (3rd ed.) by Jum Nunnally and Ira Bernstein New York: McGraw-Hill, 1994, xxiv + 752 pp. Applied Psychological Measurement, 19(3), 303-305. https://doi.org/10.1177/014662169501900308
38. Viladrich, C., Angulo-Brunet, A., & Doval, E. (2017). Un viaje alrededor de alfa y omega para estimar la fiabilidad de consistencia interna. Anales de Psicología, 33(3), 755 –782. https://doi.org/10.6018/analesps.33.3.268401
39. Yung, Y., Thissen, D., & McLeod, L. D. (1999). On the relationship between the higher-order factor model and the hierarchical factor model. Psychometrika, 64(2), 113-128. https://doi.org/10.1007/bf02294531
40. Zhang, Z., & Yuan, K. (2015). Robust coefficients alpha and omega and confidence intervals with outlying observations and missing data. Educational And Psychological Measurement, 76(3), 387-411. https://doi.org/10.1177/0013164415594658
41. Zinbarg, R. E., Revelle, W., Yovel, I., & Li, W. (2005). Cronbach’s α, revelle’s β, and mcdonald’s ωh: their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70(1), 123-133. https://doi.org/10.1007/s11336-003-0974-7
42. Zumbo, B. D., & Kroc, E. (2019). A measurement is a choice and stevens’ scales of measurement do not help make it: A response to chalmers. Educational And Psychological Measurement, 79(6), 1184-1197. https://doi.org/10.1177/0013164419844305