The Effects of Sample Size, Correlation Technique, and Factor Extraction Method on Reliability Coefficients

Nuri Doğan; Abdullah Faruk Kılıç

doi:10.24106/kefdergi.413303

TR EN

Örneklem Büyüklüğü, Korelasyon Tekniği ve Faktör Çıkarma Yönteminin Güvenirlik Katsayılarına Etkisi

Abstract

Bu araştırmanın amacı, güvenirlik katsayılarını örneklem büyüklüğüne (250, 500, 1000, 2500, 5000 ve 9773), EFA faktör çıkarma yöntemine (PCA, PA, ULS, WLS ve MLE); CFA kestirim yöntemine (UL, ML ve GL) ve korelasyon matrisine (Pearson, phi ve tetrakorik) göre karşılaştırmaktır. Bu amaçla temel araştırma yöntemine başvurulmuştur. Araştırma gerçek verilerle yürütülmüş olup araştırma verileri, 2014 yılında uygulanan Temel Eğitimden Ortaöğretime Geçiş sınavları Türkçe alt testine cevap veren öğrenci cevaplarından oluşturulmuştur. Araştırmada McDonald ω, McDonald ω_h, Maximal güvenirlik, Armor Ɵ, Heise ve Bohrnstedt Ω, Revelle β ve Standartlaştırılmış Alfa katsayıları karşılaştırılmıştır. Araştırma sonucunda aynı korelasyon matrisi için örneklem büyüklüğünün çok değişikliğe neden olmadığı gözlenmiştir. Tetrakorik korelasyonla hesaplanan McDonald ω_h ve Revelle β katsayılarının 1’den büyük olduğu bazı koşulların bulunduğu araştırmanın bir diğer bulgusudur. Araştırma sonuçlarına göre bu katsayıların konjerik tek faktörlü yapılar için phi korelasyonuyla hesaplanmasını önerilmiştir. Diğer bulgular literatür eşliğinde tartışılmış ve önerilerde bulunulmuştur.

Keywords

güvenirlik, McDonald ω, maximal güvenirlik, Armor Ɵ, Heise ve Bohrnstedt Ω, Revelle β, standartlaştırılmış alfa

The Effects of Sample Size, Correlation Technique, and Factor Extraction Method on Reliability Coefficients

Abstract

This study aims to compare reliability coefficients according to sample size (250, 500, 1,000, 2,500, 5,000, and 9,773), EFA factor extraction methods (PCA, PA, ULS, WLS, and MLE), CFA estimation methods (UL, ML, and GL), and correlation matrices (Pearson, phi, and tetrachoric). Therefore, it employs a basic research method. The study was conducted with real data, and the data were collected from students’ answers to a Turkish sub-test in the Test for Transition from Basic Education into Secondary Education administered in 2014. Within the scope of the study, McDonald ω, McDonald ω_h, maximal reliability, Armor Ɵ, Heise and Bohrnstedt Ω, Revelle β, and standardized alpha coefficients were compared. Consequently, it was found that sample size in the same correlation matrices did not lead to serious changes. It was also found that McDonald ω_hand Revelle β coefficients calculated with a tetrachoric correlation were bigger than 1 in some conditions. It was recommended in consequence that those coefficients should be calculated through phi correlations for congeric one-factor structures. Other findings obtained support the literature, and necessary suggestions are made.

Keywords

: reliability, McDonald ω, maximal reliability, Armor Ɵ, Heise and Bohrnstedt Ω, Revelle β, standardized alpha

References

AERA, APA, NCME, American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement In Education (NCME). (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
Allen, M. J., & Yen, W. M. (1979). Introduction to measurement theory. Monterey: Brooks/Cole Publishing Company.
Armor, D. J. (1974). Theta reliability and factor scaling. Sociological Methodology, 5, 17–50.
Carmines, E. G., & Zeller, R. A. (1987). Reliability and validity assessment (9th ed.). Beverly Hills, California: SAGE.
Chou, C. P., & Bentler, P. M. (1995). Estimates and tests in structural equation modeling. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications. Thousand Oaks, CA: SAGE.
Comrey, A. L. (1988). Factor-analytic methods of scale development in personality and clinical psychology. Journal of Consulting and Clinical Psychology, 56(5), 754–761. https://doi.org/10.1037/0022-006X.56.5.754
Crocker, L., & Algina, J. (2008). Introduction of classical and modern test theory. Ohio: Cengage Learning.
Curran, P. J., West, S. G., & Finch, J. F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1(1), 16–29. https://doi.org/10.1037/1082-989X.1.1.16
Ercan, I., Yazici, B., Sigirli, D., Ediz, B., & Kan, I. (2007). Examining Cronbach alpha, theta, omega reliability coefficients according to sample size. Journal of Modern Applied Statistical Methods, 6(1), 291–303. Retrieved from http://digitalcommons.wayne.edu/jmasm/vol6/iss1/27
Fox, J. (2016). polycor: Polychoric and polyserial correlations. Retrieved from https://cran.r-project.org/package=polycor

Fraenkel, J. R., Wallen, N. E., & Huyn, H. H. (2012). How to design and evaluate research in education (8th ed.). New York: McGraw-Hill.
Gay, L. R., Mills, G. E., & Airasian, P. (2012). Educational research: Competencies for analysis and applications (10th ed.). New Jersey: Pearson.
Gorsuch, R. L. (1974). Factor Analysis (1st ed.). Toronto: W. B. Saunders Company.
Green, S. B., & Yang, Y. (2009). Reliability of summed item scores using structural equation modeling: An alternative to coefficient alpha. Psychometrika, 74(1), 155–167. https://doi.org/10.1007/s11336-008-9099-3
Guadagnoli, E., & Velicer, W. F. (1988). Relation of sample size to the stability of component patterns. Psychological Bulletin, 103(2), 265–275.
Gulliksen, H. (1950). Theory of mental tests. New York: Wiley.
Hancock, G. R., & Mueller, R. O. (2001). Rethinking construct reliability within latent variable systems. In Structural Equation Modeling: Present and Future: A Festschrift in Honor of Karl Jöreskog (pp. 195–216).
Hancock, G. R., & Mueller, R. O. (2013). Structural equating modelling: A second course. B. G. Tabachnick & L. S. Fidell (Orgs.), Using multivariate statistics (2nd ed.). Charlotte, NC: Information Age Publishing.
Heise, D. R., & Bohrnstedt, G. W. (1970). Validity, invalidity, and reliability. In E. F. Borgatta & G. W. Bohrnstedt (Eds.), Sociological Methodology (pp. 104–129). San Francisco: Jossey-Bass.
McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: LEA Publisher.
Meyer, J. P. (2010). Reliability. New York: Oxford University Press.
Osburn, H. G. (2000). Coefficient alpha and related internal consistency reliability coefficients. Psychological Methods, 5(3), 343–355. https://doi.org/10.1037/1082-989X.5.3.343
Price, L. R. (2017). Psychometric methods: Theory and practice. New York, NY: The Guilford Press.
R Core Team. (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.r-project.org/.
Revelle, W. (1979). Hierarchical cluster analysis and the internal structure of tests. Multivariate Behavioral Research, 14, 57–74. https://doi.org/10.1207/s15327906mbr1401_4
Revelle, W. (2016). psych: Procedures for Psychological, Psychometric, and Personality Research. Evanston, Illinois. Retrieved from https://cran.r-project.org/package=psych
Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized latent variable modelling: Multilevel, longitudinal, and structural equation models. New York: Chapman & Hall.
Streiner, D. L. (1994). Figuring out factors: the use and misuse of factor analysis. Canadian Journal of Psychiatry, 39(3), 135–140.
Zinbarg, R. E., Revelle, W., Yovel, I., & Li, W. (2005). Cronbach’s, α Revelle’s β and McDonald’s ω H: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70(1), 123–133. https://doi.org/10.1007/s11336-003-0974-7

Details

Primary Language

English

Subjects

Studies on Education

Journal Section

Research Article

Authors

Nuri Doğan This is me

Abdullah Faruk Kılıç

Publication Date

May 15, 2018

Submission Date

April 17, 2017

Acceptance Date

August 25, 2017

Published in Issue

Year 2018 Volume: 26 Number: 3

DOI

https://doi.org/10.24106/kefdergi.413303

IZ

https://izlik.org/JA79BF95BH

APA

Doğan, N., & Kılıç, A. F. (2018). The Effects of Sample Size, Correlation Technique, and Factor Extraction Method on Reliability Coefficients. Kastamonu Education Journal, 26(3), 697-706. https://doi.org/10.24106/kefdergi.413303

AMA

1.Doğan N, Kılıç AF. The Effects of Sample Size, Correlation Technique, and Factor Extraction Method on Reliability Coefficients. Kastamonu Education Journal. 2018;26(3):697-706. doi:10.24106/kefdergi.413303

Chicago

Doğan, Nuri, and Abdullah Faruk Kılıç. 2018. “The Effects of Sample Size, Correlation Technique, and Factor Extraction Method on Reliability Coefficients”. Kastamonu Education Journal 26 (3): 697-706. https://doi.org/10.24106/kefdergi.413303.

EndNote

Doğan N, Kılıç AF (May 1, 2018) The Effects of Sample Size, Correlation Technique, and Factor Extraction Method on Reliability Coefficients. Kastamonu Education Journal 26 3 697–706.

IEEE

[1]N. Doğan and A. F. Kılıç, “The Effects of Sample Size, Correlation Technique, and Factor Extraction Method on Reliability Coefficients”, Kastamonu Education Journal, vol. 26, no. 3, pp. 697–706, May 2018, doi: 10.24106/kefdergi.413303.

ISNAD

Doğan, Nuri - Kılıç, Abdullah Faruk. “The Effects of Sample Size, Correlation Technique, and Factor Extraction Method on Reliability Coefficients”. Kastamonu Education Journal 26/3 (May 1, 2018): 697-706. https://doi.org/10.24106/kefdergi.413303.

JAMA

1.Doğan N, Kılıç AF. The Effects of Sample Size, Correlation Technique, and Factor Extraction Method on Reliability Coefficients. Kastamonu Education Journal. 2018;26:697–706.

MLA

Doğan, Nuri, and Abdullah Faruk Kılıç. “The Effects of Sample Size, Correlation Technique, and Factor Extraction Method on Reliability Coefficients”. Kastamonu Education Journal, vol. 26, no. 3, May 2018, pp. 697-06, doi:10.24106/kefdergi.413303.

Vancouver

1.Nuri Doğan, Abdullah Faruk Kılıç. The Effects of Sample Size, Correlation Technique, and Factor Extraction Method on Reliability Coefficients. Kastamonu Education Journal. 2018 May 1;26(3):697-706. doi:10.24106/kefdergi.413303

Cited By

Drawing a Sample with Desired Properties from Population in R Package “drawsample”

Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi

https://doi.org/10.21031/epod.790449

Factors Affecting Urban Resilience and Sustainability: Case of Slum Dwellers in Islamabad, Pakistan

Journal of Urban Planning and Development

https://doi.org/10.1061/(ASCE)UP.1943-5444.0000749

Kastamonu Education Journal is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license.