Item Removal Strategies Conducted in Exploratory Factor Analysis: A Comparative Study

Year 2022, , 165 - 180, 10.03.2022

Meltem Acar Güvendir , Yeşim Özer Özkan

https://doi.org/10.21449/ijate.827950

Cited By: 21

Abstract

The aim of this study is to examine how the practice of different item removal strategies during exploratory factor analysis (EFA) phase of scale development change the number of factors, factor loadings, explained variance ratio, and reliability values (α and ω) explained. In the study, data obtained from 379 university students were used for the development of a 46-item scale. As the first item removal strategy, crossloading items on two factors and where the difference between factor loadings was less than .10 were identified. Then, items were removed one by one, starting with the item with the least difference between the loadings on the factors. As the second strategy, the items that loaded on two factors and where the difference between factor loadings was less than .10 were found, and these items were removed from the scale as a whole. As the third strategy, the items that gave high loading on more than two factors and where the difference between these factors was less than .10 were identified. The item removal process was started with these items. The study results show that the factor numbers obtained using three different strategies during the item removal process of EFA were the same; however, the number of items on the scale, the explained variance ratio, and the total scale, and reliability values differed. Furthermore, the items in the factors were not all the same. The study results underscore the importance of theoretical competence in the scale development process.

Keywords

Exploratory Factor Analysis, Item removal strategies, Crossloading items, Construct validity, Scale development.

References

• Albayrak, A.S. (2006). Uygulamalı çok değişkenli istatistik teknikleri [Applied multivariate statistical techniques]. Asil Yayın Dağıtım.
• Albayrak, A.S. (2005). Çoklu doğrusal bağlantı halinde en küçük kareler tekniğinin alternatifi yanlı tahmin teknikleri ve bir uygulama [Alternative to the minimum square technique: a multi-linear connection balanced estimating techniques and an application]. Zonguldak Karaelmas Üniversitesi Sosyal Bilimler Dergisi, 1(1), 105 126.
• Bartlett, M.S. (1950). Tests of significance in factor analysis. British Journal of Psychology, 3, 77-85.https://doi.org/10.1111/j.2044-8317.1950.tb00285.x
• Basto, M., & Pereira, J.M. (2012). An SPSS R-Menu for ordinal factor analysis. Journal of statistical software, 46(4), 1-29. https://doi.org/10.18637/jss.v046.i04
• Bornstein, R.F. (1996). Face validity in psychological assessment: Implications for a unified model of validity. American Psychologist, 51(9), 983-984. https://doi.org/10.1037/0003-066X.51.9.983
• Brown, J.D. (2009). Statistics Corner Questions and answers about language testing statistics: Principal components analysis and exploratory factor analysis, In. Definitions, differences, and choices. Shiken: JALT Testing & Evaluation SIG Newsletter, 13(1), 19 - 23. https://hosted.jalt.org/test/PDF/Brown30.pdf
• Büyüköztürk, Ş. (2007). Veri Analizi El Kitabı [Data analysis handbook for social sciences]. Ankara: Pegem Yayınları.
• Bryman, A., & Cramer, D. (2011). Quantitative data analysis with IBM SPSS 17, 18 and 19: A guide for social scientists. Routledge-Cavendish/Taylor & Francis Group.
• Can, A. (2016). SPSS ile bilimsel araştırma sürecinde nicel veri analizi [Quantitative data analysis in the process of scientific research with SPSS]. Ankara: Pegem Akademi.
• Cattell, R.B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1(2), 245-276. https://doi.org/10.1207/s15327906mbr0102_10
• Clark, L.A., & Watson, D. (2016). Constructing validity: Basic issues in objective scale development. In A. E. Kazdin (Ed.), Methodological issues and strategies in clinical research (pp. 187 203). American Psychological Association. https://doi.org/10.1037/14805-012
• Comrey, A.L. (1962). The minimum residual method of factor analysis. Psychological Reports, 11(1), 15-18. https://doi.org/10.2466/pr0.1962.11.1.15
• Comrey, A.L., & Lee, H.B. (1973). A first course in factor analysis. Academic Press.
• Costello, A.B., & Osborne, J. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical assessment, research, and evaluation, 10(7), 1-9. https://doi.org/10.7275/jyj1-4868.
• Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. CBS College Publishing.
• Cronbach, L.J., & Meehl, P.E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302. https://doi.org/10.1037/h0040957.
• Çokluk, Ö., Şekercioğlu, G., & Büyüköztürk, Ş. (2010). Sosyal bilimler için çok değişkenli istatistik [Multivariate statistics for social sciences]. Pegem Akademi.
• Erkuş, A. (2012). Psikolojide ölçme ve ölçek geliştirme [Measurement and scale development in psychology]. Pegem Akademi Yayınları.
• Erkuş, A., Sünbül, Ö., Sünbül, S Ö., Yormaz, S., & Aşiret, S. (2017). Psikolojide ölçme ve ölçek geliştirme II [Measurement and scale development in psychology II]. Pegem Akademi.
• Ekström, J. (2011). A Generalized Definition of the Polychoric Correlation Coefficient. UCLA: Department of Statistics, UCLA. https://escholarship.org/uc/item/583610fv
• Field, A. (2005). Discovering statistics using SPSS. (2nd ed.). London: Sage
• Finney, S.J., & DiStefano, C. (2013). Nonnormal and categorical data in structural equation modeling. G. R. Hancock ve R. O. Mueller (Ed.), Structural equation modeling: A second course (2nd ed., 439– 492). Charlotte.
• Ford, J.K., MacCallum, R.C., & Tait, M. (1986). The Application of exploratory factor analysis in applied psychology: A Critical review and analysis. Personnel Psychology, 39(2), 291-314. https://doi.org/10.1111/j.1744-6570.1986.tb00583.x
• Gorsuch, R.L. (1983). Factor analysis (2nd ed.). Hillside, NJ: Lawrence Erlbaum Associates
• Hauben, M., Hung, E., & Hsieh, W.Y. (2017). An exploratory factor analysis of the spontaneous reporting of severe cutaneous adverse reactions. Therapeutic advances in drug safety, 8(1), 4-16. https://doi.org/10.1177/2042098616670799
• Hayton, J.C., Allen, D.G., & Scarpello, V. (2004). Factor retention decisions in exploratory factor analysis: A tutorial on parallel analysis. Organizational Research Methods, 7(2), 191-205. https://doi.org/10.1177/1094428104263675
• Henson, R.K., & Roberts, J.K. (2006). Use of exploratory factor analysis in published research: Common errors and some comment on improved practice. Educational and Psychological Measurement, 66(3), 393-416. https://doi.org/10.1177/0013164405282485
• Horn, J.L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179-185. https://doi.org/10.1007/BF02289447
• Johnson, R.A., & Wichern, D.W. (2002). Applied multivariate statistical analysis. Upper Saddle River.
• Kass, R.A., & Tinsley, H.E.A. (1979). Factor analysis. Journal of Leisure Research, 11, 120-138. https://doi.org/10.1080/00222216.1979.11969385
• Kerlinger, F.N. (1979). Behavioral research: A conceptual approach. Rinehart & Winston.
• Kline, P. (1994). An easy guide to factor analysis. Routledge.
• Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd Edition). The Guilford Press.
• Leech, N.L., Barrett, K.C., & Morgan, G.A. (2005) SPSS for Intermediate Statistics, Use and Interpretation. 2nd Edition. Lawrence Erlbaum.
• Loevinger, J. (1957). Objective tests as instruments of psychological theory. Psychological Reports, 3(3), 635–694. https://doi.org/10.2466/pr0.1957.3.3.635
• MacCallum, R.C., Widaman, K. F., Zhang, S., & Hong, S. (1999). Sample size in factor analysis. Psychological Methods, 4(1), 84-99. https://doi.org/10.1037/1082-989X.4.1.84
• Mardia, K.V. (1970). Measures of multivariate skewnees and kurtosis with applications. Biometrika, 57(3), 519-530. https://doi.org/10.2307/2334770
• Mertler, C.A., & Vannatta, R.A. (2005). Advanced and multivariate statistical methods: Practical application and interpretation (3th ed.). Pyrczak Publishing.
• Messick, S. (1981). Evidence and ethics in the evaluation of tests 1. ETS Research Report Series, 1981(1), 1-41. https://doi.org/10.1002/j.2333-8504.1981.tb01244.x
• Murphy, K.R., & Davidshofer, C.O. (2005). Psychological testing: principles and applications. Pearson Education International.
• Netemeyer, R.G., Bearden, W.O., & Sharma, S. (2003). Scaling procedures. SAGE Publications, Inc.
• Nunally, J.C. (1978). Psychometric theory. McGraw Hill.
• Özgüven, E. (1994). Psikolojik testler [Psychological tests]. Yeni Doğuş Matbaası.
• Park, H.S., Dailey, R., & Lemus, D. (2002). The use of exploratory factor analysis and principal components analysis in communication research. Human Communication Research, 28(4), 562-577. https://doi.org/10.1111/j.1468-2958.2002.tb00824.x
• Pett, M.A., Lackey, N.R., & Sullivan, J.J. (2003). Making sense of factor analysis: The use of factor analysis for instrument development in health care research. SAGE.
• Raubenheimer, J. (2004). An item selection procedure to maximize scale reliability and validity. SA Journal of Industrial Psychology, 30(4), 59 64. https://doi.org/10.4102/sajip.v30i4.168
• Samuels, P. (2017). Advice on Exploratory Factor Analysis. Technical Report. Centre for Academic Success, Birmingham City University.
• Scherer, R.F., Luther, D.C., Wiebe, F.A., & Adams, J.S. (1988). Dimensionality of coping: Factor stability using the ways of coping questionnaire. Psychological Reports, 62(3), 763-770. https://doi.org/10.2466/pr0.1988.62.3.763
• Sarstedt M., & Mooi E. (2014). Factor analysis. In: A concise guide to market research. springer texts in business and economics. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-53965-7_8
• Spicer J. (2005). Making sense of multivariate data analysis: An Intuitive approach. SAGE.
• Stapleton, C.D. (1997). Basic concepts and procedures of confirmatory factor analysis [Paper presentation]. The Annual Meeting of the South West Educational Research Association. Austin.
• Stewens, J. (1996). Appied multivariate statistics for the social science (Third Edition). Lawrence Erlbaum Associates.
• Şencan, H. (2005). Sosyal ve davranışsal ölçümlerde güvenirlik ve geçerlilik [Reliability and validity in social and behavioral assessments]. Seçkin Yayıncılık.
• Tabachnick, B.G., & Fidell, L.S. (1996). Using multivariate statistics. Harper Collins.
• Tabachnick, B.G., & Fidell, L.S. (2001). Using multivariate statistics. 4th Edition, Allyn and Bacon. MA.
• Tavşancıl E. (2002). Tutumların ölçülmesi ve SPSS ile veri analizi [Measurement of attitudes and data analysis with SPSS]. Nobel Yayınevi.
• Velicer, W.F. (1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 41(3), 321-327. https://doi.org/10.1007/BF02293557
• Ziegler, M. (2014). Comments on item selection procedures. European Journal of Psychological Assessment, 30(1), 1-2. https://doi.org/10.1027/1015-5759/a000196