Comparing the Automatic Item Selection Procedure and Exploratory Factor Analysis in Determining Factor Structure

Yıl 2022, Cilt: 9 Sayı: 2, 416 - 436, 01.03.2022

Asiye Şengül Avşar

https://doi.org/10.17275/per.22.47.9.2

Cited By: 1

Öz

It is necessary to supply proof regarding the construct validity of the scales. Especially, when new scales are developed the construct validity is researched by the Exploratory Factor Analysis (EFA). Generally, factor extraction is performed via the Principal Component Analysis (PCA) which is not exactly factor analysis and the Principal Axis Factoring (PAF) among EFA methods. Factors may also be determined with different techniques depending on the advances in psychometry. In the context of nonparametric item response theory, the Mokken Scale Analysis (MSA) and the Automatic Item Selection Procedure (AISP) provide significant contributions to researchers in scale development studies. The aim of the current study is to compare the AISP and the EFA methods in determining the factor structures. The Revised Life Orientation Test, whose factor structure was previously known and validated, and the draft Expectation Scale from Academics in Distance Education Scale, which was at the preliminary stage of the scale development process with the unknown factor structure, were considered in this comparison. It was determined that the consistency of the findings obtained from the EFA, and the ones obtained from the AISP provided strong evidence in ensuring the construct validity. The PAF and the AISP produced similar results in this research. The PAF results should be taken into consideration instead of the PCA, especially in scale development studies. It is recommended that the AISP and the PAF results be evaluated together for strong evidence in the investigation of construct validity in scale development studies

Anahtar Kelimeler

construct validity, principal component analysis, principal axis factoring, Mokken scale analysis, automatic item selection procedure

Kaynakça

• Aljubaily, H. Y. (2010). Measuring university students' perceptions of characteristics of ideal university instructor in Saudi Arabia and the united states: An application of nonparametric item response theory study (Doctoral Dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3434898).
• Anastasia, A. (1976). Psychological testing (6th ed.). New York: Macmillan Publishing.
• Bagnasco, A., Watson, R., Zanini, M., Rosa, F., Rocco, G., & Sasso, L. (2015). Preliminary testing using Mokken scaling of an Italian translation of the Edinburgh Feeding Evaluation in Dementia (EdFED-I) scale. Applied Nursing Research, 28(4), 391-396. doi: 10.1016/j.apnr.2015.02.003
• Bech, P., Carrozzino, D., Austin, S. F., Møller, S. B., & Vassend, O. (2016). Measuring euthymia within the Neuroticism Scale from the NEO Personality Inventory: a Mokken analysis of the Norwegian general population study for scalability. Journal of Affective Disorders, 193, 99-102. doi: 10.1016/j.jad.2015.12.039
• Chen, Y., Watson, R., & Hilton, A. (2016). An exploration of the structure of mentors' behavior in nursing education using exploratory factor analysis and Mokken scale analysis. Nurse Education Today, 40, 161-167. doi: 10.1016/j.nedt.2016.03.001
• Chou, Y. H., Lee, C. P., Liu, C. Y., & Hung, C. I. (2017). Construct validity of the depression and somatic symptoms scale: evaluation by Mokken scale analysis. Neuropsychiatric Disease and Treatment, 13, 205-211. doi: 10.2147/NDT.S11882
• Cohen, R. J., & Swerdlik, M. (2009). Psychological assessment: an introduction to tests and measurements (7th ed.). McGraw-Hill Primis.
• Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98-104. doi: 10.1037/0021-9010.78.1.98
• Crișan, D. R., Tendeiro, J., & Meijer, R. (2019, September 24). The crit value as an effect size measure for violations of model assumptions in Mokken scale analysis for binary data. doi: 10.31234/osf.io/8ydmr
• Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Cengage Learning: USA.
• De Ayala, R. J. (2009). The theory and practice of item response theory. Guilford Publications: New York.
• de Cock, E. S., Emons, W. H., Nefs, G., Pop, V. J., & Pouwer, F. (2011). Dimensionality and scale properties of the Edinburgh Depression Scale (EDS) in patients with type 2 diabetes mellitus: the DiaDDzoB study. BMC Psychiatry, 11(1), 141. doi: 10.1186/1471-244X-11-14
• DeVellis, R. F. (2017). Scale development: theory and applications (4th ed.). Sage publications: Los Angeles.
• Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates: New Jersey.
• Emons, W. H., Sijtsma, K., & Pedersen, S. S. (2012). Dimensionality of the Hospital Anxiety and Depression Scale (HADS) in cardiac patients: comparison of Mokken scale analysis and factor analysis. Assessment, 19(3), 337-353. doi: 10.1177/1073191110384951
• Field, A. (2009). Discovering statistics using IBM SPSS statistics. (4th ed.). Sage Publication: London.
• Gudergan, S., Mathies, C., Kyngdon, A., & Kozicki, S. (2004, December). Negotiation style measurement scale development and testing. Paper presented at the Australian and New Zealand Marketing Academy Conference, Wellington. Abstract retrieved from https://opus.lib.uts.edu.au/handle/10453/3133
• Gustems-Carnicer, J., Calderón, C., & Santacana, M. F. (2017). Psychometric properties of the Life Orientation Test (LOT-R) and its relationship with psychological well-being and academic progress in college students. Revista Latinoamericana de Psicología, 49(1), 19-27. doi: 10.1016/j.rlp.2016.05.001
• Hair, J. F., Black, C. W., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis (8th ed.). Annabel Ainscow: United Kingdom.
• Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: principles and application. Boston: Kluwer Academic Publishers Group.
• Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. USA: Sage Publications.
• Howard, M. C. (2016). A review of exploratory factor analysis decisions and overview of current practices: what we are doing and how can we improve? International Journal of Human-Computer Interaction, 32(1), 51-62. doi: 10.1080/10447318.2015.1087664
• Kline, P. (1994). An easy guide to factor analysis. Routledge: USA.
• Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563-575.
• Lee, C. P., Chen, Y., Jiang, K. H., Chu, C. L., Chiu, Y. W., Chen, J. L., & Chen, C. Y. (2016). Development of a short version of the Aging Males’ Symptoms scale: Mokken scaling analysis and Rasch analysis. The Aging Male, 19(2), 117-123. doi: 10.3109/13685538.2016.1157861
• Lee, C. P., Fu, T. S., Liu, C. Y., & Hung, C. I. (2017). Psychometric evaluation of the Oswestry Disability Index in patients with chronic low back pain: factor and Mokken analyses. Health and Quality of Life Outcomes, 15(1), 1-7. doi: 10.1186/s12955-017-0768-8
• Meijer, R. R., & Baneke, J. J. (2004). Analyzing psychopathology items: a case for nonparametric item response theory modeling. Psychological Methods, 9(3), 354-368. doi: 10.1037/1082-989X.9.3.354
• Mokken, R. J. (1997). Nonparametric models for dichotomous responses. In W. J. van der Linden, & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 351-368). New York: Springer-Verlag.
• Mooij, T. (2012). A Mokken scale to assess secondary pupils’ experience of violence in terms of severity. Journal of Psychoeducational Assessment, 30(5), 496-508. doi: 10.1177/0734282912439387
• Scheier, M. F., Carver, C. S., & Bridges, M. W. (1994). Distinguishing optimism from neuroticism (and trait anxiety, self-mastery, and self-esteem): a reevaluation of the Life Orientation Test. Journal of Personality and Social Psychology, 67(6), 1063. doi: 10.1037/0022-3514.67.6.1063
• Shenkin, S.D., Watson, R., Laidlaw, K., Starr, J. M., & Deary, I. J. (2014). The attitudes to ageing questionnaire: Mokken scaling analysis. PLOS ONE 9(9): e108766. doi: 10.1371/journal.pone.0099100
• Sijtsma, K., & Molenaar, I. W. (2002). Introduction to nonparametric item response theory. Sage Publications.
• Ski, C. F., Thompson, D. R., Hare, D. L., Stewart, A. G., & Watson, R. (2012). Cardiac Depression Scale: Mokken scaling in heart failure patients. Health and Quality of Life Outcomes, 10(141). doi: 10.1186/1477-7525-10-141
• Stewart, M. E., Allison, C., Baron-Cohen, S., & Watson, R. (2015). Investigating the structure of the autism-spectrum quotient using Mokken scaling. Psychological Assessment, 27(2), 596-604. doi: 10.1037/pas0000058
• Stout, W. (2001). Nonparametric item response theory: a maturing and applicable measurement modeling approach. Applied Psychological Measurement, 25(3), 300-306. doi: 10.1177/01466210122032109
• Tabachnick, B. G., & Fidell, L. S. (2014). Using multivariate statistics (6th ed.). Pearson: United States of America.
• Van der Ark, L. A. (2020). Package Mokken. https://cran.rproject.org/web/packages/mokken/mokken.pdf
• VandenBos, G. R. (2015). APA dictionary of psychology (2nd ed.). Washington DC: American Psychiatric Pub. doi: 10.1037/14646-000
• Vaughan, B., & Grace, S. (2018). A Mokken scale analysis of the peer physical examination questionnaire. Chiropractic & Manual Therapies, 26(1), 6. doi: 10.1186/s12998-018-0176-0
• Watson, R., van der Ark, L. A., Lin, L. C., Fieo, R., Deary, I. J., & Meijer, R. R. (2012). Item response theory: how Mokken scaling can be used in clinical practice. Journal of Clinical Nursing, 21(19pt20), 2736-2746. doi: 10.1111/j.1365-2702.2011.03893.x
• Wismeijer, A. A., Sijtsma, K., van Assen, M. A., & Vingerhoets, A. J. (2008). A comparative study of the dimensionality of the self-concealment scale using principal components analysis and Mokken scale analysis. Journal of Personality Assessment, 90(4), 323-334. doi: 10.1080/00223890802107875
• Yoon, S., Shaffer, J. A., & Bakken, S. (2015). Refining a self-assessment of informatics competency scale using Mokken scaling analysis. Journal of Interprofessional Care, 29(6), 579-586. doi: 10.3109/13561820.2015.1049340