Application of the Rasch model in streamlining an instrument measuring depression among college students

Year 2023, , 257 - 278, 26.06.2023

Sherwin Balbuena

https://doi.org/10.21449/ijate.1210479

Abstract

Depression is a latent characteristic that is measured through self-reported or clinician-mediated instruments such as scales and inventories. The Precision of depression estimates largely depends on the validity of the items used and on the truthfulness of people responding to these items. The existing methodology in instrumentation based on a factor-analytic approach has limited applicability, especially in the detection of sources of measurement error in item- and person-level analyses. While there are probabilistic approaches such as the use of Item Response Theory and the Rasch model in validating instruments, there are no definite guidelines on the sequence of steps to follow. This study explored the suitability of the Rasch model in assessing and streamlining the University Student Depression Inventory (USDI) using a sequential strategy based on the item response model assumptions, which involves fitting the data to the model through the elimination of misfits, analyzing retained items, and constructing measures. The strategy was applied to two sets of survey data collected from the same population of college students enrolled in a Philippine university but in different semesters. Results showed that the Rasch procedure was able to detect misfit items and persons, which guided decisions regarding the removal of problematic items and persons while preserving the reliability of the original scale. The methodology used was found to be replicable, as the analyses for the two datasets yielded comparable results in terms of number of items retained, item estimates and severity ordering, and distribution of student depression measures.

Keywords

Rasch, depression, precision, measurement, USDI

References

• American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Mental Disorders: DSM-5. Washington, D.C: American Psychiatric Association.
• Andrich, D. (1978). Application of a psychometric rating model to ordered categories which are scored with successive integers. Applied Psychological Measurement, 2(4), 581-594.
• Andrich, D. (1982). An index of person separation in latent trait theory, the traditional KR. 20 index, and the Guttman scale response pattern. Education Research and Perspectives, 9(1), 95-104.
• Avery, L.M., Russell, D.J., Raina, P.S., Walter, S.D., & Rosenbaum, P.L. (2003). Rasch analysis of the Gross Motor Function Measure: validating the assumptions of the Rasch model to create an interval-level measure. Archives of Physical Medicine and Rehabilitation, 84(5), 697-705.
• Balsamo, M., Giampaglia, G., & Saggino, A. (2014). Building a new Rasch-based self-report inventory of depression. Neuropsychiatric Disease and Treatment, 10, 153.
• Beck, A.T., Ward, C.H., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for measuring depression. Archives of General Psychiatry, 4(6), 561-571.
• Bond, T.G., & Fox, C.M. (2013). Applying the Rasch model: Fundamental measurement in the human sciences. Psychology Press.
• Boyle, G.J. (1985). Self-report measures of depression: some psychometric considerations. British Journal of Clinical Psychology, 24, 45–59.
• Chen, W.H., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22(3), 265-289.
• Cronbach, L.J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334.
• Deb, S., Banu, P.R., Thomas, S., Vardhan, R.V., Rao, P.T., & Khawaja, N. (2016). Depression among Indian university students and its association with perceived university academic environment, living arrangements and personal issues. Asian Journal of Psychiatry, 23, 108-117.
• Forkmann, T., Gauggel, S., Spangenberg, L., Brähler, E., & Glaesmer, H. (2013). Dimensional assessment of depressive severity in the elderly general population: Psychometric evaluation of the PHQ-9 using Rasch Analysis. Journal of Affective Disorders, 148(2-3), 323-330.
• Galesic, M., & Bosnjak, M. (2009). Effects of questionnaire length on participation and indicators of response quality in a web survey. Public Opinion Quarterly, 73(2), 349-360.
• Gesinde, A.M., & Sanu, O.J. (2014). Prevalence and gender difference in self-reported depressive symptomatology among Nigerian university students: Implication for depression counselling. The Counsellor, 33(2), 129-140.
• Guttman, L. (1950). The basis for scalogram analysis. In S.A. Stoufer, L. Guttman, E.A. Suchman, P.L. Lazarsfeld, S.A. Star, and J.A. Clausen (Eds.), Studies in social psychology in World War II: Vol. IV. Measurement and prediction (pp. 60–90). Princeton, NJ: Princeton University Press.
• Habibi, M., Khawaja, N.G., Moradi, S., Dehghani, M., & Fadaei, Z. (2014). University student depression inventory: Measurement model and psychometric properties. Australian Journal of Psychology, 66(3), 149-157.
• Hamilton, M. (1960). A rating scale for depression. Journal of Neurology, Neurosurgery, and Psychiatry, 23(1), 56-62.
• Hankin, B.L. (2006). Adolescent depression: Description, causes, and interventions. Epilepsy and Behavior, 8(1), 102-114.
• Hyde, J.S., Mezulis, A.H., & Abramson, L.Y. (2008). The ABCs of depression: Integrating affective, biological, and cognitive models to explain the emergence of the gender difference in depression. Psychological Review, 115(2), 291-313.
• Jeong, H.J., & Lee, W.C. (2016). The level of collapse we are allowed: Comparison of different response scales in Safety Attitudes Questionnaire. Biometrics and Biostatistics International Journal, 4(4), 1-7.
• Khawaja, N.G., & Bryden, K.J. (2006). The development and psychometric investigation of the University Student Depression Inventory. Journal of Affective Disorders, 96(1-2), 21-29.
• Khawaja, N.G., Santos, M.L.R., Habibi, M., & Smith, R. (2013). University students' depression: A cross-cultural investigation. Higher Education Research and Development, 32(3), 392-406.
• Kohli, N., Koran, J., & Henn, L. (2015). Relationships among classical test theory and item response theory frameworks via factor analytic models. Educational and Psychological Measurement, 75(3): 389-405.
• Lailo, J.M.A. (2018). Determinants of depressive symptoms in undergraduate UPLB students: A joint correspondence analysis. Institute of Statistics, UPLB.
• Lee, R.B., Maria, M.S., Estanislao, S., & Rodriguez, C. (2013). Factors associated with depressive symptoms among Filipino university students. PloS One, 8(11): e79825.
• Linacre, J.M. (1997). Guidelines for rating scales MESA Research Note #2. Available at http://www.rasch.org/rn2.htm.
• Linacre, J.M. (2002). What do infit and outfit, mean-square and standardized mean? Rasch Measurement Transactions, 16(2), 878.
• Linacre, J.M., & Wright, B.D. (1994). Chi-square fit statistics. Rasch Measurement Transactions, 8(2), 350.
• Lim, G.Y., Tam, W.W., Lu, Y., Ho, C.S., Zhang, M.W., & Ho, R.C. (2018). Prevalence of depression in the community from 30 countries between 1994 and 2014. Scientific Reports, 8(1), 2861.
• Lovibond, P.F., & Lovibond, S.H. (1995). The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behaviour Research and Therapy, 33(3), 335-343.
• Mair, P., & Hatzinger, R. (2007). Extended Rasch Modeling: The eRm package for the application of IRT models in R. Journal of Statistical Software, 20(9), 1–20. Available at http://www.jstatsoft.org/v20/i09 .
• Maloney, P., Grawitch, M.J., & Barber, L.K. (2011). Strategic item selection to reduce survey length: Reduction in validity? Consulting Psychology Journal: Practice and Research, 63, 162-175.
• Marcus, M., Yasamy, M.T., Van Ommeren, M., Chisholm, D., & Saxena, S. (2012). Depression: A Global Public Health Concern. Geneva: World Health Organization. Available at http://www.who.int/mental_health/management/depression/who_paper_depression_wfmh_2012.pdf.
• Masters, G.N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149-174.
• Mikolajczyk, R.T., Maxwell, A.E., El Ansari, W., Naydenova, V., Stock, C., Ilieva, S., ..., & Nagyova, I. (2008). Prevalence of depressive symptoms in university students from Germany, Denmark, Poland and Bulgaria. Social Psychiatry and Psychiatric Epidemiology, 43(2), 105-112.
• Nord, M. (2014). Introduction to Item Response Theory Applied to Food Security Measurement: Basic Concepts, Parameters, and Statistics. Technical Paper. Rome: FAO. Available at http://www.fao.org/economic/ess/ess-fs/voices/en
• O'Connell, M.E., Boat, T., & Warner, K.E. (Eds.). (2009). Committee on the prevention of mental disorders and substance abuse among children, youth, and young adults: Research advances and promising interventions. Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. National Academies Press.
• Olsen, L.R., Jensen, D.V., Noerholm, V., Martiny, K., & Bech, P. (2003). The internal and external validity of the Major Depression Inventory in measuring severity of depressive states. Psychological Medicine, 33(2), 351-356.
• Radloff, L.S. (1977). The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1(3), 385-401.
• Rasch, G. (1960). Probabilistic Models for Some Intelligence and Attainment Tests. Copenhagen: Danish Institute for Educational Research. Chapters V-VII, X.
• Romaniuk, M., & Khawaja, N.G. (2013). University Student Depression Inventory (USDI): Confirmatory factor analysis and review of psychometric properties. Journal of Affective Disorders, 150(3), 766-775.
• Sharif, A.R., Ghazi-Tabatabaei, M., Hejazi, E., Askarabad, M.H., & Dehshiri, G.R. (2011). Confirmatory factor analysis of the University Student Depression Inventory (USDI). Procedia-Social and Behavioral Sciences, 30, 4-9.
• Shea, T.L., Tennant, A., & Pallant, J.F. (2009). Rasch model analysis of the Depression, Anxiety and Stress Scales (DASS). BMC Psychiatry, 9(1), 1-10.
• Smith, R.M. (2000). Fit analysis in latent trait measurement models. Journal of applied Measurement, 1(2), 199-218.
• Spitzer, R.L., Kroenke, K., Williams, J.B., & Patient Health Questionnaire Primary Care Study Group. (1999). Validation and utility of a self-report version of PRIME-MD: The PHQ primary care study. JAMA, 282(18), 1737-1744.
• Stansbury, J.P., Ried, L.D., & Velozo, C.A. (2006). Unidimensionality and bandwidth in the Center for Epidemiologic Studies Depression (CES–D) scale. Journal of Personality Assessment, 86(1), 10-22.
• Stanton, J.M., Sinar, E.F., Balzer, W.K., & Smith, P.C. (2002). Issues and strategies for reducing the length of self-report scales. Personnel Psychology, 55, 167-193.
• Swaminathan, H., & Rogers, H.J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27(4), 361-370.
• Tennant, A. (2004). Disordered thresholds: An example from the functional independence measure. Rasch Measurement Transactions, 17(4), 945-948
• Tennant, A., & Conaghan, P.G. (2007). The Rasch measurement model in rheumatology: What is it and why use it? When should it be applied, and what should one look for in a Rasch paper?. Arthritis Care and Research, 57(8), 1358-1362.
• UPLB INSTAT. (2018). Utak at Puso: A Survey on the Mental Health Status of UPLB Students (STAT 173 Survey). University of the Philippines Los Baños, Laguna.
• Wainer, H., & Kiely, G.L. (1987). Item clusters and computerized adaptive testing: A case for testlets. Journal of Educational Measurement, 24(3), 185-201.
• Wongpakaran, N., Wongpakaran, T., & Kuntawong, P. (2019). Evaluating hierarchical items of the geriatric depression scale through factor analysis and item response theory. Heliyon, 5(8), e02300.
• World Health Organization. (2017). Depression and other common mental disorders: Global health estimates. Geneva: Author. http://apps.who.int/iris/bitstream/handle/10665/254610/WHO-MSD-MER-2017.2-eng.pdf?sequence=1
• Wright B.D., & Linacre, J.M. (1987). Dichotomous Rasch model derived from specific objectivity. Rasch Measurement Transactions, 1(1), 5-6
• Wright, B.D., & Masters, G.N. (1982). Rating Scale Analysis. Chicago, IL: University of Chicago, MESA Press.
• Wright, B.D., & Panchapakesan, N. (1969). A procedure for sample-free item analysis. Educational and Psychological Measurement, 29(1), 23-48.
• Yu, C.H. (2011). A Simple Guide to the Item Response Theory (IRT) and Rasch Modeling. Available at www.creative-wisdom.com/computer/sas/IRT.pdf.
• Zigmond, A.S., & Snaith, R.P. (1983). The Hospital Anxiety and Depression Scale. Acta Psychiatrica Scandinavica, 67(6), 361-370.
• Zung, W.W. (1965). A self-rating depression scale. Archives of General Psychiatry, 12(1), 63-70.