A novel approach for calculating the item discrimination for Likert type of scales

Abstract

Item analysis is performed by developers as an integral part of the scale development process. Thus, items are excluded from the scale depending on the item analysis prior to the factor analysis. Existing item discrimination indices are calculated based on correlation, yet items with different response patterns are likely to have a similar item discrimination index. This study proposed a new item discrimination index that can be used in Likert type of scales and examined its effect on factor analysis results. For this purpose, simulative datasets were generated, and items were excluded from the analysis according to the .20, .30 and .35 item discrimination index criteria, and exploratory factor analysis was performed for a single factor. Accordingly, it was found that more variance could be explained by a single factor with fewer items compared to other discrimination indices when the .20 criterion of the slope coefficient was used as suggested in this study. Similar findings were obtained using the .35 criterion with other discrimination indices. In this context, it is recommended to use the slope coefficient as an additional discrimination index calculation method in the scale development process.

Keywords

• Item discrimination index
• Likert-type scale
• Exploratory factor analysis
• Slope coefficient
• Monte-Carlo simulation

References

1. Beuchert, A.K., & Mendoza, J.L. (1979). A Monte Carlo comparison of ten item discrimination indices. Journal of Educational Measurement, 16(2), 109 117. http://www.jstor.org/stable/1434454
2. Crocker, L., & Algina, J. (2008). Introduction to Classical and Modern Test Theory. Cengage Learning.
3. Cum, S. (2021). Examining the discrimination of binary scored test items with ROC analysis. International Journal of Assessment Tools in Education, 8(4), 948-958. https://doi.org/10.21449/ijate.894851
4. Engelhart, M.D. (1965). A comparison of several item discrimination indices. Journal of Educational Measurement, 2, 69-76.https://doi.org/10.1111/j.1745-3984.1965.tb00393.x
5. Johnson, A.P. (1951). Notes on a suggested index of item validity: The U-L index. Journal of Educational Psychology, 42(8), 499-504. https://doi.org/10.1037/h0060855
6. Kelley, T.L. (1939). The selection of upper and lower groups for the validation of test items. Journal of Educational Psychology, 30(1), 17-24. https://doi.org/10.1037/h0057123
7. Livingston, S.A., & Dorans, N.J. (2004). A Graphical Approach to Item Analysis. ETS Research Report.
8. Magis, D., & Raiche, G. (2012). Random generation of response patterns under computerized adaptive testing with the R package catR. Journal of Statistical Software, 48(8), 1-31. https://doi.org/10.18637/jss.v048.i08

9. Martinkova, P., & Drabinova, A. (2018). ShinyItemAnalysis for teaching psychometrics and to enforce routine analysis of educational tests. The R Journal, 10(2), 503-515. https://doi.org/10.32614/RJ-2018-074
10. Moses, T. (2017). A review of developments and applications in item analysis. In Advancing Human Assessment (Eds. R. E. Bennett & M. von Davier). Springer Open.
11. Oosterhof, A.C. (1976). Similarity of various item discrimination indices. Journal of Educational Measurement, 13(2), 145-150. http://www.jstor.org/stable/1434235
12. Popham, W. J. (2014). Classroom Assessment: What teachers need to know? Pearson.
13. R Core Team (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL: https://www.R-project.org/
14. Revelle, W. (2022) psych: Procedures for Personality and Psychological Research, Northwestern University, Evanston, Illinois, USA URL: https://CRAN.R-project.org/package=psych
15. Tabachnick, B.G., & Fidell, L.S. (2013). Using Multivariate Statistics. Pearson.
16. Wickham, H. (2016). ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag.
17. Wickham, H., Francois, R., Henry, L., & Müller, K. (2022). dplyr: A Grammar of Data Manipulation. R package version 1.0.9 URL: https://CRAN.R-project.org/package=dplyr
18. Vonglao, P. (2017). Application of fuzzy logic to improve the Likert scale to measure latent variables. Kasetsart Journal of Social Sciences, 38(3), 337 344. https://doi.org/10.1016/j.kjss.2017.01.002