Monte Carlo Simulation Studies in Item Response Theory with the R Programming Language

Abstract

Monte Carlo simulation studies play an important role in operational and academic research in educational measurement and psychometrics. Item response theory (IRT) is a psychometric area in which researchers and practitioners often use Monte Carlo simulations to address various research questions. Over the past decade, R has been one of the most widely used programming languages in Monte Carlo studies. R is a free, open-source programming language for statistical computing and data visualization. Many user-created packages in R allow researchers to conduct various IRT analyses (e.g., item parameter estimation, ability estimation, and differential item functioning) and expand these analyses to comprehensive simulation scenarios where the researchers can investigate their specific research questions. This study aims to introduce R and demonstrate the design and implementation of Monte Carlo simulation studies using the R programming language. Three IRT-related Monte Carlo simulation studies are presented. Each simulation study involved a Monte Carlo simulation function based on the R programming language. The design and execution of the R commands is explained in the context of each simulation study.

Keywords

• psikometri,ölçme,MTK,simülasyon,R

R Programlama Dili ile Madde Tepki Kuramında Monte Carlo Simülasyon Çalışmaları

Abstract

.Eğitimde ölçme ve psikometri alanlarında yapılan akademik ve uygulamaya dönük araştırmalarda Monte Carlo simülasyon çalışmaları önemli bir rol oynamaktadır. Psikometrik çalışmalarda araştırmacıların Monte Carlo simülasyonlarına sıklıkla başvurduğu temel konulardan birisi Madde Tepki Kuramı’dır (MTK). Geçtiğimiz son on yılda MTK ile ilgili yapılan simülasyon çalışmalarında R'ın sıklıkla kullanıldığı görülmektedir. R istatiksel hesaplama ve görsel üretme için kullanılan ücretsiz ve açık kaynak bir programlama dilidir. R kullanıcıları tarafından üretilen birçok paket program ile madde parametrelerini kestirme, madde yanlılık analizleri gibi birçok MTK temelli analiz yapılabilmektedir. Bu çalışma, R programına dair giriş niteliğinde bilgiler vermek ve R programlama dili MTK temelli Monte Carlo simülasyon çalışmalarının nasıl yapılabileceğini göstermeyi amaçlamaktadır. R programlama dilini örneklerle açıklamak için üç farklı Monte Carlo simülasyon çalışması gösterilmektedir. Her bir çalışmada, simülasyon içerisindeki R komutları ve fonksiyonları MTK kapsamında açıklanmaktadır.

Keywords

• psychometrics,measurement,IRT,simulation,R

References

1. Albano, A. D. (2016). equate: An R package for observed-score linking and equating. Journal of Statistical Software, 74(8), 1–36.
2. Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York: Guilford.
3. Browne, M., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.). Testing structural equation models. Newbury Park, CA: Sage.
4. Bulut, O. (2013). Between-person and within-person subscore reliability: Comparison of unidimensional and multidimensional IRT models. (Unpublished doctoral dissertation). University of Minnesota, Minneapolis, MN.
5. Bulut, O. (2015). Applying item response theory models to Entrance Examination for Graduate Studies: Practical issues and insights. Journal of Measurement and Evaluation in Education and Psychology, 6(2), 313–330.
6. Cai, L. (2013). flexMIRT version 2.00: A numerical engine for flexible multilevel multidimensional item analysis and test scoring [Computer software]. Chapel Hill, NC: Vector Psychometric Group.
7. Cai, L., Thissen, D., & du Toit, S. H. C. (2011). IRTPRO: Flexible, multidimensional, multiple categorical IRT modeling [Computer software]. Lincolnwood, IL: Scientific Software International.
8. Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29.

9. Chalmers, R. P. (2016). Generating adaptive and non-adaptive test interfaces for multidimensional item response theory applications. Journal of Statistical Software, 71(5), 1–39.
10. Choi, S. W., Gibbons, L. E., & Crane, P. K. (2016). lordif: Logistic ordinal regression differential item functioning using IRT [Computer software]. Available from https://CRAN.R-project.org/package=lordif.
11. Feinberg, R. A., & Rubright, J. D. (2016). Conducting simulation studies in psychometrics. Educational Measurement: Issues and Practice, 35(2), 36–49.
12. Finch, H. (2005). The MIMIC model as a method for detecting DIF: Comparison with Mantel-Haenszel, SIBTEST, and the IRT likelihood ratio. Applied Psychological Measurement, 29, 278–295.
13. Hallgren, K. A. (2013). Conducting simulation studies in the R programming environment. Tutorials in Quantitative Methods for Psychology, 9(2), 43–60.
14. Hambleton, R. K., Swaminathan, H., & Rogers, J. H. (1991). Fundamentals of item response theory. New York: Sage publications.
15. Harwell, M., Stone, C. A., Hsu, T., & Kirisci, L. (1996). Monte Carlo studies in item response theory. Applied Psychological Measurement, 20(2), 101–125.
16. Hu, L.T., & Bentler, P.M. (1999). Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.
17. Jiang, S., Wang, C., & Weiss, D. J. (2016). Sample size requirements for estimation of item parameters in the multidimensional graded response model. Frontiers in Psychology, 7(109), 1–10.
18. Lee, S., Bulut, O., & Suh, Y. (2016). Multidimensional extension of multiple indicators multiple causes models to detect DIF. Educational and Psychological Measurement. Advance online publication. doi: 10.1177/0013164416651116.
19. Mair, P., Hatzinger, R., & Maier M. J. (2016). eRm: Extended Rasch modeling [Computer software]. Available from http://CRAN.R-project.org/package=eRm.
20. Magis, D., Beland, S., Tuerlinckx, F., & De Boeck, P. (2010). A general framework and an R package for the detection of dichotomous differential item functioning. Behavior Research Methods, 42, 847–862.
21. Magis, D., & Raiche, G. (2012). Random generation of response patterns under computerized adaptive testing with the R Package catR. Journal of Statistical Software, 48(8), 1–31.
22. Muthén, L. K., & Muthén, B. O. (1998-2015). Mplus User’s Guide. Seventh Edition. Los Angeles, CA: Muthén & Muthén.
23. Partchev, I. (2016). irtoys: A collection of functions related to item response theory (IRT) [Computer software]. Available from https://CRAN.R-project.org/package=irtoys.
24. R Core Team (2017). R: A language and environment for statistical computing [Computer software]. Vienna, Austria: R Foundation for Statistical Computing. Available from https://cran.r-project.org/. Raju, N. S., van der Linden, W. J., & Fleer, P. F., (1995). An IRT-based internal measure of test bias with application of differential item functioning. Applied Psychological Measurement, 19, 353–368.
25. Revolution Analytics, & Weston, S. (2015). doParallel: Foreach parallel adaptor for the 'parallel' package [Computer software]. Available from https://CRAN.R-project.org/package=doParallel.
26. Rizopoulos, D. (2006). ltm: An R package for latent variable modelling and item response theory analyses. Journal of Statistical Software, 17(5), 1–25.
27. Robitzsch, A. (2017). sirt: Supplementary item response theory models [Computer software]. Available from https://CRAN.R-project.org/package=sirt.
28. Robitzsch, A. & Rupp, A. A. (2009). The impact of missing data on the detection of differential item functioning. Educational Psychological Measurement, 69, 18–34.
29. Rusch, T., Mair, P., & Hatzinger, R. (2013). Psychometrics with R: A review of CRAN packages for item response theory. Retrieved from http://epub.wu.ac.at/4010/1/resrepIRThandbook.pdf
30. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores (Psychometric Monograph No. 17). Richmond, VA: Psychometric Society.
31. SAS Institute Inc. (2014). Version 9.4 of the SAS system for Windows. Cary, NC: SAS Institute Inc.
32. Shealy, R., & Stout, W. (1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DIF as well as item bias/DTF. Psychometrika, 58, 159–194.
33. Sinharay, S., Stern, H. S., & Russell, D. (2001). The use of multiple imputation for the analysis of missing data. Psychological Methods, 6, 317–329.
34. StataCorp. (2015). Stata statistical software: Release 14. College Station, TX: StataCorp LP.
35. Stout, W., Li, H., Nandakumar, R., & Bolt, D. (1997). MULTISIB – A procedure to investigate DIF when a test is intentionally multidimensional. Applied Psychological Measurement, 21, 195–213.
36. Suh, Y., & Cho, S.-J. (2014). Chi-square difference tests for detecting functioning in a multidimensional IRT model: A Monte Carlo study. Applied Psychological Measurement, 38, 359–375.
37. Ünlü, A., & Yanagida, T. (2011). R you ready for R?: The CRAN psychometrics task view. British Journal of Mathematical and Statistical Psychology, 64(1), 182–186.
38. Venables, W. N., & Ripley, B. D. (2002). Modern applied statistics with S. New York, NY: Springer.
39. Woods, C. M., & Grimm, K. J. (2011). Testing for nonuniform differential item functioning with multiple indicator multiple cause models. Applied Psychological Measurement, 35, 339–361.
40. Yao, L. (2003). BMIRT: Bayesian multivariate item response theory [Computer software]. Monterey, CA: Defense Manpower Data Center. Available from http://www.bmirt.com.