Research Article
BibTex RIS Cite

Automating Simulation Research for Item Response Theory using R

Year 2018, Volume: 5 Issue: 4, 682 - 700, 16.12.2018
https://doi.org/10.21449/ijate.472185

Abstract

A simulation study is a useful tool in examining how validly item response theory (IRT) models can be applied in various settings. Typically, a large number of replications are required to obtain the desired precision. However, many standard software packages in IRT, such as MULTILOG and BILOG, are not well suited for a simulation study requiring a large number of replications because they were developed as a stand-alone software package that is best suited for a single run. This article demonstrated how built-in R functions can be used to automate the simulation study using the stand-alone software packages in IRT. For a demonstration purpose, MULTILOG was used in the example codes in the appendices, but the overall framework of a simulation study and the built-in R functions used in this article can be applied for a simulation study using other stand-alone software packages as well.

References

  • Bandalos, D. L. (2006). The use of monte carlo studies in structural equation modeling research. In Structural equation modeling: A second course (pp. 385–426).
  • Greenwich, CT: Information Age.
  • De Ayala, R. J. (2009). Theory and practice of item response theory. New York, NY: The Guilford Press. Finch, H. (2008). Estimation of item response theory parameters in the presence of missing data. Journal of Educational Measurement, 45, 225–245.
  • Friedl, J. (2006). Mastering regular expressions. Sebastopol, CA: O’Reilly Media, Inc.
  • Harwell, M., Stone, C. A., Hsu, T.-C., & Kirisci, L. (1996). Monte carlo studies in item response theory. Applied Psychological Measurement, 20, 101–125.
  • Kim, H. J., Brennan, R. L., & Lee, W. C. (2017). Structural Zeros and Their Implications With Log‐Linear Bivariate Presmoothing Under the Internal‐Anchor Design. Journal of Educational Measurement, 54, 145-164.
  • Kim, K. Y., & Lee, W. C. (2017). The Impact of Three Factors on the Recovery of Item Parameters for the Three-Parameter Logistic Model. Applied Measurement in Education, 30, 228-242.
  • Kim, S., & Lee, W. C. (2006). An Extension of Four IRT Linking Methods for Mixed‐Format Tests. Journal of Educational Measurement, 43, 53-76.
  • Nader, I. W., Tran, U. S., & Voracek, M. (2015). Effects of Initial Values and Convergence Criterion in the Two-Parameter Logistic Model When Estimating the Latent Distribution in BILOG-MG 3. PloS one, 10, e0140163.
  • Partchev, I. (2009). irtoys: Simple interface to the estimation and plotting of irt models. R package version 0.1, 2.
  • R Core Team. (2015). R: A language and environment for statistical computing [Computer software manual]. Vienna, Austria. Retrieved from http://www.R-project.org/ (ISBN 3-900051-07-0)
  • Reckase, M. D. (1979). Unifactor latent trait models applied to multifactor tests: Results and implications. Journal of Educational and Behavioral Statistics, 4, 207–230.
  • Spector, P. (2008). Data manipulation with r. New York, NY: Springer.
  • Stone, C. A. (2000). Monte Carlo based null distribution for an alternative goodness‐of‐fit test statistic in IRT models. Journal of Educational Measurement, 37, 58-75.
  • Thissen, D., Chen, W.-H., & Bock, R. D. (2003). Multilog 7 for windows: Multiple-category item analysis and test scoring using item response theory [computer software]. lincolnwood, il: Scientific software international. IL: Scientific Software International.
  • Zimowski, M. F., Muraki, E., Mislevy, R. J., & Bock, R. D. (1996). Bilog-mg: Multiple-group irt analysis and test maintenance for binary items. Chicago: Scientific Software International, 4(85), 10.

Automating Simulation Research for Item Response Theory using R

Year 2018, Volume: 5 Issue: 4, 682 - 700, 16.12.2018
https://doi.org/10.21449/ijate.472185

Abstract

A simulation study is a useful tool in examining
how validly item response theory (IRT) models can be applied in various settings.
Typically, a large number of replications are required to obtain the desired precision.
However, many standard software packages in IRT, such as MULTILOG and BILOG, are
not well suited for a simulation study requiring a large number of replications
because they were developed as a stand-alone software package that is best suited
for a single run. This article demonstrated how built-in R functions can be used
to automate the simulation study using the stand-alone software packages in IRT.
For a demonstration purpose, MULTILOG was used in the example codes in the appendices,
but the overall framework of a simulation study and the built-in R functions used
in this article can be applied for a simulation study using other stand-alone software
packages as well.

References

  • Bandalos, D. L. (2006). The use of monte carlo studies in structural equation modeling research. In Structural equation modeling: A second course (pp. 385–426).
  • Greenwich, CT: Information Age.
  • De Ayala, R. J. (2009). Theory and practice of item response theory. New York, NY: The Guilford Press. Finch, H. (2008). Estimation of item response theory parameters in the presence of missing data. Journal of Educational Measurement, 45, 225–245.
  • Friedl, J. (2006). Mastering regular expressions. Sebastopol, CA: O’Reilly Media, Inc.
  • Harwell, M., Stone, C. A., Hsu, T.-C., & Kirisci, L. (1996). Monte carlo studies in item response theory. Applied Psychological Measurement, 20, 101–125.
  • Kim, H. J., Brennan, R. L., & Lee, W. C. (2017). Structural Zeros and Their Implications With Log‐Linear Bivariate Presmoothing Under the Internal‐Anchor Design. Journal of Educational Measurement, 54, 145-164.
  • Kim, K. Y., & Lee, W. C. (2017). The Impact of Three Factors on the Recovery of Item Parameters for the Three-Parameter Logistic Model. Applied Measurement in Education, 30, 228-242.
  • Kim, S., & Lee, W. C. (2006). An Extension of Four IRT Linking Methods for Mixed‐Format Tests. Journal of Educational Measurement, 43, 53-76.
  • Nader, I. W., Tran, U. S., & Voracek, M. (2015). Effects of Initial Values and Convergence Criterion in the Two-Parameter Logistic Model When Estimating the Latent Distribution in BILOG-MG 3. PloS one, 10, e0140163.
  • Partchev, I. (2009). irtoys: Simple interface to the estimation and plotting of irt models. R package version 0.1, 2.
  • R Core Team. (2015). R: A language and environment for statistical computing [Computer software manual]. Vienna, Austria. Retrieved from http://www.R-project.org/ (ISBN 3-900051-07-0)
  • Reckase, M. D. (1979). Unifactor latent trait models applied to multifactor tests: Results and implications. Journal of Educational and Behavioral Statistics, 4, 207–230.
  • Spector, P. (2008). Data manipulation with r. New York, NY: Springer.
  • Stone, C. A. (2000). Monte Carlo based null distribution for an alternative goodness‐of‐fit test statistic in IRT models. Journal of Educational Measurement, 37, 58-75.
  • Thissen, D., Chen, W.-H., & Bock, R. D. (2003). Multilog 7 for windows: Multiple-category item analysis and test scoring using item response theory [computer software]. lincolnwood, il: Scientific software international. IL: Scientific Software International.
  • Zimowski, M. F., Muraki, E., Mislevy, R. J., & Bock, R. D. (1996). Bilog-mg: Multiple-group irt analysis and test maintenance for binary items. Chicago: Scientific Software International, 4(85), 10.
There are 16 citations in total.

Details

Primary Language English
Subjects Studies on Education
Journal Section Articles
Authors

Sunbok Lee This is me 0000-0002-0924-7056

Youn-jeng Choi This is me

Allan S. Cohen This is me

Publication Date December 16, 2018
Submission Date August 22, 2018
Published in Issue Year 2018 Volume: 5 Issue: 4

Cite

APA Lee, S., Choi, Y.-j., & Cohen, A. S. (2018). Automating Simulation Research for Item Response Theory using R. International Journal of Assessment Tools in Education, 5(4), 682-700. https://doi.org/10.21449/ijate.472185

Cited By

Computer Adaptive Testing Simulations in R
International Journal of Assessment Tools in Education
BAŞAK ERDEM KARA
https://doi.org/10.21449/ijate.621157

23824         23823             23825