Building On-Demand Test Forms in R

Abstract

Automated Test Assembly (ATA) plays important role in test development, especially in large scale test administrations. However, there is a lack of tutorials showing how to solve ATA problems. This tutorial aims to show to how build on-demand test forms easily for researchers and practitioners, and share the R codes for their use. The study presents the annotated R codes for thirty-nine unique examples. The examples include building one form, multiple forms and more complex ones under different constraint conditions across equal or different form lengths. All examples were solved by using “xxIRT” R package. The graphical depictions of the form-level information functions for all examples were also provided. Some important notes about the codes were also provided at the end of the paper in case one did not find a solution. The thirty-six examples were provided in the main body of the paper, the other three complex examples were given in the Supplementary material.

Keywords

• On-demand test forms,automated test assembly,xxIRT,R

References

1. Berkelaar, M., & others (2015). R Package ‘lpSolve’. CRAN. URL https://cran.r-project.org/web/packages/lpSolve/lpSolve.pdf
2. Berkelaar, M., Eikland, K., & Notebaert, P. (2004). Lp solve 5.5, open source (mixed-integer) linear programming system. URL http://lpsolve.sourceforge.net/5.5/
3. Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick. In statistical theories of mental test scores (pp. 397–479). Reading, MA: Addison-Wesley.
4. Cor, K., Alves, C., & Gierl, M. (2008). Conducting automated test assembly using the Premium Solver Platform Version 7.0 with Microsoft Excel and the large-Scale LP/QP solver engine add-in. Applied Psychological Measurement, 32, 652-663.
5. Cor, K., Alves, C., & Gierl, M. (2009). Three applications of automated test assembly within a user-friendly modeling environment. Practical Assessment, Research, & Evaluation, 14(14). URL http://pareonline.net/getvn.asp?v=14%26n=14
6. Diao, Q., & van der Linden, W. J. (2011). Automated test assembly using lp_solve version 5.5 in R. Applied Psychological Measurement, 35(5), 398-409.
7. Donoghue, J. R. (2015). Comparison of integer programming (IP) solvers for automated test assembly (ATA). ETS Research Report Series, 2015(1), 1-12.
8. Gierl, M.J., Daniels, L., & Zhang, X. (2017). Creating Parallel Forms to Support On-Demand Testing for Undergraduate Students in Psychology. Journal of Measurement and Evaluation in Education and Psychology, 8, 288-302.

9. Han, K. T., & Rudner, L. M. (2014). Item Pool Construction Using Mixed Integer Quadratic Programming (MIQP). GMAC® Research Report RR-14-01. Graduate Management Admission Council. URL https://files.eric.ed.gov/fulltext/ED558455.pdf
10. ILOG. (2006). ILOG CPLEX 10.0 [User’s manual]. Paris, France: ILOG SA.
11. Konis, K., & Konis, M. K. (2016). Package ‘lpSolveAPI’. URL https://cran.r-project.org/web/packages/lpSolveAPI/lpSolveAPI.pdf
12. Luecht, R. M. (1998). Computer-assisted test assembly using optimization heuristics. Applied Psychological Measurement, 22, 224-236.
13. Luo, X. (2018). R Package ‘xxIRT’. CRAN. URL https://cran.r-project.org/web/packages/xxIRT/xxIRT.pdf
14. Van der Linden, W. J. (2006). Linear models for optimal test design. Springer Science & Business Media.