Test Equating with the Rasch Model to Compare Pre-test and Post-test Measurements

Abstract

The purpose of this study is to prove the equitability of pre and post-tests with the Rasch Model and to provide the observability of individual and interindividual ability changes by evaluating the equated tests with stack analysis within the scope of the Rasch Measurement Theory. The pre-test and post-test data that are applied in this study were derived from the project named A Model Proposal to Increase Turkey’s Success in the field of Mathematics in International Large-Scale Exams: Effectiveness of the Cognitive Diagnosis based Monitoring Model No. 115K531, which started on 15/11/2015 and was supported by the TUBİTAK SOBAG 3501 program. The tests were analyzed with the Rasch model, and the fit of the data to the Rasch model was evaluated, and then the Rasch Model and the Separate estimation-Common person method were applied for equating process. Lastly, individual and interindividual ability changes were observed by applying the stack analysis method with the Rasch model. As a result of the analysis of pre and post-tests with the Rasch model, it was concluded that they meet the requirements of the model. As a consequence of the equating process, the equitability of pre-test and post-test was proved, and it was observed that the individual and interindividual ability change could be evaluated by analyzing the pre-test and post-test data with the stack analysis method.

Keywords

• Rasch model
• test equating
• stack analysis

References

1. Angoff, W. H. (1971). Scales, norms and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 508-600). Washington, DC: American Council on Education.
2. Anselmi, P., Vidotto, G., Bettinardi, O., & Bertolotti, G. (2015). Measurement of change in health status with Rasch models. Health and Quality of Life Outcomes, 13(1), 1-7. doi: 10.1186/s12955-014-0197-x
3. Başokçu, O. T., Bardakçı, V., Çakıroğlu, E., Öğretmen, T., Yurdakul, B., & Akyüz, G. (2018). Uluslararası geniş ölçekli sınavlarda Türkiye’nin matematik başarısını arttırabilmek için bir model önerisi: Bilişsel tanıya dayalı izleme modelinin etkililiği (Proje No. SOBAG 3501). Retrieved from https://open.metu.edu.tr/bitstream/handle/11511/50310/TVRnMU56SXk.pdf
4. Cavanagh, R. F. (2012, December). Engagement in classroom learning: Ascertaining the proportion of students who have a balance between what they can do and what they are expected to do. Paper presented at the 2012 Annual International Conference of the Australian Association for Research in Education, Sydney, Australia.
5. Chang, K. C., Wang, J. D., Tang, H. P., Cheng, C. M., & Lin, C. Y. (2014). Psychometric evaluation, using Rasch analysis, of the WHOQOL-BREF in heroin-dependent people undergoing methadone maintenance treatment: Further item validation. Health and Quality of Life Outcomes, 12(1), 1-9. Retrieved from https://link.springer.com/article/10.1186/s12955-014-0148-6
6. Crocker, L., & Algina J. (1986). Introduction to classical and modern test theory. New York: Harcourt Brace Jovanovich College Publishers.
7. Cunningham, J. D., & Bradley, K. D. (2010, May). Applying the Rasch model to measure change in student performance over time. In American Educational Research Association Annual Meeting, Denver, CO.
8. Fujita, T., & Mayekawa, S. I. (2011). A comparison between common item equating with pre-and post-reading and listening tests. In C Ho, M.-F. G. Lin (Eds.), E-Learn: World conference on e-learning in corporate, government, healthcare, and higher education (pp. 626-631). Association for the Advancement of Computing in Education (AACE).

9. Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston, MA: Academic Publishers Group.
10. Herrmann-Abell, C. F., Flanagan, J. C., & Roseman, J. E. (2012, March). Results from a pilot study of a curriculum unit designed to help middle school students understand chemical reactions in living systems (Online Submission). Paper presented at the NARST Annual International Conference, Indianapolis.
11. Kolen, M. J. (1981). Comparison of traditional and item response theory methods for equating tests. Journal of Educational Measurement, 18(1), 1-11. Retrieved from http://www.jstor.org/stable/1434813
12. Kolen, M. J., & Brennan, R. L. (2004). Test equating, scaling, and linking: Methods and practices. New York: Springer.
13. Linacre, J. M. (2016). Winsteps® (Version 3.92. 0) [Computer Software]. Winsteps, Beaverton, OR, USA. Retrieved from www.winsteps.com
14. Ling, M. T., Pang, V., & Ompok, C. C. (2018). Measuring change in early mathematics ability of children who learn using games: Stacked analysis in Rasch measurement. In Q. Zhang (Ed.), Pacific rim objective measurement symposium (proms) 2016 conference proceedings (pp. 215-226). Singapore: Springer.
15. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
16. Masters, G. N. (1985). Common-person equating with the Rasch model. Applied Psychological Measurement, 9(1), 73-82. doi: 10.1177/014662168500900107
17. Popp, S. E. O., & Jackson, J. C. (2009, April). Can assessment of student conceptions of force be enhanced through linguistic simplification? A rasch model common person equating of the FCI and the SFCI. Paper presented at the Annual Meeting of the American Educational Research Association, San Diego, CA.
18. Taylor, W. J., & McPherson, K. M. (2007), Using Rasch analysis to compare the psychometric properties of the Short Form 36 physical function score and the Health Assessment Questionnaire disability index in patients with psoriatic arthritis and rheumatoid arthritis. Arthritis Care & Research, 57(5), 723-729. doi: 10.1002/art.22770
19. Wright, B. D. (1996a). Reliability and separation. Rasch Measurement Transactions, 9(4). Retrieved from https://www.rasch.org/rmt/rmt94n.htm
20. Wright, B. D. (1996b). Time 1 to time 2 (pre-test post-test) comparisons and equating: Racking and stacking. Rasch Measurement Transactions, 10(1). Retrieved from https://www.rasch.org/rmt/rmt101f.htm
21. Wright, B. D. (2003). Rack and Stack: Time 1 vs. time 2 pre-post. Rasch Measurement Transactions, 17(1), 905-906. Retrieved from https://www.rasch.org/rmt/rmt171a.htm