R Ortamında Çok Boyutlu Aşamalı Tepki Modeli Kullanılarak Madde Parametrelerinin Geri Kazanımı

Öz

Bu çalışmanın amacı, R ortamında Çok Boyutlu Aşamalı Tepki Modeli (ÇBATM) için madde parametrelerinin geri kazanım sürecini adım adım göstermektir. Bu kapsamda, test uzunluğunun 20 ve 40 olduğu, boyutlar arası korelasyonların ,3 ve ,7 olarak belirlendiği ve örneklem büyüklüğünün 2000 olarak sabit tutulduğu bir simülasyon deseni oluşturulmuştur. Üç boyutlu ATM için üretilen veri setlerinden parametre tahminleri elde edilmiş, yanlılık (bias) ve Kök Ortalama Kare Hata (Root Mean Square Error – RMSE) değerleri hesaplanarak görselleştirilmiştir. Çalışmanın amacı doğrultusunda, sürecin tüm aşamalarına ilişkin R kodları ayrıntılı açıklamalarla birlikte sunulmuş ve araştırmacıların bu işlemleri yeniden üretebilmesine ve kendi çalışmalarına uyarlayabilmesine olanak sağlanmıştır. Bu çalışmanın, ÇBATM kapsamında madde parametrelerinin geri kazanımının uygulanmasına yönelik pratik bir rehber sunarak alanyazına katkı sağlaması ve veri üretimi, parametre tahmini ve görselleştirme yöntemleri bakımından araştırmacılara yarar sağlaması beklenmektedir.

Anahtar Kelimeler

• Çok boyutlu aşamalı tepki modeli
• çok kategorili modeller
• madde parametrelerinin geri kazanımı
• Monte Carlo simülasyonu
• veri üretimi
• R programlama dili.

Etik Beyan

Çalışma simülasyon çalışması olup, etik kurul onayı gerektirmemektedir.

Item Parameter Recovery Using the Multidimensional Graded Response Model in R

Öz

The purpose of this study is to provide a step-by-step demonstration of item recovery for the Multidimensional Graded Response Model (MGRM) using R. For this purpose, an illustrative simulation setup was employed, including test lengths of 20 and 40 items, interdimensional correlations of 0.3 and 0.7, and a sample size of 2000. Parameter estimates were obtained from datasets generated under the MGRM, and bias and Root Mean Square Error (RMSE) values were calculated and visualized as part of the demonstration process. Rather than evaluating recovery performance across conditions, the study focuses on explaining each stage of the analysis, data generation, parameter estimation, evaluation metrics, and visualization, through clearly annotated R code. By providing reproducible and accessible implementations, the study aims to serve as a practical guide for researchers interested in conducting item parameter recovery analyses within the MGRM framework. In addition, the presented workflow may be useful for researchers seeking guidance on specific steps such as data generation, estimation, or visualization.

Anahtar Kelimeler

• Multidimensional graded response model
• polytomous models
• item recovery
• Monte Carlo simulation
• data generation
• R language

Etik Beyan

This study is a simulation study and does not require ethical committee approval.

Kaynakça

1. Ackerman, T. (1996). Graphical representation of multidimensional item response theory analyses. Applied Psychological Measurement, 20(4), 311-329. https://doi.org/10.1177/014662169602000402
2. Ackerman, T.A., Gierl, M.J., & Walker, C.M. (2003). Using multidimensional item response theory to evaluate educational and psychological tests. Educational Measurement: Issues and Practice, 22(3), 4-54. https://doi.org/10.1111/j.1745-3992.2003.tb00136.x
3. Adams, R. J., Wilson, M., & Wang, W. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21(1), 1-23. https://doi.org/10.1177/0146621697211001
4. Alarcon, G. M., Lee, M. A., & Johnson, D. (2023). A Monte Carlo study of IRTree models’ ability to recover item parameters. Frontiers in Psychology, 14, 1003756. https://doi.org/10.3389/fpsyg.2023.1003756
5. Andrich, D. (1978). Application of a psychometric rating model to ordered categories which are scored with successive integers. Applied Psychological Measurement, 2(4), 581–594. https://doi.org/10.1177/014662167800200413
6. Baker, F. B. (2001). The basics of item response theory (2nd ed.). ERIC Clearinghouse on Assessment and Evaluation.
7. Bilicioglu Gunes, A. (2024). Acomparison of estimates obtained with different calibration methods in test equating in the presence of item parameter drift (Thesis No: 839827) [Doctoral dissertation, Ankara University]. Council of Higher Education National Thesis Center. https://tez.yok.gov.tr/UlusalTezMerkezi/giris.jsp
8. Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 397-479). Addison-Wesley.

9. Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29-51. https://doi.org/10.1007/BF02291411
10. Bolt, D. M. & Lall, V. F. (2003). Estimation of compensatory and noncompensatory multidimensional item response models using Markov chain Monte Carlo. Applied Psychological Measurement, 27(6), 395-414. https://doi.org/10.1177/0146621603258350
11. Bulut, O., & Sünbül, Ö. (2017). R programlama dili ile madde tepki kuramında monte carlo simülasyon çalışmaları. Journal of Measurement and Evaluation in Education and Psychology, 8(3), 266-287. https://doi.org/10.21031/epod.305821
12. Carle, A., Jaffee, D., Vaughan, N., & Eder, D. (2009). Psychometric properties of three new national survey of student engagement based engagement scales: An item response theory analysis. Research in Higher Education, 50(8), 775-794. https://doi.org/10.1007/s11162-009-9141-z
13. Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1-29. https://doi.org/10.18637/jss.v048.i06
14. Classe, F., & Kern, C. (2024). Detecting differential item functioning in multidimensional graded response models with recursive partitioning. Applied Psychological Measurement, 48(3), 83-103. https://doi.org/10.1177/01466216241238743
15. De Ayala, R. J. (1994). The influence of multidimensionality on the graded response model. Applied Psychological Measurement, 18(2), 155-170. https://doi.org/10.1177/014662169401800205
16. De Ayala, R. J. (2009). The theory and practice of item response theory. Guilford Press. de la Torre, J. (2008). Multidimensional scoring of abilities: The ordered polytomous response case. Applied Psychological Measurement, 32(5), 355-370. https://doi.org/10.1177/0146621607303784
17. de la Torre, J. (2009). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34(1), 115-130. https://doi.org/10.3102/1076998607309474
18. de la Torre, J., Hong, Y., & Deng, W. (2010). Factors affecting the item parameter estimation and classification accuracy of the DINA model. Journal of Educational Measurement, 47(2), 227-249. https://doi.org/10.1111/j.1745-3984.2010.00110.x
19. de la Torre, J. & Patz, R.J. (2005). Making the most of what we have: a practical application of multidimensional item response theory in test scoring. Journal of Educational and Behavioral Statistics, 30(3), 295-311. https://doi.org/10.3102/10769986030003295
20. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates. Feinberg, R. A., & Rubright, J. D. (2016). Conducting simulation studies in psychometrics. Educational Measurement: Issues and Practice, 35(2), 36-49. https://doi.org/10.1111/emip.12111
21. French, B. F., & Vo, T. T. (2020). Differential item functioning of a truancy assessment. Journal of Psychoeducational Assessment, 38(5), 642-648. https://doi.org/10.1177/0734282919863215
22. Furlow, C. F., Raiford Ross, T., & Gagné, P. (2009). The impact of multidimensionality on the detection of differential bundle functioning using simultaneous item bias test. Applied Psychological Measurement, 33(6), 441-464. https://doi.org/10.1177/0146621609331959
23. Genz, A., & Bretz, F. (2009). Computation of multivariate normal and t probabilities (Vol. 195). Springer Science & Business Media.
24. Guo, W., & Choi, Y. J. (2023). Assessing dimensionality of IRT models using traditional and revised parallel analyses. Educational and Psychological Measurement, 83(3), 609-629. https://doi.org/10.1177/00131644221111838
25. Gurdil, H., & Demir, E. (2024). The use of multidimensional item response theory estimations in controlling differential item functioning. Measurement: Interdisciplinary Research and Perspectives, 1-14. https://doi.org/10.1080/15366367.2024.2359450
26. Gürdil, H., Soğuksu, Y. B., Salihoglu, S., & Coşkun, F. (2025). Eğitimde ölçmede yapay zekanın entegrasyonu: Madde tepki kuramı kapsamında veri üretiminde ChatGPT’nin etkililiği. Trakya Eğitim Dergisi, 15(2), 887-918. https://doi.org/10.24315/tred.1509299
27. Hallgren, K. A. (2013). Conducting simulation studies in the R programming environment. Tutorials in Quantitative Methods for Psychology, 9(2), 43-60. https://doi.org/10.20982/tqmp.09.2.p043
28. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Sage. Han, K. C. T., & Paek, I. (2014). A review of commercial software packages for multidimensional IRT modeling. Applied Psychological Measurement, 38(6), 486-498. https://doi.org/10.1177/0146621614536770
29. Hartig, J., & Höhler, J. (2009). Multidimensional IRT models for the assessment of competencies. Studies in Educational Evaluation, 35, 57-63. https://doi.org/10.1016/j.stueduc.2009.10.002
30. Harwell, M., Stone, C. A., Hsu, T., & Kirisci, L. (1996). Monte Carlo studies in item response theory. Applied Psychological Measurement, 20(2), 101-125. https://doi.org/10.1177/014662169602000201
31. Hattie, J. (1985). Methodology review: Assessing unidimensionality of tests and items. AppliedPsychological Measurement, 9, 139-164. https://doi.org/10.1177/014662168500900204
32. Jiang, S., Wang, C., & Weiss, D. J. (2016). Sample size requirements for estimation of item parameters in the multidimensional graded response model. Frontiers in Psychology, 7, 109. https://doi.org/10.3389/fpsyg.2016.00109
33. Kehinde, O. J., Dai, S., & French, B. (2022). Item parameter estimations for multidimensional graded response model under complex structures. Frontiers in Education, 7. https://doi.org/10.3389/feduc.2022.947581
34. Kroese, D. P., Brereton, T., Taimre, T., & Botev, Z. I. (2014). Why the Monte Carlo method is so important today. Wiley Interdisciplinary Reviews: Computational Statistics, 6(6), 386-392. https://doi.org/10.1002/wics.1314
35. Kuo, T. C., & Sheng, Y. (2015). Bayesian estimation of a multi-unidimensional graded response IRT model. Behaviormetrika, 42(2), 79-94. https://doi.org/10.2333/bhmk.42.79
36. Kuo, T. C., & Sheng, Y. (2016). A comparison of estimation methods for a multi-unidimensional graded response IRT model. Frontiers in Psychology, 7, 880. https://doi.org/10.3389/fpsyg.2016.00880
37. Lord, F. M. (1952). A theory of test scores (Psychometric Monograph No. 7). Psychometric Corporation. http://www.psychometrika.org/journal/online/MN07.pdf
38. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Routledge. https://doi.org/10.4324/9780203056615
39. Meijer, R. R., & Tendeiro, J. N. (2018). Unidimensional item response theory. In P. Irwing, T. Booth, & D. J. Hughes (Eds.), The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development (pp. 413-443). Wiley Blackwell. https://doi.org/10.1002/9781118489772.ch15
40. Muraki, E., & Carlson, J. E. (1995). Full-information factor analysis for polytomous item responses. Applied Psychological Measurement, 19(1), 73-90. https://doi.org/10.1177/014662169501900109
41. Reckase, M. D. (2009). Multidimensional item response theory. Springer.
42. Reise, S. P., & Yu, J. (1990). Parameter recovery in the graded response model using MULTILOG. Journal of Educational Measurement, 27(2), 133-144. https://doi.org/10.1111/j.1745-3984.1990.tb00738.x
43. Ren, K. (2023). rlist: A toolbox for non-tabular data manipulation (R package version 0.4.6.2). https://doi.org/10.32614/CRAN.package.rlist
44. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores (Psychometric Monograph No. 17). Psychometric Society. http://www.psychometrika.org/journal/online/MN17.pdf
45. Sharkness, J., & DeAngelo, L. (2011). Measuring student involvement: A comparison of classical test theory and item response theory in the construction of scales from student surveys. Research in Higher Education, 52, 480-507. https://doi.org/10.1007/s11162-010-9202-3
46. Sijtsma, K., & Junker, B.W. (2006). Item response theory: Past performance, present developments and future expectations. Behaviormetrika, 1, 75-102. https://doi.org/10.2333/bhmk.33.75
47. Spearman, C. (1904). The proof and measurement of association between two things. American Journal of Psychology, 15, 72-101. https://doi.org/10.2307/1412159
48. Strachan, T., Ip, E., Fu, Y., Ackerman, T., Chen, S.-H., & Willse, J. (2020). Robustness of projective IRT to misspecification of the underlying multidimensional model. Applied Psychological Measurement, 44(5), 362-375. https://doi.org/10.1177/0146621620909894
49. Thissen, D.M. (1976). Information in wrong responses to the raven progressive matrices. Journal of Educational Measurement, 13, 201-214. https://doi.org/10.1111/j.1745-3984.1976.tb00011.x
50. Uttaro, T., & Lehman, A. (1999). Graded response modeling of the Quality of Life Interview. Evaluation and Program Planning, 22(1), 41-52. https://doi.org/10.1016/S0149-7189(98)00039-1
51. Walker, C. M., & Gocer Sahin, S. (2016). Using a multidimensional IRT framework to better understand differential item functioning (DIF): A tale of three DIF detection procedures. Educational and Psychological Measurement, 77(6), 945-970. https://doi.org/10.1177/0013164416657137
52. Wickham, H. (2016). ggplot2: Elegant graphics for data analysis. Springer-Verlag. https://ggplot2.tidyverse.org Wickham, H., & Bryan, J. (2025). readxl: Read Excel files (R package version 1.4.5)
53. Xu, J., Paek, I., & Xia, Y. (2017). Investigating the behaviors of M2 and RMSEA2 in fitting a unidimensional model to multidimensional data. Applied Psychological Measurement, 41(8), 632-644. https://doi.org/10.1177/0146621617710464
54. Yao, L., & Schwarz, R. D. (2006). A multidimensional partial credit model with associated item and test statistics: An application to mixed-format tests. Applied Psychological Measurement, 30(6), 469-492. https://doi.org/10.1177/0146621605284537
55. Zhang, J. (2012). Calibration of response data using MIRT models with simple and mixed structures. Applied Psychological Measurement, 36(5), 375-398. https://doi.org/10.1177/0146621612445904