Scale Transformation for Multidimensional Tests According to Unidimensional and Multidimensional Methods

Yaşar Mehmet Zor; Hülya Kelecioğlu

doi:10.21031/epod.1645385

Scale Transformation for Multidimensional Tests According to Unidimensional and Multidimensional Methods

Abstract

In this study, it was aimed to examine the performance of scale transformation methods using MIRT and UIRT. For this purpose, two-dimensional and simple structured test data were generated. The conditions addressed in the study are sample size, interdimensional correlation, common item ratio, difference in ability distribution and parameter estimation model. Mean-mean, mean-sigma and Stocking-Lord methods were used as scale transformation methods and RMSE was taken as an evaluation criterion. RMSE values were estimated as a result of the scale transformations using NEAT design and compared with the independent samples t-test. Lower RMSE values were estimated in unidimensional methods for parameter a, multidimensional methods for parameter b and ability parameter, and statistically significant differences were found in favour of unidimensional methods for parameter a and multidimensional methods for parameter b and ability parameter. Lower RMSE values were obtained in conditions with high sample size and common item ratio. Multidimensional methods performed better when the interdimensional correlation was low, and unidimensional methods performed better when the interdimensional correlation was high. Lower RMSE values were estimated when the difference in ability distribution was low.

Keywords

References

Angoff, W. H. (1984). Scales, norms and equivalent scores. New Jersey: Educational Testing Service.
Atar, B. & Yeşiltaş, G. (2017). Çok boyutlu eşitleme yöntemlerinin eşdeğer olmayan gruplarda ortak madde deseni için performanslarının incelenmesi. Eğitimde ve Psikolojide Ölçme ve DeğerlendirmeDergisi, 8(4), 421-434. https://doi.org/10.21031/epod.335284
Braun, H. I. & Holland, P. W. (1982). Observed-score test equating: A mathematical analysis of some ETS equating procedures. In P. W. Holland and D.B. Rubin (Ed.), Test equating (s. 9-49). New York: Academic Press.
Beguin, A. A. & Hanson, B. A. (2001). Effect of noncompensatory multidimensionality on separate and concurrent estimation in IRT observed score equating. Paper presented at the The Annual Meeting of the National Council on Measurement in Education, Seattle, WA.
Bökeoğlu, Ö., Uçar, A. and Balta, E. (2022). Madde tepki kuramına dayalı gerçek puan eşitlemede ölçek dönüştürme yöntemlerinin incelenmesi. Ankara Üniversitesi Eğitim Bilimleri Fakültesi Dergisi. 55(1), 1-36. https://doi.org/10.30964/auebfd.1001128
Brossman, B. G. (2010). Observed score and true score equating procedures for multidimensional item response theory (Doctoral dissertation). University of Iowa, Iowa.
Chalmers, R. P. (2012). Mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48, 1-29. https://doi.org/10.18637/jss.v048.i06
Chu, K. L. & Kamata, A. (2003). Test equating with the presence of DIF. Paper presented at the annual meeting of American Educational Research Association, Chicago.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum de Ayala, R. J. (2009). The theory and practice of item response theory. New York: Guilford Press.
Draper, N. R., & Smith, H. (1998). Applied regression analysis (3rd ed.). John Wiley & Sons.
Gök, B. & Kelecioğlu, H. (2014). Denk olmayan gruplarda ortak madde deseni kullanılarak madde tepki kuramına dayalı eşitleme yöntemlerinin karşılaştırılması. Mersin Üniversitesi Eğitim Fakültesi Dergisi, 10(1), 120-136. https://doi.org/10.17860/efd.78698
Gübeş, N. Ö. (2014). The Effects of Test Dimensionality, Common Item Format, Ability Distribution and Scale Transformation Methods on Mixed-Format Test Equating (Doctoral dissertation). Hacettepe University, Ankara.
Gübeş, N. Ö. (2019). Test Eşitlemede Çok Boyutluluğun Eş Zamanlı ve Ayrı Kalibrasyona Etkisi. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 34(4), 1061-1074. https://doi.org/10.16986/HUJE.2019049186
Han, K. T. (2009). IRTEQ: Windows application that implements IRT scaling and equating [computer program]. Applied Psychological Measurement, 33(6), 491-493. https://doi.org/10.1177/0146621608319513
Hanson, B. A. & Beguin, A.A. (2002). Obtaining a common scale for item response theory item parameters using separate versus concurrent estimation in the common-item equating design. Applied Psychological Measurement, 26(1), 3-24. https://doi.org/10.1177/0146621602026001001
Kelecioğlu, H. (2024). Test Eşitleme. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Derneği E-Yayınları. Retrieved from https://kitap.epodder.org/test-esitleme
Kilmen, S. & Demirtaşlı, N. (2012). Comparison of test equating methods based on item response theory. Procedia-Social and Behavioral Sciences 46(2012), 130-134. https://doi.org/10.1016/j.sbspro.2012.05.081
Kim, H.Y. (2014). A comparison of smoothing methods for the common item nonequivalent groups design. (Unpublished Doctoral Dissertation). University of Iowa, Iowa
Kim, S. Y. (2018). Simple structure MIRT equating for multidimensional tests (Doctoral Dissertation). University of Iowa, Iowa.
Kim, S., Lee, W. C. and Kolen, M. J. (2020). Simple-Structure Multidimensional Item Response Theory Equating for Multidimensional Tests. Educational and Psychological Measurement. 80(1), 91-125. https://doi.org/10.1177/0013164419854208
Kim, S.Y. and Lee, W.C. (2023), Several Variations of Simple-Structure MIRT Equating. Journal of Educational Measurement, 60: 76-105. https://doi.org/10.1111/jedm.12341
Kolen, M. J. & Brennan, R. L. (2014). Test equating, scaling, and linking: Methods and practices (3rd edition). New York: Springer.
Kumlu, G. (2019). Test ve alt testlerde eşitlemenin farklı koşullar açısından incelenmesi (Doktora tezi). Hacettepe Üniversitesi Eğitim Bilimleri Enstitüsü, Ankara.
Loyd, B. H. & Hoover, H. D. (1980). Vertical equating using the rasch model. Journal of Educational Measurement, 17(3), 179-193. https://doi.org/10.1111/j.1745-3984.1980.tb00825.x
Marco, G. L. (1977). Item characteristic curve solutions to three intractable testing problems. Journal of Educational Measurement, 14(2), 139-160. https://doi.org/10.1111/j.1745-3984.1977.tb00033.x
Min, K. S. (2003). The impact of scale dilation on the quality of the linking of multidimensional item response theory calibrations. Unpublished Dissertation, Michigan State University, East Lansing, MI.
R Core Team. (2021). R: A language and environment for statistical computing: R foundation for statistical computing.
Reckase, M. D. (2009). Multidimensional item response theory. New York: Springer.
Speron, E. (2009). A comparison of metric linking procedures in Item Response Theory (doctoral dissetration). University of Illinois, Chicago, Illinois.
Stocking, M. L. & Lord, F. M. (1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7, 201-210. https://doi.org/10.1177/014662168300700208
Wei, Y. (2008). A simulation study on the performance of four multidimensional IRT scale linking methods. Unpublished doctoral dissertation. University of Florida.
Xu, Y. (2009). Measuring change in jurisdiction achievement over time: Equating issues in current international assessment programs (Doctoral dissertation). University of Toronto, Toronto.
Yao, L. (2009). LinkMIRT: Linking of multivariate item response model. Monterey, CA: Defense Manpower Data Center.
Zhang, J. (2012). Calibration of response data using MIRT models with simple and mixed structures. Applied Psychological Measurement, 36(5), 375-398. https://doi.org/10.1177/0146621612445904

Details

Primary Language

English

Subjects

Statistical Analysis Methods, Item Response Theory, Measurement Equivalence, Test Theories, Testing, Assessment and Psychometrics (Other)

Journal Section

Research Article

Authors

Yaşar Mehmet Zor ^*
0000-0002-4149-5981
Türkiye

Hülya Kelecioğlu
0000-0002-0741-9934
Türkiye

Publication Date

July 2, 2026

Submission Date

February 24, 2025

Acceptance Date

April 26, 2026

Published in Issue

Year 2026 Volume: 17 Number: 2

DOI

https://doi.org/10.21031/epod.1645385

IZ

https://izlik.org/JA44WC33FL

Cite

RIS / Bibtex

APA

Zor, Y. M., & Kelecioğlu, H. (2026). Scale Transformation for Multidimensional Tests According to Unidimensional and Multidimensional Methods. Journal of Measurement and Evaluation in Education and Psychology, 17(2), 83-97. https://doi.org/10.21031/epod.1645385

AMA

1.Zor YM, Kelecioğlu H. Scale Transformation for Multidimensional Tests According to Unidimensional and Multidimensional Methods. JMEEP. 2026;17(2):83-97. doi:10.21031/epod.1645385

Chicago

Zor, Yaşar Mehmet, and Hülya Kelecioğlu. 2026. “Scale Transformation for Multidimensional Tests According to Unidimensional and Multidimensional Methods”. Journal of Measurement and Evaluation in Education and Psychology 17 (2): 83-97. https://doi.org/10.21031/epod.1645385.

EndNote

Zor YM, Kelecioğlu H (July 1, 2026) Scale Transformation for Multidimensional Tests According to Unidimensional and Multidimensional Methods. Journal of Measurement and Evaluation in Education and Psychology 17 2 83–97.

IEEE

[1]Y. M. Zor and H. Kelecioğlu, “Scale Transformation for Multidimensional Tests According to Unidimensional and Multidimensional Methods”, JMEEP, vol. 17, no. 2, pp. 83–97, July 2026, doi: 10.21031/epod.1645385.

ISNAD

Zor, Yaşar Mehmet - Kelecioğlu, Hülya. “Scale Transformation for Multidimensional Tests According to Unidimensional and Multidimensional Methods”. Journal of Measurement and Evaluation in Education and Psychology 17/2 (July 1, 2026): 83-97. https://doi.org/10.21031/epod.1645385.

JAMA

1.Zor YM, Kelecioğlu H. Scale Transformation for Multidimensional Tests According to Unidimensional and Multidimensional Methods. JMEEP. 2026;17:83–97.

MLA

Zor, Yaşar Mehmet, and Hülya Kelecioğlu. “Scale Transformation for Multidimensional Tests According to Unidimensional and Multidimensional Methods”. Journal of Measurement and Evaluation in Education and Psychology, vol. 17, no. 2, July 2026, pp. 83-97, doi:10.21031/epod.1645385.

Vancouver

1.Yaşar Mehmet Zor, Hülya Kelecioğlu. Scale Transformation for Multidimensional Tests According to Unidimensional and Multidimensional Methods. JMEEP. 2026 Jul. 1;17(2):83-97. doi:10.21031/epod.1645385