Linking high-stakes test forms without common items: The pseudo-equivalent item approach in item pool development

Selma Şenel; Sebahat Gören; Serkan Arıkan; Eren Can Aybek; Tuba Gündüz; Ayşegül Bozdağ Kasap

Linking high-stakes test forms without common items: The pseudo-equivalent item approach in item pool development

Abstract

This study evaluates pseudo-equivalent items as an alternative method for linking high-stakes test forms when common items are unavailable. The findings offer valuable insights for developing sustainable item pools, particularly within Computerized Adaptive Testing (CAT) frameworks. Data were obtained from a national foreign language test administered across six consecutive years. Separate annual IRT calibrations were compared against simultaneous calibration results under a multiple matrix sampling (MMS) design, using fourteen pseudo-equivalent items as anchor indicators. The 2PL model was determined to be the most reliable model for the data. Difficulty parameters from the separate and simultaneous calibrations correlated very highly, with no statistically significant difference between the two sets, and difficulty estimates from Classical Test Theory and IRT were consistently and strongly related, as theoretically expected. Measurement precision was concentrated around average ability levels. The findings provide empirical evidence that pseudo-equivalent items represent a promising approach for linking forms without common items, and that combining this approach with an MMS design provides a feasible strategy for building large, sustainable item pools suited to large-scale and CAT-based testing programs. These results have direct implications for testing organizations seeking to maintain form equivalence and scale stability across administrations without relying on traditional common-item designs, offering a scalable pathway for item pool expansion that preserves measurement comparability over time.

Keywords

Supporting Institution

Measurement, Selection, and Placement Center (ÖSYM)

Thanks

We would like to express our sincere thanks to Dr. Mustafa Tosun for his software development support as a member of the project team, and to Dr. Seçil Alaca for her valuable contribution through her expertise in English items.

References

American Educational Research Association [AERA], American Psychological Association [APA], & National Council on Measurement in Education [NCME]. (2014). Standards for educational and psychological testing. American Educational Research Association.
Ames, A.J., & Penfield, R.D. (2015). An NCME instructional module on item-fit statistics for item response theory models. Educational Measurement: Issues and Practice, 34(3), 39 48. https://doi.org/10.1111/emip.12067
Aybek, E.C., Arıkan, S., & Ertaş, G. (2024). A practical guide to item bank calibration with multiple matrix sampling. International Journal of Assessment Tools in Education, 11(4), 647 659. https://doi.org/10.21449/ijate.1440316
Baldwin, P., & Clauser, B.E. (2022). Historical perspectives on score comparability issues raised by innovations in testing. Journal of Educational Measurement, 59(2), 140 160. https://doi.org/10.1111/jedm.12318
de Ayala, R.J. (2009). The theory and practice of item response theory. Guilford Press.
Dorans, N.J., & Haberman, S.J. (2022). Recent challenges to maintaining score comparability: A commentary. Journal of Educational Measurement, 59(2), 251 264. https://doi.org/10.1111/jedm.12319
Fan, X., Zhang, M., & Peng, H. (2025). Is pseudo-equivalent groups equating via minimum discriminant information adjustment effective? Evidence from simulation and real data. Frontiers in Education, 10, 1–15. https://doi.org/10.3389/feduc.2025.1667220
Gonzalez, E.J., & Rutkowski, L. (2010). Principles of multiple matrix booklet designs and parameter recovery in large-scale assessments. In M. von Davier & D. Hastedt (Eds.), IERI Monograph Series: Issues and Methodologies in Large-Scale Assessments (Vol. 3, pp. 125–156). IERI.

Haberman, S.J. (2015). Pseudo-equivalent groups and linking. Journal of Educational and Behavioral Statistics, 40(3), 254–273. https://doi.org/10.3102/1076998615574772
Haebara, T. (1980). Equating logistic ability scales by a weighted least squares method. Japanese Psychological Research, 22(3), 144–149. https://doi.org/10.4992/psycholres1954.22.144
Hambleton, R.K., & Jones, R.W. (1993). An NCME instructional module on comparison of classical test theory and item response theory and their applications to test development. Educational Measurement: Issues and Practice, 12(3), 38 47. https://doi.org/10.1111/j.1745 3992.1993.tb00543.x
Hambleton, R.K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Kluwer-Nijhoff.
Hambleton, R.K., Swaminathan, H., & Rogers, H.J. (1991). Fundamentals of item response theory. Sage Publications.
Han, Y., Zhang, J., Jiang, Z., & Shi, D. (2023). Is the area under curve appropriate for evaluating the fit of psychometric models? Educational and Psychological Measurement, 83(3), 586–608. https://doi.org/10.1177/00131644221098182
Hanson, B.A., & Béguin, A.A. (2002). Obtaining a common scale for item response theory item parameters using separate versus concurrent estimation in the common-item equating design. Applied Psychological Measurement, 26(1), 3–24. https://doi.org/10.1177/0146621602026001001
Kaplan, D., & Su, D. (2016). On matrix sampling and imputation of context questionnaires with implications for the generation of plausible values in large-scale assessments. Journal of Educational and Behavioral Statistics, 41(1), 57–80. https://doi.org/10.3102/1076998615622221
Kim, S., & Lu, R. (2018). The pseudo-equivalent groups approach as an alternative to common-item equating. ETS Research Report Series, 2018(1), 1–15. https://doi.org/10.1002/ets2.12195
Kolen, M.J., & Brennan, R L. (2014). Test equating, scaling, and linking: Methods and practices (3rd ed.). Springer.
Laukaityte, I., Wallin, G., & Wiberg, M. (2026). Combining propensity scores and common items for test score equating. Applied Psychological Measurement, 50(3), 115 144. https://doi.org/10.1177/01466216251363240
Lord, F.M. (1980). Applications of item response theory to practical testing problems. Lawrence Erlbaum Associates.
Lu, R., & Guo, H. (2018). A simulation study to compare nonequivalent groups with anchor test equating and pseudo equivalent group linking. ETS Research Report Series, 2018(1), 1 16. https://doi.org/10.1002/ets2.12196
Mislevy, R.J., Johnson, E.G., & Muraki, E. (1992). Scaling procedures in NAEP. Journal of Educational Measurement, 29(3), 205–225. https://doi.org/10.1111/j.1745-3984.1992.tb00371.x
Nydick, S.W., & Weiss, D.J. (2009). A hybrid simulation procedure for the development of CATs. In D.J. Weiss (Ed.), Proceedings of the 2009 GMAC Conference on Computerized Adaptive Testing. http://swnydick.github.io/assets/pubs/Nydick2009.pdf
Ölçme, Seçme ve Yerleştirme Merkezi (ÖSYM). (2026). Yükseköğretim kurumları sınavı (YKS) başvuru kılavuzu [2026-YKS application guide]. ÖSYM.
Pokropek, A., & Borgonovi, F. (2020). Linking via pseudo-equivalent group design: Methodological considerations and an application to the PISA and PIAAC assessments. Journal of Educational Measurement, 57(4), 527–546. https://doi.org/10.1111/jedm.12261
Rutkowski, L. (2014). Sensitivity of achievement estimation to conditioning model misclassification. Applied Measurement in Education, 27(2), 115–132. https://doi.org/10.1080/08957347.2014.880440
San Martín, E., González, J., & Tuerlinckx, F. (2015). On the unidentifiability of the fixed-effects 3PL model. Psychometrika, 80(2), 450–467. https://doi.org/10.1007/s11336-014-9404-2
Shoemaker, D.M. (1973). Principles and procedures of multiple matrix sampling. Ballinger.
Sideridis, G.D., & Alghamdi, M. (2025). Guessing during testing is a person attribute not an instrument parameter. Educational and Psychological Measurement, 86(1), 191 211. https://doi.org/10.1177/00131644251374302
Stocking, M.L., & Lord, F.M. (1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7(2), 201–210. https://doi.org/10.1177/014662168300700208
Ul Hassan, M., & Miller, F. (2019). Optimal item calibration for computerized achievement tests. Psychometrika, 84(4), 1101–1128. https://doi.org/10.1007/s11336-019-09673-6
van der Linden, W.J., & Barrett, M.D. (2016). Linking item response model parameters. Psychometrika, 81(3), 650–673. https://doi.org/10.1007/s11336-015-9469-6
van der Linden, W.J., & Glas, C.A.W. (2010). Elements of adaptive testing. Springer.
van der Linden, W.J., & Pashley, P.J. (2010). Item selection and ability estimation in adaptive testing. In W.J. van der Linden & C.A.W. Glas (Eds.), Elements of adaptive testing (pp. 3–30). Springer.
Weiss, D.J. (1982). Improving measurement quality and efficiency with adaptive testing. Applied Psychological Measurement, 6(4), 473–492. https://doi.org/10.1177/014662168200600408
Willmott, C.J., & Matsuura, K. (2005). Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate Research, 30(1), 79–82. http://www.jstor.org/stable/24869236
Yen, W.M. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30(3), 187 213. https://doi.org/10.1111/j.1745 3984.1993.tb00423.x
Zhu, Z., Wang, C., & Tao, J. (2019). A two-parameter logistic extension model: An efficient variant of the three parameter logistic model. Applied Psychological Measurement, 43(6), 449 463. https://doi.org/10.1177/0146621618800273

Details

Primary Language

English

Subjects

Measurement Theories and Applications in Education and Psychology, Computer Based Exam Applications

Journal Section

Research Article

Authors

Selma Şenel ^*
0000-0002-5803-0793
Türkiye

Sebahat Gören
0000-0002-6453-3258
Türkiye

Serkan Arıkan
0000-0001-9610-5496
Türkiye

Eren Can Aybek
0000-0003-3040-2337
Türkiye

Tuba Gündüz
0000-0002-0921-9290
Türkiye

Ayşegül Bozdağ Kasap
0000-0002-5333-8617
Türkiye

Early Pub Date

October 26, 2026

Publication Date

-

Submission Date

April 14, 2026

Acceptance Date

October 1, 2026

Published in Issue

Year 2026 Number: Advanced Online Publication

IZ

https://izlik.org/JA38DZ28PY

Cite

RIS / Bibtex

APA

Şenel, S., Gören, S., Arıkan, S., Aybek, E. C., Gündüz, T., & Bozdağ Kasap, A. (2026). Linking high-stakes test forms without common items: The pseudo-equivalent item approach in item pool development. International Journal of Assessment Tools in Education, Advanced Online Publication, 1156-1171. https://izlik.org/JA38DZ28PY

AMA

1.Şenel S, Gören S, Arıkan S, Aybek EC, Gündüz T, Bozdağ Kasap A. Linking high-stakes test forms without common items: The pseudo-equivalent item approach in item pool development. Int. J. Assess. Tools Educ. 2026;(Advanced Online Publication):1156-1171. https://izlik.org/JA38DZ28PY

Chicago

Şenel, Selma, Sebahat Gören, Serkan Arıkan, Eren Can Aybek, Tuba Gündüz, and Ayşegül Bozdağ Kasap. 2026. “Linking High-Stakes Test Forms Without Common Items: The Pseudo-Equivalent Item Approach in Item Pool Development”. International Journal of Assessment Tools in Education, no. Advanced Online Publication: 1156-71. https://izlik.org/JA38DZ28PY.

EndNote

Şenel S, Gören S, Arıkan S, Aybek EC, Gündüz T, Bozdağ Kasap A (October 1, 2026) Linking high-stakes test forms without common items: The pseudo-equivalent item approach in item pool development. International Journal of Assessment Tools in Education Advanced Online Publication 1156–1171.

IEEE

[1]S. Şenel, S. Gören, S. Arıkan, E. C. Aybek, T. Gündüz, and A. Bozdağ Kasap, “Linking high-stakes test forms without common items: The pseudo-equivalent item approach in item pool development”, Int. J. Assess. Tools Educ., no. Advanced Online Publication, pp. 1156–1171, Oct. 2026, [Online]. Available: https://izlik.org/JA38DZ28PY

ISNAD

Şenel, Selma - Gören, Sebahat - Arıkan, Serkan - Aybek, Eren Can - Gündüz, Tuba - Bozdağ Kasap, Ayşegül. “Linking High-Stakes Test Forms Without Common Items: The Pseudo-Equivalent Item Approach in Item Pool Development”. International Journal of Assessment Tools in Education. Advanced Online Publication (October 1, 2026): 1156-1171. https://izlik.org/JA38DZ28PY.

JAMA

1.Şenel S, Gören S, Arıkan S, Aybek EC, Gündüz T, Bozdağ Kasap A. Linking high-stakes test forms without common items: The pseudo-equivalent item approach in item pool development. Int. J. Assess. Tools Educ. 2026;:1156–1171.

MLA

Şenel, Selma, et al. “Linking High-Stakes Test Forms Without Common Items: The Pseudo-Equivalent Item Approach in Item Pool Development”. International Journal of Assessment Tools in Education, no. Advanced Online Publication, Oct. 2026, pp. 1156-71, https://izlik.org/JA38DZ28PY.

Vancouver

1.Selma Şenel, Sebahat Gören, Serkan Arıkan, Eren Can Aybek, Tuba Gündüz, Ayşegül Bozdağ Kasap. Linking high-stakes test forms without common items: The pseudo-equivalent item approach in item pool development. Int. J. Assess. Tools Educ. [Internet]. 2026 Oct. 1;(Advanced Online Publication):1156-71. Available from: https://izlik.org/JA38DZ28PY