Bayes ve SPRT Tabanlı Bilgisayarlı Sınıflama Test Algoritmalarının Performans Analizi

Öz

Bu çalışmada, bilgisayarlı uyarlanabilir ve sınıflama temelli test algoritmalarının doğruluk, karar hataları ve test verimliliği açısından performansları karşılaştırılmıştır. Bilgisayarlı Uyarlanabilir Test (CAT), Bilgisayarlı Sınıflama Testi (CCT), Bilgisayarlı Uyarlanabilir Ardışık Test (CAST) ve Çok Aşamalı Test (MST) ailelerine ait on algoritma/varyant 100 replikasyonlu Monte Carlo simülasyonu ile incelenmiştir. Ayrıca klasik MST’nin sabit modül uzunluğu sınırlılığını azaltmak amacıyla Bayesçi ve Ardışık Olasılık Oran Testi (SPRT) tabanlı iki değişken maddeli MST varyantı geliştirilmiştir. Her replikasyonda 1000 öğrenci ve 200 maddelik madde havuzu üretilmiş; algoritmalar sınıflama doğruluğu, dengeli doğruluk, duyarlılık, özgüllük, yanlış pozitif/negatif kararlar, F1 değeri, ortalama madde sayısı ve işlem süresi bakımından değerlendirilmiştir. Bulgular, CCTSPRT ve CASTSPRT algoritmalarının en yüksek doğruluk-verimlilik dengesini sunduğunu göstermiştir. Bayes tabanlı algoritmalar daha yüksek duyarlılık üretirken özgüllük bakımından daha sınırlı kalmıştır. MSTSPRT ise klasik MST’ye yakın doğrulukla daha kısa test uzunluğu sağlamıştır. Sonuçlar, algoritma seçiminin ölçme amacı, hata maliyeti ve e-değerlendirme bağlamındaki gereksinimler birlikte dikkate alınarak yapılması gerektiğini göstermektedir.

Anahtar Kelimeler

• Bilgisayarlı sınıflama testleri
• bilgisayarlı uyarlanabilir testler
• çok aşamalı test
• değişken maddeli MST
• SPRT
• bayesçi sınıflama

Etik Beyan

Bu çalışma Hacettepe Üniversitesi Eğitim Bilimleri Enstitüsü Araştırma Etik Kurulunun 07.01.2026 tarihinde yapmış olduğu toplantıda incelenmiş olup, etik açıdan uygun bulunmuştur.

Performance Analysis of Bayesian and SPRT-Based Computerized Classification Testing Algorithms

Öz

This study compared the performance of computerized adaptive and classification based testing algorithms in terms of classification accuracy, decision errors, and testing efficiency. Ten algorithms/variants from the Computerized Adaptive Testing (CAT), Computerized Classification Testing (CCT), Computerized Adaptive Sequential Testing (CAST), and Multistage Testing (MST) families were examined through a Monte Carlo simulation with 100 replications. In addition, two variable-length MST variants based on Bayesian and Sequential Probability Ratio Test (SPRT) decision rules were developed to address the fixed module-length limitation of conventional MST. For each replication, a sample of 1000 examinees and an item pool of 200 items were generated. Algorithms were evaluated using classification accuracy, balanced accuracy, sensitivity, specificity, false-positive and false-negative classifications, F1 score, average test length, and computational time. The results indicated that the CCTSPRT and CASTSPRT algorithms provided the most favorable balance between classification accuracy and testing efficiency. Bayesian algorithms yielded higher sensitivity but lower specificity. The MSTSPRT variant achieved classification accuracy comparable to that of conventional MST while requiring fewer administered items. Overall, the findings suggest that algorithm selection should be guided by the intended measurement purpose, the relative costs of classification errors, and the practical requirements of e-assessment environments.

Anahtar Kelimeler

• Computerized classification testing
• computerized adaptive testing
• multistage testing
• variable-length MST
• SPRT
• bayesian classification

Etik Beyan

This study was reviewed and approved by the Research Ethics Committee of the Hacettepe University Institute of Educational Sciences on January 7, 2026.

Kaynakça

1. Baker, F. B., ve Kim, S. H. (2004). Item responsetheory: Parameter estimation techniques. CRC press.Bao, Y., Shen, Y., Wang, S., veBradshaw, L. (2021). Flexible computerized adaptive tests to detect misconceptions and estimateability simultaneously. Applied Psychological Measurement, 45(1), 3-21.
2. Black, P., ve Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy and Practice, 5(1), 7–74.https://doi.org/10.1080/0969595980050102
3. Chang, H. H., Wang, C., ve Zhang, S. (2021). Statistical applications in educational measurement. Annual Review of Statistics and Its Application, 8(1), 439-461.https://doi.org/10.1146/annurev-statistics-042720-104044
4. Eggen, T. J. H. M. (1999). Item selection in adaptive testing with the sequential probability ratio test. Applied Psychological Measurement, 23(3), 249–261.https://doi.org/10.1177/01466219922031365
5. Eggen, T. J. H. M., ve Straetmans, G. J. J. M. (2000). Computerized adaptive testing for classifying examinees into three categories. Educational and Psychological Measurement, 60(5), 713–734.https://doi.org/10.1177/00131640021970862
6. Feinberg, R. A., ve Rubright, J. D. (2016). Conducting simulation studies in psychometrics. Educational Measurement: Issues and Practice, 35(2), 36-49.https://doi.org/10.1111/emip.12111
7. Kara, B. E., ve Doğan, N. (2022). The effect of ratio of items indicating differential item functioning on computer adaptive and multi-stagetests. International Journal of Assessment Tools in Education, 9(3), 682-696.https://doi.org/10.21449/ijate.1105769
8. Kingsbury, G. G., ve Weiss, D. J. (1983). A comparison of IRT-based adaptive mastery testing and a sequential mastery testing procedure. In D. J. Weiss (Ed.), New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 257–283). Academic Press.https://doi.org/10.1016/B978-0-12-742780-5.50024-X

9. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Lawrence Erlbaum Associates.
10. Luecht, R. M. (2000, April). Implementing the computer-adaptive sequential testing (CAST) framework to mass produce high quality computer-adaptive and mastery tests. Paper presented at the Annual Meeting of the National Council on Measurement in Education, New Orleans, LA.
11. Luecht, R. M., ve Nungester, R. J. (1998). Some practical examples of computer-adaptive sequential testing. Journal of Educational Measurement, 35(3), 239–249.https://doi.org/10.1111/j.1745-3984.1998.tb00537.x
12. Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.https://doi.org/10.3102/0034654307313795
13. Sokolova, M., & Lapalme, G. (2009). A systematic analysis of performance measures for classification tasks. Information processing & management, 45(4), 427-437.
14. Spray, J. A., ve Reckase, M. D. (1996). Comparison of SPRT and sequential Bayes procedures for classifying examinees into two categories using a computerized test. Journal of Educational and Behavioral Statistics, 21(4), 405–414.https://doi.org/10.3102/10769986021004405
15. Swets, J. A. (1988). Measuring the accuracy of diagnostic systems. Science, 240(4857), 1285-1293.https://doi.org/10.1126/science.3287615
16. Tharwat, A. (2021). Classification assessment methods. Applied computing and informatics, 17(1), 168-192.
17. Thompson, N. A. (2009). Item selection in computerized classification testing. Educational and Psychological Measurement, 69(5), 778–793.https://doi.org/10.1177/0013164408324460
18. van der Linden, W. J.,ve Glas, C. A. W. (Eds.). (2010). Elements of adaptive testing. Springer.
19. Wainer, H., Dorans, N. J., Flaugher, R., Green, B. F., Mislevy, R. J., Steinberg, L., ve Thissen, D. (2000). Computerized adaptive testing: A primer (2nd ed.). Lawrence Erlbaum Associates.
20. Wald, A. (1947). Sequential analysis. Wiley. Wang, C., Chen, P., ve Huebner, A. (2021). Stopping rules for multi‐categorycomputerized classification testing. British Journal of Mathematical and Statistical Psychology, 74(2), 184-202.https://doi.org/10.1111/bmsp.12202
21. Weiss, D. J. (1982). Improving measurement quality and efficiency with adaptive testing. Applied Psychological Measurement, 6(4), 473–492.https://doi.org/10.1177/014662168200600408
22. Yan, D., von Davier, A. A., ve Lewis, C. (Eds.). (2014). Computerized multi stage testing: Theory and applications. CRC Press. Yiğiter, M. S., ve Doğan, N. (2023). Comparison of different computerized adaptive testing approaches with shadow test under different test length and ability estimation method conditions. Journal of Measurement and Evaluation in Education and Psychology, 14(4), 396-412.https://doi.org/10.21031/epod.1202599
23. Zenisky, A., Hambleton, R. K., ve Luecht, R. M. (2009). Multi stage testing: Issues, designs, and research. In W. J. van der Linden ve C. A. W. Glas (Eds.), Elements of adaptive testing (pp. 355–372). Springer.