A Comparison of Traditional and Kernel Equating Methods

Abstract

In this study, the equated score results of the kernel equating (KE) method compared with the results of traditional equating methods—equipercentile and linear equating and 9th grade 2009 ÖBBS Form B of Social Sciences and 2009 ÖBBS Form D of Social Sciences was used under an equivalent groups (EG) design. Study sample consists of 16.249 students taking booklets B and another 16.327 students taking D in that test. The analysis of the test forms was carried out in four steps. First, descriptive statistics were calculated for the data and then it was checked whether the data obtained from the two booklets satisfy the equating conditions. In the second step, the booklets were equated according to methods. Lastly, the errors for each equating methods were calculated. Kernel equating results were nearly same to the results from the corresponding traditional equating methods. In Kernel equating, when parameter h was selected as optimal, equated scores provided almost identical results as traditional equipercentile equating. When it was selected large, this time the equated scores provided results almost identical to traditional linear equating. It is concluded that Kernel equating methods are relatively more the most appropriate equating method method than traditional equating methods.

Keywords

• Kernel equating
• Traditional equating Methods
• The equivalent groups design

A Comparison of Traditional and Kernel Equating Methods

Abstract

In this study, the equated score results of the kernel equating (KE) method compared with the results of traditional equating methods—equipercentile and linear equating and 9th grade 2009 ÖBBS Form B of Social Sciences and 2009 ÖBBS Form D of Social Sciences was used under an equivalent groups (EG) design. Study sample consists of 16.249 students taking booklets B and another 16.327 students taking D in that test. The analysis of the test forms was carried out in four steps. First, descriptive statistics were calculated for the data and then it was checked whether the data obtained from the two booklets satisfy the equating conditions. In the second step, the booklets were equated according to methods. Lastly, the errors for each equating methods were calculated. Kernel equating results were nearly same to the results from the corresponding traditional equating methods. In Kernel equating, when parameter h was selected as optimal, equated scores provided almost identical results as traditional equipercentile equating. When it was selected large, this time the equated scores provided results almost identical to traditional linear equating. It is concluded that Kernel equating methods are relatively more the most appropriate equating method method than traditional equating methods.

Keywords

• Kernel equating
• Traditional equating Methods
• The equivalent groups design

References

1. Akhun, İ. (1984). “İki korelasyon katsayısı arasındaki manidarlığın test edilmesi”. Ankara Üniversitesi Eğitim Fakültesi Dergisi. 17, 1-7.
2. Albano, A. D. (2016). “equate: An R packageforobserved-score linking and equating”. Journal of Statistical Software, 74(8), 1-36.
3. Andersson, B.,Branberg, K., Wiberg, M. (2013). “Performing the Kernel Method of Test Equating with the Package kequate”. Journal of Statistical Software, 55(6), 1–25.
4. Baykul, Y. (1996). İstatistik: Metodlar ve uygulamalar (3. Baskı). Ankara: Anı Puplishing
5. Büyüköztürk, Ş. (2007).Sosyal bilimler için veri analizi el kitabı (8.Baskı). Ankara: Pegem A Yayıncılık.
6. Choi, S. I. (2009). “A comparison of kernel equating and traditional equipercentile equating methods and the parametric bootstrap methods for estimating Standard errors in equipercentile equating”.Unpublished Doctoral Dissertation.University of Illinois at Urbana-Champaign.
7. Dorans, N. J.,&Holland, P. W. (2000). “Population invariance and the equatability of tests: Basic theory and the linearcase”. ETS Research Report Series, (2).
8. Eğitim, Araştırma ve Geliştirmesi Daire Başkanlığı (EARGED).(2010). Ortaöğretim ÖBBS raporu 2009. Ankara, Milli Eğitim Bakanlığı.

9. Grant, M. C.,Zhang, L., &Damiano, M. (2009). “An Evaluation of Kernel Equating: Parallel Equating with Classical Methods in the SAT SubjectTests [TM] Program”. ETS Research Report Series, Educational Testing Service.
10. Hambleton, R. K. &Swaminathan, H. (1985).“Item response theory: principles and applications”. Boston: Academic Puslishers Group.
11. Holland, P. W. (2007). A frame work and history for score linking. In Linking and aligning scores and scales (pp. 5-30).Springer, New York, NY.
12. Kelecioğlu, H.,& Öztürk Gübeş, N. (2013). “Comparing linear equating and equipercentile equating methods using random groups design”. International. Online Journal of Educational Sciences, 5(1), 227-241.
13. Kolen, M. J. (1988). “An NCME instructional module on traditional equating methodology”.Educational Measurement: Issues and Practice, 7, 29-36.
14. Kolen, M. J.,&Brennan, R. L. (2004). Test equating, scaling, and linking: Methods and practices (2nd. ed.). New York: Springer.
15. Lee, Y. H.,&vonDavier, A. A. (2011). Equating through alternative kernels.In Statistical models for test equating, scaling, and linking (pp. 159-173).Springer New York.
16. Livingston, S. A. (2014). “Equating test scores (without IRT)”. Educational testing service.
17. Mao, X. (2006).“An investigation of the accuracy of the estimates of Standard errors for the kernel equating functions”.Unpublished Doctoral Dissertation, TheUniversity of Iowa.
18. Mao, X.,Davier, A. A., &Rupp, S. (2006). “Comparisons of the Kernel Equating Method with the Traditiona Equating Methods on Praxis™ Data”.ETS Research Report Series, 2006(2).
19. Ricker, K. L.,&Davier, A. A. (2007). “Theimpact of anchor test length on equating results in a nonequivalent groupsdesign”. ETS Research Report Series, 2007(2).
20. vonDavier, A. A.,Holland, P. W., Livingston, S. A., Casabianca, J., Grant, M. C., & Martin, K. (2006). “An Evaluation of the Kernel Equating Method: A Special Study with Pseudo tests Constructed From Real Test Data”. ETS Research Report Series,2006(1).
21. von Davier, A., Holland, P. W., & Thayer, D. T. (2004). The Kernel method of equating. New York, NY: Springer.
22. Zhu, W. (1998).“Test equating: What, why, how?”.Research Quarterly for Exercise and Sport, 69(1), 11-23