Comparison of Different Bandwidth Determination Methods in Kernel Equating

Abstract

The study aims to compare the presented methods for determining the bandwidth parameter in the kernel equating method on a real data set. A bandwidth parameter needs to be determined when kernel equating is used to equate two test forms. The bandwidth parameters determine the smoothness of the continuousized score distributions, so their effect on equating results is inevitable. Gaussian Kernel, Logistic Kernel and Uniform Kernel methods were used for bandwidth selection and the results were compared according to the Percentage Relative Error (PRE), the Standard Error and the Standard Error of Equating Difference (SEED). The findings of the study show that the three different approaches to minimizing the penalty function have similar results. Although the standard errors of the equated scores obtained with the uniform kernel method were slightly smaller, the results were almost the same as the other two approaches. When the three equating methods are compared according to the percent relative error, the distribution obtained from Gaussian kernel equating is more consistent with the population distribution.

Keywords

• kernel equating
• bandwidth selection
• continuousization

References

1. Andersson, M. B., Branberg, K., & Wiberg, M. (2013). Performing the kernel method of test equating with the package kequate. Journal of Statistical Software, 55(6), 1-25. Doi: 10.18637/jss.v055.i06
2. Andersson, B. (2014). Contributions to kernel equating. Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Social Sciences, 106(24).
3. Andersson, B., & von Davier, A. A. (2014). Improving the bandwidth selection in kernel equating. Journal of Educational Measurement, 51(3), 223-238. https://doi.org/10.1111/jedm.12044
4. Cid, J. A., & von Davier, A. A. (2014). Examining potential boundary bias effects in Kernel smoothing on equating an introduction for the adaptive and Epanechnikov Kernels. Applied Psychological Measurement, 39(3) 208–222. https://doi.org/10.1177/0146621614555901
5. Dorans, N. J., & Holland, P. W. (2000). Population invariance and the equitability of tests: Basic theory and the linear case. Journal of educational measurement, 37(4), 281-306. https://doi.org/10.1111/j.1745-3984.2000.tb01088.x
6. Häggström, J. & Wiberg, M. (2014), Optimal bandwidth selection in observed-score kernel equating. Journal of Educational Measurement, 51: 201-211. https://doi.org/10.1111/jedm.12042
7. Holland, P. W., & Thayer, D. T. (1987). Notes on the use of log‐linear models for fitting discrete probability distributions. ETS Research Report Series, (2), 1-40. https://doi.org/10.1002/j.2330-8516.1987.tb00235.x
8. Holland, P. W., & Thayer, D. T. (1989). The Kernel method of equating score distributions. ETS Research Report Series, (1), 1-45. https://doi.org/10.1002/j.2330-8516.1989.tb00333.x

9. Kolen, M. J., & Brennan, R. L. (2014). Test Equating, Scaling, and Linking. Springer.
10. Lee, Y. H., & von Davier, A. A. (2008). Comparing alternative Kernels for the Kernel method of test equating: Gaussian, logistic, and uniform Kernels. ETS Research Report Series, 2008(1), i-26. https://doi.org/10.1002/j.2333-8504.2008.tb02098.x
11. Liang, T., & von Davier, A. A. (2014). Cross-Validation an Alternative Bandwidth-Selection Method in Kernel Equating. Applied Psychological Measurement, 38(4), 281-295. https://doi.org/10.1177/0146621613518094
12. Liou, M., Cheng, P. E., & Johnson, E. G. (1996). Standard errors of the Kernel equating methods under the common‐item design. ETS Research Report Series, (1), 1-36. https://doi.org/10.1177/01466216970214005
13. Liu, J., & Low, A. C. (2008). A comparison of the Kernel equating method with traditional equating methods using SAT® Data. Journal of Educational Measurement, 45(4), 309-323. https://doi.org/10.1111/j.1745-3984.2008.00067.x
14. Livingston, S. A. (1993). An empirical try out of Kernel equating. ETS Research Report Series, (2), 1-9.
15. Mao, X., Davier, A. A., & Rupp, S. (2006). Comparisons of the Kernel equating method with the traditional equating methods on Praxis™ Data. ETS Research Report Series, (2), 1-31. https://doi.org/10.1002/j.2333-8504.2006.tb02036.x
16. Moses, T., & Holland, P. (2007). Kernel and traditional equipercentile equating with degrees of presmoothing. ETS Research Report Series, (1), 1-39. https://doi.org/10.1002/j.2333-8504.2007.tb02057.x
17. Silverman, B. W. (1986). Density estimation for statistics and data analysis (26). CRC Press.
18. Soh, Y., Hae, Y., Mehmood, A., Ashraf, R. H., & Kim, I. (2013). Performance evaluation of various functions for Kernel density estimation. Open J Appl Sci, 3(1), 58-64.
19. Tapia, R. A., & Thompson, J. R. (1978). Nonparametric probability density estimation. Johns Hopkins University Press.
20. von Davier, A. A., Holland, P. W., & Thayer, D. T. (2004). Kernel Equating versus Other Equating Methods. The Kernel Method of Test Equating, 87-95. Springer.
21. von Davier, A. (Ed.). (2010). Statistical models for test equating, scaling, and linking. Springer.
22. Wallin, G., Häggström, J., & Wiberg, M. (2021). How important is the choice of bandwidth in kernel equating?. Applied Psychological Measurement, 45(7-8), 518-535.
23. Wang, T. (2008). The continuized log-linear method: An alternative to the Kernel method of continuization in test equating. Applied Psychological Measurement. https://doi.org/10.1177/0146621607314043