HOW VALID ARE OPEN UNIVERSITY EXAM PASSING STANDARDS? CONSISTENCY OF CLASSIFICATION BY CUT-OFF POINTS
Abstract
Evaluation decisions regarding students’ success in Open Education faculties such as pass/fail based on cut-off scores affect the quality of these systems. The qualification of Open Education students to obtain a bachelor’s or associate’s degree is determined by their passing grade. The purpose of this study was to investigate whether the minimum passing scores used in Open Education examinations differ from the currently used minimum passing scores according to different standard-setting methods and the classification consistency of the cut-off scores obtained by these methods with the currently used cut-off scores and with each other. The participants consisted of 15 experts, consisting of textbook authors and lecturers of the Basic Disaster Knowledge Course this course. The results showed that in the final and midterm examinations, the percentage of successful students according to the cutoff points identified using the Angoff and Nedelsky methods was significantly lower than the percentage of successful students according to the current cut-off scores of the Open Education Examinations. The standards to be determined based on academic principles were different from the ones identified by administrative decisions. Further, the pass/fail decisions based on academic principles differed from those based on administrative decisions.
Keywords
Assessment and evaluation in distance education, assessment and evaluation in open education, standard-setting, cut-off score, the Angoff method, the Nedelsky method
References
- Afrashteh, M. Y. (2021). Comparison of the validity of bookmark and Angoff standard-setting methods in medical performance tests. BMC Medical Education, 21(1), 1. https://doi.org/10.1186/s12909-020-02436-3
- Alkin, M. C., & King, J. A. (2017). Definitions of evaluation use and misuse, evaluation influence, and factors affecting use. American Journal of Evaluation, 38(3), 434–450. https://doi.org/10.1177/1098214017717015
- American Educational Research Association, American Psychological Association, & American Council on Measurement in Education (1999). Standards for educational and psychological testing. American Educational Research Association. American Educational Research Association,
- American Psychological Association, & National Council on Measurement in Education (2014). Standards for educational and psychological testing. American Educational Research Association.
- Angoff, W. H. (1984). Scales, norms, and equivalent scores. Princeton (NJ): Educational Testing Service.