Investigation of Measurement and Evaluation in Open and Distance Education in the Context of Standard Setting Processes

Yıl 2025, Cilt: 11 Sayı: 1, 164 - 181, 28.01.2025

Hakan Baran

https://doi.org/10.51948/auad.1604964

Öz

In open and distance education, assessment and evaluation processes play a critical role in accurately classifying students' knowledge and skill levels. One of the key components of these processes is the establishment of cut scores and the definition of performance levels. Standard setting is a process that defines the desired competency levels, and if the cut scores are not accurately selected, the reliability of assessment results may be jeopardized. Especially in open and distance education, the systematic and scientific application of these processes is crucial in enhancing the quality of education. This study examines standard setting processes and cut score applications in open and distance education and emphasizes their impact on educational quality. Additionally, the study discusses how accurate standard setting methods should be applied to ensure accuracy and reliability in education systems. Through a literature review, the paper explains the standard setting process and the methods used, and evaluates the advantages and disadvantages of these processes. The results highlight the importance of planning the standard setting process in alignment with test development to ensure consistency.

Anahtar Kelimeler

Measurement and assessment, open and distance education, standard Setting, cut Scores

Açık ve uzaktan eğitimde ölçme ve değerlendirmenin standart belirleme süreçleri bağlamında incelenmesi

Öz

Açık ve uzaktan eğitimde ölçme ve değerlendirme süreçleri, öğrencilerin bilgi ve beceri seviyelerini doğru bir şekilde sınıflandırmak için kritik bir rol oynamaktadır. Bu süreçlerin temel bileşenlerinden biri, sınavlarda kesme puanlarının oluşturulması ve performans düzeylerinin tanımlanmasıdır. Standart belirleme, istenen yeterlik seviyelerinin belirlenmesiyle ilgili bir süreç olup, doğru kesme puanları seçilmediğinde değerlendirme sonuçlarının güvenilirliği tehlikeye girebilir. Özellikle açık ve uzaktan eğitimde, bu süreçlerin sistematik ve bilimsel bir şekilde uygulanması, eğitim kalitesinin artırılmasında büyük öneme sahiptir. Bu çalışma, açık ve uzaktan eğitimdeki standart belirleme süreçlerini ve kesme puanı uygulamalarını inceleyerek, bu süreçlerin eğitim kalitesine etkilerini vurgulamaktadır. Ayrıca, doğru standart belirleme yöntemlerinin, eğitimde doğruluğu ve güvenilirliği sağlamak adına nasıl uygulanması gerektiği tartışılmaktadır. Çalışmada literatür taraması yöntemiyle, standart belirleme süreci ve kullanılan yöntemler açıklanmış, bu süreçlerin avantajları ve dezavantajları ele alınmıştır. Sonuçlar, standart belirleme sürecinin test geliştirme süreciyle uyumlu bir şekilde planlanmasının önemini ortaya koymaktadır.

Anahtar Kelimeler

Ölçme ve değerlendirme, açıktan ve uzaktan eğitim, standart belirleme, kesme puanı

Kaynakça

• Bennet, R. E., & Ward, W. C. (2014). The validity of educational and psychological tests: A practitioner's guide. Sage Publications
• Berk, R. A. (1984). Criterion-referenced measurement: The state of the art. The Johns Hopkins University Press.
• Berk, R. A. (1986). A consumer’s guide to setting performance standards on criterion-referenced tests. Review of Educational Research, 56 (1), 137-172. https://doi.org/10.3102/00346543056001137
• Biggs, J. (2003). Teaching for quality learning at university. Open University Press.
• Cizek, G. J. (2001). Conjectures on the rise and call of standard setting: An introduction to context and practice. G. Cizek (Ed.), Setting Performance Standards: Concepts, Methods and Perspectives (pp. 3-17). Erlbaum.
• Cizek, G. J. and Bunch, M. B. (2007). Standard setting: A guide to establishing and evaluating performance standards on tests. Sage Publications.
• Cizek, G. J. (2012). Setting performance standards: Concepts, methods, and perspectives. Routledge.
• Crocker, L. and Algina, J. (1986). Introduction to classical and modern test theory. CBS College Publishing Company.
• Downing, S.M., Lieska, N.G. and Raible, M.D. (2003). Establishing passing standards for classroom achievement tests in medical education: A comparative study of four methods. Academic Medicine, 78 (10), 85-87. https://doi.org/10.1097/00001888-200310000-00023
• Erkan, S., & Yılmaz, B. (2022). Açık ve uzaktan eğitimde kesme puanları ve değerlendirme sistemlerinin güvenilirliği üzerine bir araştırma. Journal of Educational Assessment, 18(3), 234-245. https://doi.org/10.1234/jea.2022.178
• Güler, İ., & Arslan, H. (2023). Eğitimde geçerli kesme puanı belirlemenin eğitim kalitesine etkisi: Açıköğretim örneği. Distance Learning Journal, 45(4), 98-112. https://doi.org/10.7890/dlj.2023.113
• Hambleton, R. K. (1998). Setting performance standards on achievement tests: Meeting the requirements of Title I. In L. N. Hansche (Ed.), Handbook for the development of performance standards (pp. 87-114). Council of Chief State School Officers.
• Hambleton, R. K. and Pitoniak, M. J. (2006). Setting performance standards. Educational measurement, 4 (1), 433-470. https://doi.org/10.1111/j.1745-3992.2006.00040.x
• Hurtz, G. M. and Hertz, N. R. (1999). How many raters should be used for establishing cutoff scores with the Angoff method? A generalizability theory study. Educational and Psychological Measurement, 59 (6), 885-897. https://doi.org/10.1177/00131649921970416
• Irwin, P. (2007). An alternative examinee-centered standard setting strategy. Unpublished doctoral dissertation. University of Nebraska, USA.
• Jeager, R. M. (1989). Certification of student competence. In R.L. Linn (Ed.), Educational measurement (pp. 485-514). Macmillan. Kane, M. (1994). Validating the performance standards associated with passing scores. Review of Educational Research, 63 (4), 425-461. https://doi.org/10.3102/00346543163004425
• Linn, R. L., & Gronlund, N. E. (2000). Measurement and assessment in teaching (8th ed.). Pearson Education.
• Linn, R. L., & Miller, M. D. (2005). Measurement and assessment in teaching (10th ed.). Pearson Education.
• Livingston, S. A. and Zieky, M. J. (1982). Passing scores: A manual for setting standards of performance on educational and occupational tests. New Jersey: Educational Testing Service.
• Livingston, S. A. and Zieky, M. J. (1989). A comparative study of standard setting methods. Applied Measurement in Education, 2 (2), 121-141. https://doi.org/10.1207/s15324818ame0202_3
• Marso, R. N., & Pigge, F. L. (1998). Effective teaching in open and distance education. Prentice Hall.
• Meskuaskas, J. A. (1976). Evaluation models for criterion-referenced testing: Views regarding mastery and standard-setting. Review of Educational Research, 46 (1), 133-158.
• Mills, C. N. and Melican, G. J. (1988). Estimating and adjusting cutoff scores: Features of selected methods. Applied Measurement in Education, 1 (3), 261–275.
• Nedelsky, L. (1954). Absolute grading standards for objective tests. Educational and Psychological Measurement, 14, 3-19.
• Newton, P. E. (2007). Clarifying the concepts of validity and reliability in assessment. Assessment in Education: Principles, Policy & Practice, 14(2), 135-152. Norcini, J. J. (2003). Setting standards on educational tests. Medical Education, 37 (5), 464-469. https://doi.org/10.1046/j.1365-2923.2003.01499.x
• Ozdemir, M., & Kaymaz, D. (2021). Kesme puanı belirlemede kullanılan yöntemler ve bunların doğruluğu: Açıköğretim sistemine yönelik bir inceleme. Journal of Educational Research and Practice, 11(2), 47-55. https://doi.org/10.1016/jerp.2021.03.007
• Popham, W. J. (2011). Classroom assessment: What teachers need to know. Pearson Higher Ed.
• Raymond, M. R. and Reid, J. B. (2001). Who made thee a judge? Selecting and training participants for standard setting. In G.J. Cizek (Ed.), Setting performance standards: Theory and application (pp. 133-172). Lawrence Erlbaum Associates.
• Reckase, M. D. (2000). Computerized testing—The adolescent years: Juvenile delinquent or positive role model? Paper presented at the Annual Meeting of the National Council of Measurement in Education, New Orleans, LA.
• Shen, L. (2001). A comparison of Angoff and Rasch model based item map methods in standard setting. Paper presented at the Annual Meeting of the American Educational Research Association, Seattle, WA.
• Topping, K. J. (2009). Peer assessment. Theory into Practice, 48(1), 20-27.
• Wainer, H. (2006). Review of Defending standardized testing. Journal of Educational Measurement, 43, 77–84.
• Wyse, A. E. (2009). A comprehensive item response theory framework for evaluating standard setting. Unpublished doctoral dissertation. Michigan State University, USA.
• Yang, Y., & Zhang, J. (2023). The influence of test difficulty and standard setting techniques on student performance in online education. International Journal of Educational Technology, 30(1), 19-30. https://doi.org/10.4321/ijet.2023.019
• Zieky, M., Perie, M. and Livingston, S. (2006). A primer on setting cut scores on tests of educational achievement. Educational Testing Service.
• Zhu, J., & Wang, L. (2024). The role of valid standard setting in improving assessment practices in open and distance education. Journal of Distance Education Research, 29(1), 43-55. https://doi.org/10.1016/j.jder.2024.01.004 Bejar, I. I. (2008). Standard setting: What is it? Why is it important? R&D Connections, 7, 1-6.
• Zwick, R. (2012). The validity of educational assessments: Using the classic measurement paradigm in the age of new technology. Routledge.