The complexity of the grading system in Turkish higher education

Öz

Based on the academic performance grades of university students, various high-stakes decisions are made, including determinations of pass/fail status, the awarding of diplomas, and eligibility for placement in graduate education programs. According to the criteria used, the types of assessment are divided into two assessment, criterion-referenced assessments and norm-referenced assessments. When the grading system of state universities in Turkish higher education is examined, it has been observed that some universities use criterion-referenced assessment, some use norm-referenced assessment, and some use both assessment systems. The purpose of this research is to examine whether inter-university grading systems show significant concordance in the context of university students' letter grades or not. In other words, it is to reveal whether there are skew in the grading systems of public universities. In this context, 250 individuals were simulated in a way that their class/group achievement level would show a normal distribution. Among the public universities in the 2021-2022 Academic Performance Ranking of Universities (URAP), four state universities were determined in the first quarter, second quarter, third quarter, and last quarter. The letter grades of each student's academic success grade in the relevant universities were determined and it was examined whether there was a significant concordance between the letter grades of the students. In the study, it was concluded that in the context of university students' letter grades, inter-university grading systems generally do not show significant concordance. The findings are expected to contribute to the work of the Council of Higher Education and the University Education Commissions.

Anahtar Kelimeler

• Higher education
• Grading system
• Fleiss’ kappa
• Criterion-referenced assessment
• Norm-referenced assessment

References

1. Airasian, P.W. (1994). Classroom assessment. Mc Graw Hill. Inc. New York.
2. Atalmış, E.H. (2019). A statistical comparison of norm-referenced assessment systems usepar in higher education in Turkey. Journal of Measurement and Evaluation in Education and Psychology, 10(1), 12-29. https://doi.org/10.21031/epod.487335
3. Atılgan, H., Yurdakul, B., & Öğretmen, T. (2012). A research on the relative and absolute evaluation for determination of students achievement. Inonu University Journal of the Faculty of Education, 13(2), 79-98.
4. Atılgan, H., Kan, A., & Doğan, N. (2011). Eğitimde ölçme ve değerlendirme [Measurement and evaluation in education]. Anı Yayıncılık.
5. Bartin University (2022). Bartın Üniversitesi Bağıl Değerlendirme Sistemi Uygulama Yönergesi [Bartin University Relative Assessment System Implementation Instruction]. https://kms.kaysis.gov.tr/Home/Kurum/85269548
6. Basol Gocmen, G. (2004). Değerlendirmeye genel bir bakış: Kriter-referanslı (mutlak) ya da norm-referanslı (bağıl) değerlendirme [An overview of evaluation: Criterion-referenced (absolute) or norm-referenced (relative) evaluation]. XIII. Ulusal Eğitim Bilimleri Kurultayı’nda sunulmuş bildiri. İnönü Üniversitesi Eğitim Fakültesi, Malatya.
7. Demirel, Ö. (2007). Eğitimde program geliştirme [Program development in education]. Pegem A Yayıncılık.
8. Dooley, K. (2002). Simulation research methods. In J. Baum (Ed.), Companion to organizations (pp. 829-848). Blackwell.

9. Duman, B. (2011). Sınıf öğretmeni adaylarının bağıl değerlendirmeye ilişkin görüşleri [Opinions of classroom teacher candidates regarding relative assessment]. NWSA-E Journal of New World Sciences Academy, 6(1), 536-548
10. Ebel, R.L. (1965). Measuring educational achievement. Prentice-Hall, Inc.
11. Finkelstein, I.E. (1913). The marking system in theory and prac-tice. Baltimore. https://babel.hathitrust.org/cgi/pt?id=uc1.$b264457;view=1up;seq=9
12. Fleiss J.L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 7, 378-382.
13. Fleiss, J.L. (1981) Statistical methods for rates and proportions. John Wiley, 38-46.
14. Haladyna, T.M. (1999). A Complete guide to student grading. Allyn and Bacon. A Viacom Company.
15. Han, K.T. (2011). User’s manual: SimulCAT. http://www.umass.edu/remp/software/simcata /simulcat/SimulCAT_Manual.pdf.
16. Harran University (2022). Harran Üniversitesi Bağıl Değerlendirme Yönergesi [Harran University Relative Assessment Instruction]. http://ogrenci.harran.edu.tr/assets/uploads/other/files/ogrenci/files/Ba%C4%9F%C4%B1l_De%C4%9Ferlendirme_Sistemi_Ekim_2019_web.pdf
17. Istanbul University (2022). İstanbul Üniversitesi Önlisans, Lisans Ölçme ve Değerlendirme Esasları [Istanbul University Associate Degree, Undergraduate Measurement and Evaluation Principles]. https://cdn.istanbul.edu.tr/FileHandler2.ashx?f=olcme-degerlendirme.pdf.
18. Kaya, Ü., & Semerci, Ç. (2017). The opinions about relative and absolute assessment of teaching staff in the higher education. The Journal of Academic Social Science, 5(47), 457-467. https://doi: 10.16992/asos.12321
19. Kaysi, F., Bavli, B., & Gürol, A. (2017). Educational connoisseurship and criticism: evaluation of a cooperation model between university and the sector on vocational education. Journal of Education and Practice, 8(6), 25-35.
20. Kelley, A.C., & Zarembka, P. (1968). Normalization of student test scores: An experimental justification. The Journal of Educational Research, 62(4), 160-164. https://www.jstor.org/stable/27532173
21. Keskin M., & Ertan H. (2001). İstanbul Üniversitesi’nin bağıl değerlendirme sistemi kitapçığı [Istanbul University's relative assessment system booklet]. İstanbul: İstanbul Üniversitesi.
22. Kırklareli University (2022). Kırklareli Üniversitesi Sınav ve Başarı Değerlendirme Yönergesi [Kırklareli University Examination and Success Assessment Instruction]. https://oidb.klu.edu.tr/Yardimci_Sayfalar/183-yonergeler.klu
23. Kubiszyn, T., & Borich, G. (2000). Educational testing and measurement: Classroom application and practice (6th ed.). John Wiley & Sons, Inc.
24. Landis, J.R., & Koch, G.G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.
25. Lok, B., McNaught, C., & Young, K. (2016). Criterian-referenced and norm-reference assessments: Compatibility and complementarity. Assessment & Evaluation in Higher Education, 41(3), 450-465. https://doi.org/10.1080/02602938.2015.1022136
26. Mandernach, B.J. (2003). Effective grading strategies. Park University Faculty Development Quick Tips. https://www.park.edu/center-for-excellence-in-teaching-and-learning/
27. Martin, I.G., & Jolly, B. (2002). Predictive validity and estimated cut score of an objective structured clinical examination (OSCE) used as an assessment of clinical skills at the end of the first clinical year. Medical education, 36(5), 418 425. https://doi.org/10.1046/j.1365-2923.2002.01207.x
28. Moses, M.S., & Nanna, M.J. (2007). The testing culture and the persistence of high-stakes testing reforms. Education and Culture, 55-72.
29. Nartgün, Z. (2007). Aynı puanlar üzerinden yapılan mutlak ve bağıl değerlendirme uygulamalarının notlarda farklılık oluşturup oluşturmadığına ilişkin bir inceleme [An examination of whether absolute and relative evaluation practices based on the same scores create differences in grades]. Ege Eğitim Dergisi, 8(1), 19-40.
30. Nitko, A.J., & Brookhart, S.M. (2007), Educational assessment of students. Pearson Education Inc.
31. Özçelik, D.A. (1992). Okullarda ölçme ve değerlendirme [Measurement and evaluation in schools]. ÖSYM.
32. Özkan, Y.Ö. (2016). Üniversite mezuniyet notları ve dönüşüm tablolarında yaşanan kaos [The chaos in university graduation grades and conversion tables]. Journal of Higher Education and Science, 6(1), 71-76.
33. Öztürk-Gübeş, N. (2021). An investigation into weighting problem in norm-referenced grading system. Eurasian Journal of Educational Research, 93, 337 356. https://doi.org/10.14689/ejer.2021.93.16
34. Reynolds, C.R., Livingston, R.B., & Wilson, W. (2006). The problem of bias in educational assessment in measurement and assessment in education. Pearson Education.
35. Sayın, A. (2016). The Effect of using relative and absolute criteria to decide students’ passing or failing a Course. Journal of Education and Training Studies, 4(9), 1-9. http://dx.doi.org/10.11114/jets.v4i9.1571
36. Thorndike, R.M. (2005). Measurement and evaluation in psychology and education. Pearson Education.
37. Thorndike, R.L., & Hagen, E.P. (1977). Measurement and evaluation in psychology and education. John Wiley & Sons, Inc.
38. Tinkelman, D., Venuti, E., & Schain, L. (2013). Disparate methods of combining test and assignment scores into course grades. Global Perspectives on Accounting Education, 10, 61-80. https://gpae.wcu.edu/disparate-methods-of-combining-test-and-assignment-scores-into-course-grades/
39. Turgut, M.F. (1983). Program değerlendirme. Cumhuriyet Döneminde Eğitim [Program evaluation. Education in the Republican Era]. 215-234.
40. Turgut, M.F., & Baykul, Y. (2015). Eğitimde ölçme ve değerlendirme [Measurement and evaluation in education]. Pegem Akademi.
41. URAP (2022). University Ranking by Academic Performance. https://newtr.urapcenter.org