Abstract
In this study, experts’ decisions and cut-off scores obtained from Angoff, Yes/No and Ebel standard setting methods have been compared. The research data consist of 489 student scores obtained from a achievement test of 20 items and 17 experts’ decisions, made according to three different standard setting methods for this achievement test. In the analysis of data, in order to set the compliance between experts, Kendall’s coefficient of concordance (W) for Angoff and Ebel methods, and Cohen’s Kappa statistics for Yes/No method have been calculated. Differences between the percentages of students, considered as successful according to three different standard setting methods, have been tested with the test of the difference between two correlated proportions. The students’ scores have been tailored to binary as successful-unsuccessful according to three methods; and the compliance between them has been examined with Cohen’s Kappa statistics. The compliance between the cutoff scores of three methods has been evaluated with Pearson Product-Moment Correlation Coefficient and the paired samples t test. At the end of this research, it is seen that the compliance of experts in the methods is at a reasonable level; there is significant difference on 0,01 error level among percentages of students considered as successful according to these three different standard setting methods and there is a compliance between students’ scores that have been tailored to binary as successful-unsuccessful according to three methods. It is concluded that there is a high relationship between Angoff and Yes/No methods; there is moderate relationship between Angoff and Ebel methods; there is no significant relationship between Yes/No and Ebel methods.
References
- Cizek, G. J ve Bunch, M. B. (2007). Standard setting: A guide to establishing and evaluating. California: Sage Publications.
- Crocker, L. ve Algina, J. (1986). Introduction to classical and modern test theory. Florida: Holt, Rinehart and Winston, Inc.
- Downing, S. M., Tekian, A. ve Yudkowsky, R. (2006). Research Methodology: Procedures for Establishing Defensible Absolute Passing Scores on Performance Examinations in Health Professions Education. Teaching and Learning in Medicine, 18 (1), 50 – 57.
- Ebel, R. L. (1972). Essentials of educational measurement. New Jersey: Prentice Hall.
- Hambleton, R. K. (2001). Setting performance standards on educational assessments and criteria for evaluating the process. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods and perspectives (ss. 89 – 116). Mahwah, NJ: Lawrence Erlbaum Associates.
- Impara, J. C. ve Plake, B. S. (1997). Standard Setting: An Alternative Approach. Journal of Educational Measurement, 34 (4), 353 – 366.
- Livingston, S. A. ve Zieky, M. J. (1982). Passing Scores (A Manual For Setting Standards Of Performance On Educational And Occupational Tests) [Elektronik Sürüm]. New Jersey: Educational Testing Service. Erişim: 12 Kasım 2009,
- http://www.ets.org/portal/site/ets/menuitem.c988ba0e5dd572bada20bc47c3921509/?vgnextoid=0dcfd639ef4a9010VgnVCM10000022f95190RCRDvevgnextchannel=dcb3be3a864f4010VgnVCM10000022f95190RCRD
- Murphy, K. R. ve Davidshofer, C. O. (1991). Psychological testing: Principles and applications. New Jersey: Prantice Hall.
- Zieky, M. J. (2001). So much has changed: How the setting of cutscores has evolved since the 1980s. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods and perspectives (s. 19- 51). Mahwah, NJ: Lawrence Erlbaum Associates.
Abstract
Bu araştırmada Angoff, Yes/No ve Ebel standart belirleme yöntemleri ile elde edilen uzman kararları ve kesme puanları karşılaştırılmıştır. Araştırmanın verilerini, 20 maddelik bir başarı testinden alınan 489 öğrenci puanı ve 17 uzmanın bu başarı testindeki maddeler için üç farklı standart belirleme yöntemine göre vermiş oldukları kararlar oluşturmaktadır. Verilerin analizi uzmanlar arası uyum Angoff ve Ebel yöntemleri için Kendall'ın uyuşum katsayısı (W) ve Yes/No yöntemi için Cohen'in Kappa istatistiği ile; üç farklı standart belirleme yöntemine göre başarılı sayılan öğrenci yüzdeleri arasındaki farklılıklar bağımlı iki oran arasındaki farkın testi ile; üç yöntemden elde edilen kesme puanlarına göre öğrenci puanları başarılı-başarısız olmak üzere yapay ikili hale getirilip aralarındaki uyuma Cohen'in Kappa istatistiği ile; üç yönteme ait kesme puanları arasındaki uyum ise Pearson Momentler Çarpımı Korelasyon Katsayısı ve bağımlı gruplar t testi ile test edilmiştir. Bu araştırma sonunda yöntemlerde uzmanlar arasında makul bir uyumun yakalanabildiği; üç farklı standart belirleme yöntemine göre başarılı sayılan öğrenci yüzdeleri arasında 0,01 hata düzeyinde manidar farklılık olduğu ve öğrencilerin başarılı-başarısız sayılma durumları bakımından uyum olduğu görülmüştür. Kesme puanları arasında Angoff-Yes/No yöntemleri arasında yüksek düzeyde, Angoff-Ebel yöntemleri arasında orta düzeyde bir ilişki olduğu ve Yes/No-Ebel yöntemleri arasında anlamlı bir ilişki olmadığı görülmüştür.
References
- Cizek, G. J ve Bunch, M. B. (2007). Standard setting: A guide to establishing and evaluating. California: Sage Publications.
- Crocker, L. ve Algina, J. (1986). Introduction to classical and modern test theory. Florida: Holt, Rinehart and Winston, Inc.
- Downing, S. M., Tekian, A. ve Yudkowsky, R. (2006). Research Methodology: Procedures for Establishing Defensible Absolute Passing Scores on Performance Examinations in Health Professions Education. Teaching and Learning in Medicine, 18 (1), 50 – 57.
- Ebel, R. L. (1972). Essentials of educational measurement. New Jersey: Prentice Hall.
- Hambleton, R. K. (2001). Setting performance standards on educational assessments and criteria for evaluating the process. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods and perspectives (ss. 89 – 116). Mahwah, NJ: Lawrence Erlbaum Associates.
- Impara, J. C. ve Plake, B. S. (1997). Standard Setting: An Alternative Approach. Journal of Educational Measurement, 34 (4), 353 – 366.
- Livingston, S. A. ve Zieky, M. J. (1982). Passing Scores (A Manual For Setting Standards Of Performance On Educational And Occupational Tests) [Elektronik Sürüm]. New Jersey: Educational Testing Service. Erişim: 12 Kasım 2009,
- http://www.ets.org/portal/site/ets/menuitem.c988ba0e5dd572bada20bc47c3921509/?vgnextoid=0dcfd639ef4a9010VgnVCM10000022f95190RCRDvevgnextchannel=dcb3be3a864f4010VgnVCM10000022f95190RCRD
- Murphy, K. R. ve Davidshofer, C. O. (1991). Psychological testing: Principles and applications. New Jersey: Prantice Hall.
- Zieky, M. J. (2001). So much has changed: How the setting of cutscores has evolved since the 1980s. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods and perspectives (s. 19- 51). Mahwah, NJ: Lawrence Erlbaum Associates.
Details
| Primary Language | Turkish |
|---|---|
| Journal Section | Articles |
| Authors | |
| Publication Date | March 13, 2014 |
| Published in Issue | Year 2014 Volume: 5 Issue: 1 |