The Analysis of Multiple Choice Items in Final Exam of Preparatory Students

Öz

ABSTRACT: The aim of this study is to examine the multiple choice final exams administered to the 210 non-compulsory preparatory school students. The study aims to analyze the exams in terms of three characteristics: item facility, item discrimination and distractor efficiency. The study had quantitative research design and the data were analyzed through Paired Samples T-Test and frequency analysis .The results of the study revealed that most items in final exams had moderate difficulty levels for the students. However, almost all items in the exams had low discrimination indices and some items had negative discrimination values. Furthermore, the results show that one third of the items in the exams had at least one non-functional distractor. At the end of the study, some guidelines were presented for teachers and test developers to make the items more functional.

Keywords: item analysis, item facility, item discrimination, distractor efficiency

Anahtar Kelimeler

• item facility,item discrimination,distractor efficiency

The Analysis of Multiple Choice Items in Final Exam of Preparatory Students

Öz

ÖZ: Bu çalışmanın amacı 210 isteğe bağlı hazırlık öğrencisine uygulanan çoktan seçmeli final sınavlarını incelemektir. Çalışma sınavları çoktan seçmeli soruların üç niteliği yani madde kolaylığı, madde ayırıcılığı ve çeldirici yeteneği açısından incelemeyi amaçlamaktadır. Türk alanyazında çoktan seçmeli testlerle ilgili birçok çalışma bulunmasına rağmen çoktan seçmeli soruları madde analizi bakımından inceleyen çok az çalışma bulunmaktadır. Bu yüzden bu çalışma iki çoktan seçmeli sınavı madde analizi açısından inceleyerek alanyazına katkıda bulunacaktır. Bu çalışmada nicel araştırma yöntemi kullanılmıştır ve verileri analiz etmek için tek örneklem t-testi ve sıklık analizi kullanılmıştır. Sonuçlar göstermektedir ki, sınavlardaki soruların çoğu öğrenciler için orta zorluk derecesine sahiptir. Ancak, soruların hemen hepsi çok düşük madde ayırıcılığı değerine sahiptir ve bazı soruların ayırıcılık değerinin negatif olduğu bulunmuştur. Ayrıca, sonuçlar sınavlardaki soruların üçte birinde en az bir tane işlevsiz çeldirici olduğunu göstermektedir. Çalışmanın sonunda, soruları daha etkili hale getirmek için öğretmenler ve soru geliştirenler için bazı yönergeler sunulmuştur.

Anahtar sözcükler: madde analizi,  madde kolaylığı, madde ayırıcılığı, çeldirici yeteneği

Anahtar Kelimeler

• madde kolaylığı,madde ayırıcılığı,çeldirci yeteneği

Kaynakça

1. Álvarez, I. A. (2013). Large-scale assessment of language proficiency: Theoretical and pedagogical reflections on the use of multiple-choice tests. International Journal of English Studies, 13(2), 21-38.
2. Atalmış, H. E. (2014). The impact of the Test Types and Number of Solution Steps of Multiple-Choice Items on Item Difficulty and Discrimination and Test Reliability. Published
3. Bodner, G. M. (1980). Statistical Analysis of Multiple-Choice Exams. Journal of Chemical Education, 57(3), 188-90.
4. Brown, J. D., & Hudson, T. (1998). The alternatives in language assessment. TESOL quarterly, 32(4), 653-675.
5. Brown, J. D. (2003). Norm-referenced item analysis (item facility and item discrimination). Statistics, 7(2).
6. Brown, H. D. (2004). Language assessment: Principles and classroom practice. NY: Pearson Education.
7. Bruno, J. E., & Dirkzwager, A. (1995). Determining the optimal number of alternatives to a multiple-choice test item: An information theoretic perspective. Educational and Psychological Measurement, 55(6), 959-966.
8. Buckles, S., & Siegfried, J. J. (2006). Using multiple-choice questions to evaluate in-depth learning of economics. The Journal of Economic Education, 37(1), 48-57.

9. Burton, R. F. (2001). Do Item-discrimination Indices Really Help Us to Improve Our Tests? Assessment & Evaluation in Higher Education, 26(3), 213-220.
10. Burton, R. F. (2004). Multiple choice and true/false tests: reliability measures and some implications of negative marking. Assessment & Evaluation in Higher Education, 29(5), 585-595.
11. Burton, R. F. (2005). Multiple‐choice and true/false tests: myths and misapprehensions. Assessment & Evaluation in Higher Education, 30(1), 65-72.
12. Bush, M. E. (2006). Quality assurance of multiple-choice tests. Quality Assurance in Education, 14(4), 398-404.
13. Bush, M. (2015). Reducing the need for guesswork in multiple-choice tests. Assessment & Evaluation in Higher Education, 40(2), 218-231.
14. Cechova, I., Neubauer, J., & Sedlacik, M. (2014). Computer-adaptive testing: item analysis and statistics for effective testing. In European Conference on e-Learning (p. 106). Academic Conferences International Limited.
15. Coniam, D. (2009). Investigating the quality of teacher-produced tests for EFL students and the effects of training in test development principles and practices on improving test quality. System, 37(2), 226-242.
16. Coombe, C. A., Folse, K. S., & Hubley, N. J. (2007). A practical guide to assessing English language learners. University of Michigan Press.
17. Costin, F. (1970). The optimal number of alternatives in multiple-choice achievement tests: Some empirical evidence for a mathematical proof. Educational and Psychological Measurement, 30(2), 353-358.
18. Costin, F. (1972). Three-choice versus four-choice items: Implications for reliability and validity of objective achievement tests. Educational and Psychological Measurement, 32(4), 1035-1038.
19. DiBattista, D., & Kurzawa, L. (2011). Examination of the Quality of Multiple-Choice Items on Classroom Tests. Canadian Journal for the Scholarship of Teaching and Learning, 2(2), 4.
20. Downing, S. M., & Haladyna, T. M. (1997). Test item development: Validity evidence from quality assurance procedures. Applied Measurement in Education, 10(1), 61-82.Downing, S. M. (2005). The effects of violating standard item writing principles on tests and students: the consequences of using flawed test items on achievement examinations in medical education. Advances in health sciences education, 10(2), 133-143.
21. Ebel, R. L. (1967). The relationship of item discrimination to test reliability. Journal of educational Measurement, 4(3), 125-128.
22. Ebel, R. L. (1969). Expected reliability as a function of choices per item. Educational and Psychological Measurement, 29(3), 565-570.
23. Gajjar, S., Sharma, R., Kumar, P., & Rana, M. (2014). Item and test analysis to identify quality multiple choice questions (MCQs) from an assessment of medical students of Ahmedabad, Gujarat. Indian Journal of Community Medicine, 39(1), 17.
24. Goodrich, H. C. (1977). Distractor efficiency in foreign language testing. TESOL Quarterly, 11 (1), 69-78.
25. Haladyna, T. M., & Downing, S. M. (1989). A taxonomy of multiple-choice item-writing rules. Applied measurement in education, 2(1), 37-50.
26. Haladyna, T. M., & Downing, S. M. (1993). How many options is enough for a multiple-choice test item?. Educational and Psychological Measurement, 53(4), 999-1010.
27. Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied measurement in education, 15(3), 309-333.
28. Hamzah, M. S. G., & Abdullah, S. K. (2011). Test Item Analysis: An Educator Professionalism Approach. Online Submission.
29. Jafarpur, A. (1999). Can the C-test be improved with the classical item analysis? System, 27(1), 79-89.
30. Jozefowicz, R. F., Koeppen, B. M., Case, S., Galbraith, R., Swanson, D., & Glew, R. H. (2002). The quality of in‐house médical school examinations. Academic Medicine, 77(2), 156-161.
31. Malau-Aduli, B. S., & Zimitat, C. (2012). Peer review improves the quality of MCQ examinations. Assessment & Evaluation in Higher Education, 37(8), 919-931.
32. Oluseyi, A. E., & Olufemi, A. T. (2011). The Analysis of Multiple Choice Item of the Test of an Introductory Course in Chemistry in a Nigerian University. International Journal of Learning, 18(4), 237-246.
33. Oppenheim, N. (2002). Empirical analysis of an examination based on the academy of legal studies in business test bank. Journal of Legal Studies Education, 20(2), 129-158.
34. Rogers, W. T., & Harley, D. (1999). An empirical comparison of three-and four-choice items and tests: susceptibility to testwiseness and internal consistency reliability. Educational and Psychological Measurement, 59(2), 234-247.
35. Stiggins, R. J., & Bridgeford, N. J. (1985). The ecology of classroom assessment. Journal of Educational Measurement, 22(4), 271-286.
36. Tarrant, M., Ware, J. & Mohammed, A. M. (2009). An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis. BMC Medical Education, 9(1), 40.
37. Tversky, A. (1964). On the optimal number of alternatives at a choice point. Journal of Mathematical Psychology, 1(2), 386-391.
38. Wainer, H. (1988). The future of item analysis. ETS Research Report Series, 1988(2).
39. Wallach, P. M., Crespo, L. M., Holtzman, K. Z., Galbraith, R. M., & Swanson, D. B. (2006). Use of a committee review process to improve the quality of course examinations. Advances in Health Sciences Education, 11(1), 61-68.
40. Ware, J., & Vik, T. (2009). Quality assurance of item writing: during the introduction of multiple choice questions in medicine for high stakes examinations. Medical teacher, 31(3), 238-243.