Three, four, and none of the above options in multiple-choice items

Year 2017, Volume: 6 Issue: 4, 143 - 157, 10.10.2017

Erkan Atalmış , Neal Kingston

https://doi.org/10.19128/turje.333687

Cited By: 2

Abstract

High-quality
multiple-choice (MC) items are essential for creating efficient, valid
assessments. Haladyna, Downing, and Rodriguez (2002) suggested that using
plausible distractors is crucial to achieving this goal, although distractor
creation can be time-consuming and challenging. Haladyna et al. (2002) provided
two related test development guidelines: #18, “Write as many plausible
distractors as you can,” and #25, “Use carefully None of the above.”
This research aims to test the impact of these two guidelines on item
difficulty (p), item discrimination (r), and test reliability for mathematics
items empirically. The research findings have revealed that item discrimination
and test reliability were not statistically different across MC items with four
options, three options, and NOTA options while the
means of item difficulty of four-option MC items was not statistically
different from those of three-option and NOTA-option MC items. However, the
mean of item difficulty of NOTA-option MC items was statistically lower than
those of three-option.

Keywords

assessment, item-writing guidelines, multiple-choice items, number of options, none of the above

Çoktan seçmeli sorularda üç, dört ve hiçbiri seçenekleri

Abstract

Etkili ve geçerli
değerlendirmeler yapabilmek için yüksek kaliteli çoktan seçmeli sorular
(maddeler) oluşturmak gereklidir. Bu amaç doğrultusunda Haladyna, Downing, and
Rodriguez (2002) çoktan seçmeli madde oluşturma sürecinde uygun çeldiricilerin
kullanılmasının önemli olduğunu vurgulamaktadırlar. Ancak bu çeldirilerin
oluşturulması zaman alıcı ve zahmetli bir süreçtir. Bu bağlamda Haladyna ve
diğerleri (2002) test geliştirme ile ilişkili olan "Olabildiği kadar uygun ve mantıklı çeldiricileri yazınız" ve
"Hiçbiri seçeneğini dikkatli kullanınız"
ilkelerini rapor etmektedirler. Bu araştırmanın amacı bu iki ilke kullanımının
matematik sorularınn zorluk derecesine, ayırt edicilik derecesine ve test
güvenirliğine etkisini test etmektir. Araştırma sonucunda, madde ayırt edicilik
derecesi ve test güvenirliği dört seçenekli, üç seçenekli, ve “hiçbiri”
seçeneğine sahip maddeler arasında istatistiksel olarak anlamlı bir farklılık
göstermediği bulunmuştur. Benzer şekilde, dört seçenekli maddelerin madde
zorluk dereceleri, üç seçenekli ve “hiçbiri” seçeneğine sahip maddelerin madde
zorluk derecelerinden istatistiksel olarak anlamlı bir farklılık göstermediği
bulunurken, “hiçbiri” seçeneğine sahip maddelerin madde zorluk dereceleri üç
seçenekli maddelerin madde zorluk derecelerinden istatistiksel olarak düşük
olduğu elde edilmiştir.

Keywords

Değerlendirme, soru-yazma teknikleri, çoktan seçmeli sorular, seçenek sayısı, yukarıdakilerden hiçbiri

References

• Abad, F. J., Olea, J., & Ponsoda, V. (2001). Analysis of the optimum number of alternatives from the Item Response Theory. Psicothema, 13(1), 152-158.
• Baghaei, P., & Amrahi, N. (2011). The effects of the number of options on the psychometric characteristics of multiple choice items. Psychological Test and Assessment Modeling, 53(2), 192-211.
• Burton, S.J., Sudweeks, R. R., Merrill, P. F., & Wood, B. (1991). How to prepare better multiple-choice test items: Guidelines for university faculty. Brigham Young University Department of Instructional Science. Retrieved from http://testing.byu.edu/info/handbooks/betteritems.pdf
• Cizek, G. J., & O'Day, D. M. (1994). Further investigation of nonfunctioning options in multiple-choice test items. Educational and Psychological Measurement, 54(4), 861–872. doi:10.1177/0013164494054004002
• Common Core State Standards Initiative (CCSSI). (2010). Common Core state standards for mathematics. Washington, DC: National Governors Association Center for Best Practices and the Council of Chief State School Officers.
• Crehan, K. D., & Haladyna, T. M. (1991).The validity of two item-writing rules. The Journal of Experimental Education, 59(2), 183–192. doi:10.1080/00220973.1991.10806560
• Crehan, K. D., Haladyna, T. M., & Brewer, B. W. (1993). Use of an inclusive option and the optimal number of options for multiple-choice items. Educational and Psychological Measurement, 53(1), 241-247. doi:10.1177/0013164493053001027
• Dehnad, A., Nasser, H., & Hosseini, A. F. (2014). A comparison between three-and four-option multiple choice questions. Procedia-Social and Behavioral Sciences, 98, 398–403. doi:10.1016/j.sbspro.2014.03.432
• Delgado, A. R., & Prieto, G. (1998). Further evidence favoring three-option items in multiple-choice tests. European Journal of Psychological Assessment, 14(3), 197–201. doi:10.1027/1015-5759.14.3.197
• DiBattista, D., Sinnige-Egger, J., & Fortuna, G. (2014). The “none of the above” option in multiple-choice testing: An experimental study. The Journal of Experimental Education, 82(2), 168-183. doi:10.1080/00220973.2013.795127
• Downing, S. M. (2005). The effects of violating standard item writing principles on tests and students: The consequences of using flawed test items on achievement examinations in medical education. Advances in health sciences education, 10(2), 133-143.doi:10.1007/s10459-004-4019-5
• Duhachek, A., & Iacobucci, D. (2004). Alpha's standard error (ASE): an accurate and precise confidence interval estimate. Journal of Applied Psychology, 89(5), 792–808. doi:10.1037/0021-9010.89.5.792
• Frary, R. B. (1991). The none-of-the-above option: An empirical study. Applied Measurement in Education, 4(2), 115–124. doi:10.1207/s15324818ame0402_2
• Frey, B.B., Petersen, S., Edwards, L. M., Pedrotti, J.T., & Peyton, V. (2005). Item-writing rules: Collective wisdom. Teaching and Teacher Education, 21(4), 357–364.doi:10.1016/j.tate.2005.01.008
• Haladyna, T. M., & Downing, S. M. (1989). A taxonomy of multiple-choice item-writing rules. Applied Measurement in Education, 2(1), 37–50. doi:10.1207/s15324818ame0201_3
• Haladyna, T. M., & Downing, S. M. (1993). How many options is enough for a multiple-choice test item? Educational and psychological measurement, 53(4), 999–1010.doi:10.1177/0013164493053004013
• Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choiceitem-writing guidelines for classroom assessment. Applied Measurement in Education,15(3), 309–334. doi:10.1207/S15324818AME1503_5
• Haladyna, T. M., & Rodriguez, M. C. (2013). Developing and validating test items. New York, NY: Routledge.
• Hansen, J. D., & Dexter, L. (1997). Quality multiple-choice test questions: Item-writing guidelines and an analysis of auditing testbanks. Journal of Education for Business 73(2), 94–97. doi:10.1080/08832329709601623
• IBM Corp. Released 2012. IBM SPSS Statistics for Windows, Version 21.0. Armonk, NY: IBM Corp.
• Kingston, N.M., & Kramer, L.B. (2013). Highstakes test construction and test use. In T.D. Little (Ed.),The Oxford handbook of quantitative methods: Vol. 1 foundations (pp. 189–205). New York, NY: Oxford University Press.
• Kingston, N.M., Scheuring, S.T., & Kramer, L.B. (2013). Test Development Strategies. In Kurt Geisinger (Ed.) APA Handbook of Testing and Assessment in Psychology. Washington, DC: APA Books.
• Knowles, S. L.,&Welch, C. A. (1992). A meta-analytic review of item discrimination and difficulty in multiple-choice items using "none-of-the-above.” Educational and Psychological Measurement, 52(3), 571–577. doi:10.1177/0013164492052003006
• Kolstad, R. K., & Kolstad, R. A. (1991). The option “none of these” improves multiple-choice test items. Journal of Dental Education, 55(2), 161–163.
• Landrum, R. E., Cashin, J. R., & Theis, K. S. (1993). More evidence in favor of three-option multiple-choice tests. Educational and Psychological Measurement, 53(3), 771–778. doi:10.1177/0013164493053003021
• Lee, H., & Winke, P. (2013). The differences among three-, four-, and five-option-item formats in the context of a high-stakes English-language listening test. Language Testing, 30(1), 99-123.
• McCoubrie, P. (2004). Improving the fairness of multiple-choice questions: a literature review. Medical Teacher, 26(8), 709–712. doi:10.1080/01421590400013495
• Odegard, T. N., & Koen, J. D. (2007). “None of the above” as a correct and incorrect alternative on a multiple-choice test: Implications for the testing effect. Memory, 15(8), 873–885. doi:10.1080/09658210701746621
• Rich, C. E., & Johanson, G. A. (1990, April).An item-level analysis of “none of the above.”Paper presented at the annual meeting of the American Educational Research Association, Boston, MA. Retrieved from http://files.eric.ed.gov/fulltext/ED400299.pdf
• Rodriguez, M. C. (2005). Three options are optimal for multiple-choice items: A meta-analysisof 80 years of research. Educational Measurement: Issues and Practice, 24(2), 3-13. doi:0.1111/j.1745-3992.2005.00006.x
• Rogers, W. T., & Harley, D. (1999). An empirical comparison of three-and four-choice items and tests: Susceptibility to testwiseness and internal consistency reliability. Educational and Psychological Measurement, 59(2), 234–247. doi:10.1177/00131649921969820
• StataCorp (2013). Stata: release 13 - statistical software. College Station, TX: StataCorp LP
• Shizuka, T., Takeuchi, O., Yashima, T. & Yoshizawa, K. (2006). A comparison of three-and four-option English tests for university entrance selection purposes in Japan. Language Testing, 23(1), 35–57. doi:10.1191/0265532206lt319oa
• Tarrant, M., Ware, J., & Mohammed, A. M. (2009). An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis.BMC Medical Education, 9(1), 40. doi:10.1186/1472-6920-9-40
• Tarrant, M., & Ware, J. (2010). A comparison of the psychometric properties of three-and four-option multiple-choice questions in nursing assessments. Nurse Education Today, 30(6), 539–543. doi:10.1016/j.nedt.2009.11.002
• Tollefson, N. (1987). A comparison of the item difficulty and item discrimination of multiple-choice items using the"none of the above" and one correct response options. Educational and Psychological Measurement, 47(2), 377–383.doi:10.1177/0013164487472010
• Trevisan, M. S., Sax, G., & Michael, W. B. (1991). The effects of the number of options per item and student ability on test validity and reliability. Educational and Psychological Measurement, 51(4),829–837. doi:10.1177/001316449105100404