TY  - JOUR
T1  - Three, four, and none of the above options in multiple-choice items
TT  - Çoktan seçmeli sorularda üç, dört ve hiçbiri seçenekleri
AU  - Atalmış, Erkan
AU  - Kingston, Neal
PY  - 2017
DA  - October
Y2  - 2017
DO  - 10.19128/turje.333687
JF  - Turkish Journal of Education
JO  - TURJE
PB  - Mehmet TEKEREK
WT  - DergiPark
SN  - 2147-2858
SP  - 143
EP  - 157
VL  - 6
IS  - 4
LA  - en
AB  - High-qualitymultiple-choice (MC) items are essential for creating efficient, validassessments. Haladyna, Downing, and Rodriguez (2002) suggested that usingplausible distractors is crucial to achieving this goal, although distractorcreation can be time-consuming and challenging. Haladyna et al. (2002) providedtwo related test development guidelines: #18, “Write as many plausibledistractors as you can,” and #25, “Use carefully None of the above.”This research aims to test the impact of these two guidelines on itemdifficulty (p), item discrimination (r), and test reliability for mathematicsitems empirically. The research findings have revealed that item discriminationand test reliability were not statistically different across MC items with fouroptions, three options, and NOTA options while themeans of item difficulty of four-option MC items was not statisticallydifferent from those of three-option and NOTA-option MC items. However, themean of item difficulty of NOTA-option MC items was statistically lower thanthose of three-option.
KW  - assessment
KW  - item-writing guidelines
KW  - multiple-choice items
KW  - number of options
KW  - none of the above
N2  - Etkili ve geçerlideğerlendirmeler yapabilmek için yüksek kaliteli çoktan seçmeli sorular(maddeler) oluşturmak gereklidir. Bu amaç doğrultusunda Haladyna, Downing, andRodriguez (2002) çoktan seçmeli madde oluşturma sürecinde uygun çeldiricilerinkullanılmasının önemli olduğunu vurgulamaktadırlar. Ancak bu çeldirilerinoluşturulması zaman alıcı ve zahmetli bir süreçtir. Bu bağlamda Haladyna vediğerleri (2002) test geliştirme ile ilişkili olan "Olabildiği kadar uygun ve mantıklı çeldiricileri yazınız" ve"Hiçbiri seçeneğini dikkatli kullanınız"ilkelerini rapor etmektedirler. Bu araştırmanın amacı bu iki ilke kullanımınınmatematik sorularınn zorluk derecesine, ayırt edicilik derecesine ve testgüvenirliğine etkisini test etmektir. Araştırma sonucunda, madde ayırt edicilikderecesi ve test güvenirliği dört seçenekli, üç seçenekli, ve “hiçbiri”seçeneğine sahip maddeler arasında istatistiksel olarak anlamlı bir farklılıkgöstermediği bulunmuştur. Benzer şekilde, dört seçenekli maddelerin maddezorluk dereceleri, üç seçenekli ve “hiçbiri” seçeneğine sahip maddelerin maddezorluk derecelerinden istatistiksel olarak anlamlı bir farklılık göstermediğibulunurken, “hiçbiri” seçeneğine sahip maddelerin madde zorluk dereceleri üçseçenekli maddelerin madde zorluk derecelerinden istatistiksel olarak düşükolduğu elde edilmiştir.
CR  - Abad, F. J., Olea, J., & Ponsoda, V. (2001). Analysis of the optimum number of alternatives from the Item Response Theory. Psicothema, 13(1), 152-158.
CR  - Baghaei, P., & Amrahi, N. (2011). The effects of the number of options on the psychometric characteristics of multiple choice items. Psychological Test and Assessment Modeling, 53(2), 192-211.
CR  - Burton, S.J., Sudweeks, R. R., Merrill, P. F., & Wood, B. (1991). How to prepare better multiple-choice test items: Guidelines for university faculty. Brigham Young University Department of Instructional Science. Retrieved from http://testing.byu.edu/info/handbooks/betteritems.pdf
CR  - Cizek, G. J., & O'Day, D. M. (1994). Further investigation of nonfunctioning options in multiple-choice test items. Educational and Psychological Measurement, 54(4), 861–872. doi:10.1177/0013164494054004002
CR  - Common Core State Standards Initiative (CCSSI). (2010). Common Core state standards for mathematics. Washington, DC: National Governors Association Center for Best Practices and the Council of Chief State School Officers.
CR  - Crehan, K. D., & Haladyna, T. M. (1991).The validity of two item-writing rules. The Journal of Experimental Education, 59(2), 183–192. doi:10.1080/00220973.1991.10806560
CR  - Crehan, K. D., Haladyna, T. M., & Brewer, B. W. (1993). Use of an inclusive option and the optimal number of options for multiple-choice items. Educational and Psychological Measurement, 53(1), 241-247. doi:10.1177/0013164493053001027
CR  - Dehnad, A., Nasser, H., & Hosseini, A. F. (2014). A comparison between three-and four-option multiple choice questions. Procedia-Social and Behavioral Sciences, 98, 398–403. doi:10.1016/j.sbspro.2014.03.432
CR  - Delgado, A. R., & Prieto, G. (1998). Further evidence favoring three-option items in multiple-choice tests. European Journal of Psychological Assessment, 14(3), 197–201. doi:10.1027/1015-5759.14.3.197
CR  - DiBattista, D., Sinnige-Egger, J., & Fortuna, G. (2014). The “none of the above” option in multiple-choice testing: An experimental study. The Journal of Experimental Education, 82(2), 168-183. doi:10.1080/00220973.2013.795127
CR  - Downing, S. M. (2005). The effects of violating standard item writing principles on tests and students: The consequences of using flawed test items on achievement examinations in medical education. Advances in health sciences education, 10(2), 133-143.doi:10.1007/s10459-004-4019-5
CR  - Duhachek, A., & Iacobucci, D. (2004). Alpha's standard error (ASE): an accurate and precise confidence interval estimate. Journal of Applied Psychology, 89(5), 792–808. doi:10.1037/0021-9010.89.5.792
CR  - Frary, R. B. (1991). The none-of-the-above option: An empirical study. Applied Measurement in Education, 4(2), 115–124. doi:10.1207/s15324818ame0402_2
CR  - Frey, B.B., Petersen, S., Edwards, L. M., Pedrotti, J.T., & Peyton, V. (2005). Item-writing rules: Collective wisdom. Teaching and Teacher Education, 21(4), 357–364.doi:10.1016/j.tate.2005.01.008
CR  - Haladyna, T. M., & Downing, S. M. (1989). A taxonomy of multiple-choice item-writing rules. Applied Measurement in Education, 2(1), 37–50. doi:10.1207/s15324818ame0201_3
CR  - Haladyna, T. M., & Downing, S. M. (1993). How many options is enough for a multiple-choice test item? Educational and psychological measurement, 53(4), 999–1010.doi:10.1177/0013164493053004013
CR  - Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choiceitem-writing guidelines for classroom assessment. Applied Measurement in Education,15(3), 309–334. doi:10.1207/S15324818AME1503_5
CR  - Haladyna, T. M., & Rodriguez, M. C. (2013). Developing and validating test items. New York, NY: Routledge.
CR  - Hansen, J. D., & Dexter, L. (1997). Quality multiple-choice test questions: Item-writing guidelines and an analysis of auditing testbanks. Journal of Education for Business 73(2), 94–97. doi:10.1080/08832329709601623
CR  - IBM Corp. Released 2012. IBM SPSS Statistics for Windows, Version 21.0. Armonk, NY: IBM Corp.
CR  - Kingston, N.M., & Kramer, L.B. (2013). Highstakes test construction and test use. In T.D. Little (Ed.),The Oxford handbook of quantitative methods: Vol. 1 foundations (pp. 189–205). New York, NY: Oxford University Press.
CR  - Kingston, N.M., Scheuring, S.T., & Kramer, L.B. (2013). Test Development Strategies. In Kurt Geisinger (Ed.) APA Handbook of Testing and Assessment in Psychology. Washington, DC: APA Books.
CR  - Knowles, S. L.,&Welch, C. A. (1992). A meta-analytic review of item discrimination and difficulty in multiple-choice items using "none-of-the-above.” Educational and Psychological Measurement, 52(3), 571–577. doi:10.1177/0013164492052003006
CR  - Kolstad, R. K., & Kolstad, R. A. (1991). The option “none of these” improves multiple-choice test items. Journal of Dental Education, 55(2), 161–163.
CR  - Landrum, R. E., Cashin, J. R., & Theis, K. S. (1993). More evidence in favor of three-option multiple-choice tests. Educational and Psychological Measurement, 53(3), 771–778. doi:10.1177/0013164493053003021
CR  - Lee, H., & Winke, P. (2013). The differences among three-, four-, and five-option-item formats in the context of a high-stakes English-language listening test. Language Testing, 30(1), 99-123.
CR  - McCoubrie, P. (2004). Improving the fairness of multiple-choice questions: a literature review. Medical Teacher, 26(8), 709–712. doi:10.1080/01421590400013495
CR  - Odegard, T. N., & Koen, J. D. (2007). “None of the above” as a correct and incorrect alternative on a multiple-choice test: Implications for the testing effect. Memory, 15(8), 873–885. doi:10.1080/09658210701746621
CR  - Rich, C. E., & Johanson, G. A. (1990, April).An item-level analysis of “none of the above.”Paper presented at the annual meeting of the American Educational Research Association, Boston, MA. Retrieved from http://files.eric.ed.gov/fulltext/ED400299.pdf
CR  - Rodriguez, M. C. (2005). Three options are optimal for multiple-choice items: A meta-analysisof 80 years of research. Educational Measurement: Issues and Practice, 24(2), 3-13. doi:0.1111/j.1745-3992.2005.00006.x
CR  - Rogers, W. T., & Harley, D. (1999). An empirical comparison of three-and four-choice items and tests: Susceptibility to testwiseness and internal consistency reliability. Educational and Psychological Measurement, 59(2), 234–247. doi:10.1177/00131649921969820
CR  - StataCorp (2013). Stata: release 13 -  statistical software. College Station, TX: StataCorp LP
CR  - Shizuka, T., Takeuchi, O., Yashima, T. & Yoshizawa, K. (2006). A comparison of three-and four-option English tests for university entrance selection purposes in Japan. Language Testing, 23(1), 35–57. doi:10.1191/0265532206lt319oa
CR  - Tarrant, M., Ware, J., & Mohammed, A. M. (2009). An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis.BMC Medical Education, 9(1), 40. doi:10.1186/1472-6920-9-40
CR  - Tarrant, M., & Ware, J. (2010). A comparison of the psychometric properties of three-and four-option multiple-choice questions in nursing assessments. Nurse Education Today, 30(6), 539–543. doi:10.1016/j.nedt.2009.11.002
CR  - Tollefson, N. (1987). A comparison of the item difficulty and item discrimination of multiple-choice items using the"none of the above" and one correct response options. Educational and Psychological Measurement, 47(2), 377–383.doi:10.1177/0013164487472010
CR  - Trevisan, M. S., Sax, G., & Michael, W. B. (1991). The effects of the number of options per item and student ability on test validity and reliability. Educational and Psychological Measurement, 51(4),829–837. doi:10.1177/001316449105100404
UR  - https://doi.org/10.19128/turje.333687
L1  - https://dergipark.org.tr/en/download/article-file/349036
ER  -