The Comparison of Methods in Multiple Response Item Type Scoring

Ömer Doğan; İsmail Karakaya

doi:10.52597/buje.1521294

TR EN

The Comparison of Methods in Multiple Response Item Type Scoring

Abstract

The aim of this study is to examine the scoring methods of the multiple-answer item type, where a question can have more than one correct answer. Concordantly, the differences regarding test difficulty, reliability, item difficulty and discrimination among eight scoring methods to score achievement tests comprising of multiple response item types were examined and compared. According to the data obtained, it is understood that the multiple-answer item type is applicable for both numerical and verbal lessons. In addition, it has been found that standardized scoring methods are in a way that allows the use of different scoring methods by offering alternatives according to the purpose of the measurement and evaluation, without being bound by a single rule. Thus, scoring methods can be determined according to the purpose of using the multiple-answer item type, and the use of the item type can be made widespread with similar studies.

Keywords

Multiple response item type, partial scoring, scoring methods.

References

Akhun, İ. (1995). İstatistiklerin manidarlığı ve örneklem [Significance of statistics and sampling]. Özel Basım.
Auer, S. & Tarasowa, D. (2013). Balanced scoring method for multiple-mark questions. Proceedings of the 5th International Conference on Computer Supported Education (CSEDU-2013), 411–416. https://doi.org/10.5220/0004384304110416
Baghaei, P. & Amrahi, N. (2011). The effects of the number of options on the psychometric characteristics of multiple choice items. Psychological Test and Assessment Modeling, 53(2), 192–211.
Baykul, Y. (2015). Eğitimde ve psikolojide ölçme: Klasik test teorisi ve uygulaması.[Measurement in education and psychology: Classical test theory and practice]. Pegem Akademi.
Ben-Simon, A., Budescu, D. V. & Nevo, B. (1997). A comparative study of measures of partial knowledge in multiple-choice tests. Applied Psychological Measurement, 21(1), 65–88.
Doğan, N. (2009). Çoktan seçmeli testler [Multiple choice items]. In H. Atılgan (Ed.), Eğitimde ölçme ve değerlendirme (pp. 223–268). Anı.
Doğanay, A. (2007). Üst düzey düşünme becerilerinin öğretimi [Teaching higher order thinking skills]. In A. Doğanay (Ed.), Öğretim ilke ve yöntemleri (pp. 280–327). Pegem Akademi.
Domnich, A., Panatto, D., Arata, L. Bevilacqua, I., Apprato, L., Gasparini, R. & Amicizia, D. (2015). Impact of different scoring algorithms applied to multiple-mark survey items on outcome assessment: An in-field study on health-related knowledge. Journal of Preventive Medicine and Hygiene, 56(4), E162.
Duman, E. Z. (2018). Bir düşünme türü olarak esnek düşünme [Flexible thinking as a type of thinking]. Turkish Studies Social Sciences, 13(26), 547–561.
Erkuş, A. (2016). Psikolojide ölçme ve ölçek geliştirme-I [Measurement and scale development in psychology-I]. Pegem Akademi.

Frisbie, D. A. & Sweeney, D. C. (1982). The relative merits of multiple true-false achievement tests. Journal of Educational Measurement, 19(1), 29–35.
Hsu, T.C., Moss, P.A., Khampalikit, C. (1984). The merits of multiple-answer items as evaluated by using six scoring formulas. The Journal of Experimental Education, 52(3), 152–158.
Jodoin, M. G. (2003). Measurement efficiency of innovative item formats in computer-based testing. Journal of Measurement, 40(1), 1–15.
Karakaya, İ. & Doğan, Ö. (2020). Çoklu cevaplı madde türünün psikometrik özellikleri hakkında ödm çalışanı, alan uzmanı ve akademisyenlerin görüşlerinin incelenmesi [Examination of the opinions of the staff of the assessment and evaluation center, field specialists and academicians about the psychometric properties of the multiple response item type] Ankara II. Uluslararası Bilimsel Araştırmalar Kongresi Bildirileri, 1, 271–279.
Karasar, N. (2012). Bilimsel irade algı çerçevesi ile bilimsel araştırma yöntemi [Scientific research method with the framework of perception of scientific will]. Nobel Yayınları.
Kilmen, S. (2012). Madde analizi, madde seçimi ve yorumlanması [Item analysis, item selection, and interpretation]. In N. Çıkrıkçı-Demirtaşlı (Ed.), Eğitimde ölçme ve değerlendirme, (pp. 363–385). Elhan.
Koç, N. (2009), Eğitimimizde psikolojik testlerin (standart testlerin) kullanım durumu, sorunlar ve öneriler. [Use of psychological tests (standard tests) in our education, problems and suggestions] I. Ulusal Eğitimde ve Psikolojide Ölçme ve Değerlendirme Kongresi bildirileri 1, 305–316.
Liu, O. L., Lee, H. S., & Linn, M. C. (2011). An Investigation of Explanation Multiple-Choice Items in Science Assessment. Educational Assessment, 16(3), 164–184. https://doi.org/10.1080/10627197.2011.611702
Ma, X. (2004). An investigation of alternative approaches to scoring multiple response items on a certification exam. (Publication No. 305176174) [Doctoral dissertation, University of Massachusetts Amherst]. ProQuest Dissertations Publishing.
Morgan, M. (1979). MCQ: An interactive computer program for multiple-choice self-testing. Biochemical Education, 7(3), 67–69.
Muckle, T., Becker, K. A. & Wu, B. (2011). Investigating the multiple answer multiple choice item format. Presentation and scoring consideration. 2011 Annual Meeting of the National Council on Measurement in Education, New Orleans, LA.
Özçelik, D. A. (2013). Test hazırlama kılavuzu [Test preparation guide]. Pegem Akademi.
Peterson, A., Craig M. & Danny, P. (2016). Employing multiple answer multiple choice questions. Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education, 252–253. https://doi.org/10.1145/2899415.2925503
Ripkey, D., Case, S. & Swanson, D. (1996). A “new” item format for assessing aspects of clinical competence. Academic Medicine, 71(10), 34–36.
Siddiqui, N. I., Bhavsar, V. H., Bhavsar, A. V., & Bose, S. (2016). Contemplation on marking scheme for Type X multiple choice questions, and an illustration of a practically applicable scheme. Indian journal of pharmacology, 48(2), 114–121.
Tan, Ş. (2010). Öğretimde ölçme ve değerlendirme [Measurement and evaluation in teaching]. Pegem Akademi.
Tan, Ş. (2016). SPSS ve Excel uygulamalı temel istatistik-1 [Basic statistics-1 with SPSS and Excel]. Pegem Akademi.
Turgut, M. F. & Baykul, Y. (2013). Eğitimde ölçme ve değerlendirme [Measurement and evaluation in education]. Pegem Akademi.
Umay, A. (1997). Yanıtlayıcı davranışlarının analizi yolu ile matematikte problem çözümleri için bir güvenirlik ve geçerlik araştırması [A reliability and validity study for problem solving in mathematics through the analysis of respondent behavior]. Hacettepe University Journal of Education, 13, 47–56.
Verbic, S. (2012). Information value of multiple response questions. Psihologija, 45(4), 467–485.
Wan, L. & Henly, G. A. (2012). Measurement properties of two innovative item formats in a computer-based test. Applied Measurement in Education, 25, 58–78.

Details

Primary Language

English

Subjects

Classroom Measurement Practices, Measurement and Evaluation in Education (Other)

Journal Section

Research Article

Authors

Ömer Doğan ^*
0000-0001-5169-520X
Türkiye

İsmail Karakaya
0000-0003-4308-6919
Türkiye

Publication Date

May 1, 2025

Submission Date

July 23, 2024

Acceptance Date

March 17, 2025

Published in Issue

Year 2025 Volume: 42 Number: 1

DOI

https://doi.org/10.52597/buje.1521294

IZ

https://izlik.org/JA63FX22UL

APA

Doğan, Ö., & Karakaya, İ. (2025). The Comparison of Methods in Multiple Response Item Type Scoring. Bogazici University Journal of Education, 42(1), 23-45. https://doi.org/10.52597/buje.1521294

AMA

1.Doğan Ö, Karakaya İ. The Comparison of Methods in Multiple Response Item Type Scoring. BUJE. 2025;42(1):23-45. doi:10.52597/buje.1521294

Chicago

Doğan, Ömer, and İsmail Karakaya. 2025. “The Comparison of Methods in Multiple Response Item Type Scoring”. Bogazici University Journal of Education 42 (1): 23-45. https://doi.org/10.52597/buje.1521294.

EndNote

Doğan Ö, Karakaya İ (May 1, 2025) The Comparison of Methods in Multiple Response Item Type Scoring. Bogazici University Journal of Education 42 1 23–45.

IEEE

[1]Ö. Doğan and İ. Karakaya, “The Comparison of Methods in Multiple Response Item Type Scoring”, BUJE, vol. 42, no. 1, pp. 23–45, May 2025, doi: 10.52597/buje.1521294.

ISNAD

Doğan, Ömer - Karakaya, İsmail. “The Comparison of Methods in Multiple Response Item Type Scoring”. Bogazici University Journal of Education 42/1 (May 1, 2025): 23-45. https://doi.org/10.52597/buje.1521294.

JAMA

1.Doğan Ö, Karakaya İ. The Comparison of Methods in Multiple Response Item Type Scoring. BUJE. 2025;42:23–45.

MLA

Doğan, Ömer, and İsmail Karakaya. “The Comparison of Methods in Multiple Response Item Type Scoring”. Bogazici University Journal of Education, vol. 42, no. 1, May 2025, pp. 23-45, doi:10.52597/buje.1521294.

Vancouver

1.Ömer Doğan, İsmail Karakaya. The Comparison of Methods in Multiple Response Item Type Scoring. BUJE. 2025 May 1;42(1):23-45. doi:10.52597/buje.1521294

Unless otherwise stated, all content on this site is licensed under a Creative Commons Attribution 4.0 International Licence (CC BY 4.0).

The Comparison of Methods in Multiple Response Item Type Scoring

Çoklu Cevaplı Madde Türünün Puanlanmasında Yöntemlerin Karşılaştırılması

Abstract

Keywords

The Comparison of Methods in Multiple Response Item Type Scoring

Abstract

Keywords

References

Details

Primary Language

Subjects

Journal Section

Authors

Publication Date

Submission Date

Acceptance Date

Published in Issue

DOI

IZ

Cite