A Comparative Analysis of Multiple choice and Cloze Tests in Terms of Reading Comprehension and Reader Proficiency Levels

Abstract

The aim of this study is to examine the relationship between multiple choice tests and cloze tests, and to evaluate the consistency of both test types in determining reading proficiency levels. A total of 102 middle school students from the seventh and eighth grades participated in the study. The findings revealed a strong positive correlation between scores on multiple choice and cloze tests, and that cloze test scores significantly predicted performance on the multiple choice test. These results suggest that both test types assess common cognitive processes and can be used as complementary tools for evaluating reading comprehension. Additionally, the study investigated the predictive power of multiple choice questions at different cognitive levels of Bloom’s Taxonomy on cloze test performance. Among these, questions targeting the analysis level demonstrated the strongest predictive effect. While questions at the remembering and understanding levels also significantly predicted cloze test performance, those at the application level did not yield a significant effect. This indicates that cloze tests are particularly related to midlevel cognitive processes, such as text analysis and comprehension. The analysis also revealed limited agreement between the two test formats in classifying students’ reading proficiency levels, with more than half of the participants being placed in different categories depending on the test type. These findings underscore the limitations of relying solely on a single test format and highlight the necessity of using diverse assessment tools to ensure a more comprehensive evaluation of reading skills.

Keywords

• reading comprehension
• cloze tests
• multiple choice tests
• measurment
• test development.

Çoktan Seçmeli ve Boşluk Doldurma Testlerinin Okuduğunu Anlama ve Okur Düzeyleri Açısından Karşılaştırılması

Abstract

Bu çalışmanın amacı, çoktan seçmeli testler ile boşluk doldurma testleri arasındaki ilişkiyi incelemek ve her iki test türünün okur düzeylerine ilişkin uyumunu değerlendirmektir. Araştırmaya yedinci ve sekizinci sınıfta öğrenim gören toplam 102 ortaokul öğrencisi katılmıştır. Araştırma sonuçları çoktan seçmeli test puanı ile boşluk doldurma testi puanı arasında pozitif yönde yüksek bir ilişki olduğunu ve boşluk doldurma testi puanlarının çoktan seçmeli test puanlarını anlamlı düzeyde yordadığını ortaya koymuştur. Bu bulgular, her iki test türünün ortak bilişsel süreçleri ölçtüğünü ve birbirlerini tamamlayıcı ölçme araçları olarak kullanılabileceğini göstermektedir. Ayrıca araştırmada Bloom Taksonomisi'nin farklı bilişsel düzeylerindeki çoktan seçmeli test sorularının boşluk doldurma testi başarısını yordama gücü incelenmiştir. Çözümleme düzeyindeki soruların boşluk doldurma testindeki başarıyı yordamada en güçlü etkiye sahip olduğu bulunmuştur. Hatırlama ve anlama düzeyindeki sorular da boşluk doldurma test başarısını anlamlı bir şekilde yordarken uygulama düzeyindeki soruların anlamlı bir etkisi görülmemiştir. Bu durum, boşluk doldurma testlerinin özellikle metni analiz etme ve anlama gibi orta düzey bilişsel süreçlerle ilişkili olduğunu göstermektedir. Okur düzeylerinin belirlenmesi konusunda iki test türü arasında sınırlı bir uyum tespit edilmiştir. Katılımcıların yarısından fazlası, iki test türüne göre farklı okur düzeylerine yer almaktadır. Bu sonuç, tek bir test türüne bağlı kalmanın yeterli olmayabileceğini ve farklı ölçme araçlarının birlikte kullanılmasının gerekli olduğunu göstermektedir.

Keywords

• okuduğunu anlama
• boşluk doldurma testi
• çoktan seçmeli test
• ölçme
• test geliştirme.

References

1. Aitken, K. G. (1977). Using cloze procedure as an overall language proficiency test. TESOL Quarterly, 11(1), 59–67. https://doi.org/10.2307/3585592
2. Alderson, J. C. (1979). The effect on the cloze test of changes in deletion frequency. Journal of Research in Reading, 2(2), 108–119.
3. Alderson, J. C. (1980). Native and nonnative speaker performance on cloze tests. Language Learning, 30(1), 59–76. https://doi.org/10.1111/j.1467-1770.1980.tb00151.x
4. Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., Raths, J., & Wittrock, M. C. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Pearson Education Limited.
5. Anikin, A., & Sychev, O. (2020). Ontology-based modelling for learning on Bloom’s taxonomy comprehension level. In A. V. Samsonovich (Ed.), Biologically inspired cognitive architectures 2019 (pp. 22–27). Springer International Publishing.
6. Auphan, P., Ecalle, J., & Magnan, A. (2019). Computer-based assessment of reading ability and subtypes of readers with reading comprehension difficulties: A study in French children from G2 to G9. European Journal of Psychology of Education, 34(3), 641–663. https://doi.org/10.1007/s10212-018-0396-7
7. Baghaei, P., & Ravand, H. (2019). Method bias in cloze tests as reading comprehension measures. SAGE Open, 9(1), 2158244019832706. https://doi.org/10.1177/2158244019832706
8. Bilki, U. (2011). The effectiveness of cloze tests in assessing the speaking/writing skills of university EFL learners [Master’s thesis, Bilkent University]. https://repository.bilkent.edu.tr

9. Bormuth, J. R. (1967). Comparable cloze and multiple choice comprehension test scores. Journal of Reading, 10(5), 291–299. http://www.jstor.org/stable/40009351
10. Bråten, I., Haverkamp, Y., & Anmarkrud, Ø. (2024). Gaining a deeper understanding of the deep cloze reading comprehension test: Examining potential contributors and consequences. Reading and Writing, 38, 425–446. https://doi.org/10.1007/s11145-024-10521-y
11. Büyüköztürk, Ş., Kılıç Çakmak, E., Akgün, Ö. E., Karadeniz, Ş., & Demirel, F. (2013). Bilimsel araştırma yöntemleri [Scientific research methods]. Pegem Akademi.
12. Cooper, B., & Foy, J. M. (1967). Guessing in multiple choice tests. Medical Education, 1(3), 212–215. https://doi.org/10.1111/j.1365-2923.1967.tb01699.x
13. Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Holt, Rinehart and Winston.
14. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.
15. Çetinkaya, G. (2010). Türkçe metinlerin okunabilirlik düzeylerinin tanımlanması ve sınıflandırılması [Defining and classifying the readability levels of Turkish texts] (Thesis No. 265580) [Doctoral dissertation, Ankara University]. Council of Higher Education Thesis Center. https://tez.yok.gov.tr/UlusalTezMerkezi
16. Çokluk, Ö., Şekercioğlu, G., & Büyüköztürk, Ş. (2012). Sosyal bilimler için çok değişkenli istatistik: SPSS ve LISREL uygulamaları [Multivariate statistics for social sciences: SPSS and LISREL applications] (Vol. 2). Pegem Akademi.
17. Daneman, M., & Hannon, B. (2001). Using working memory theory to investigate the construct validity of multiple choice reading comprehension tests such as the SAT. Journal of Experimental Psychology: General, 130(2), 208–223. https://doi.org/10.1037/0096-3445.130.2.208
18. Embretson, S. E., & Reise, S. P. (2000). Item response theory. Psychology Press. Fotos, S. (1991). The cloze test as an integrative measure of EFL proficiency: A substitute for essays on college entrance examinations? Language Learning, 41(3), 313–336. https://doi.org/10.1111/j.1467-1770.1991.tb00609.x
19. Fuhrman, M. (1996). Developing good multiple choice tests and test questions. Journal of Geoscience Education, 44(4), 379–384. https://doi.org/10.5408/1089-9995-44.4.379
20. Furnham, A., & Chamorro-Premuzic, T. (2004). Personality and intelligence as predictors of statistics examination grades. Personality and Individual Differences, 37, 943–955. https://doi.org/10.1016/j.paid.2003.10.016
21. Gaillard, S., & Tremblay, A. (2016). Linguistic proficiency assessment in second language acquisition research: The elicited imitation task. Language Learning, 66(2), 419–447. https://doi.org/10.1111/lang.12157
22. Gellert, A. S., & Elbro, C. (2012). Cloze tests may be quick, but are they dirty? Development and preliminary validation of a cloze test of reading comprehension. Journal of Psychoeducational Assessment, 31(1), 16–28. https://doi.org/10.1177/0734282912451971
23. Gençer, S. L. (2014). İyi uykular tatlı rüyalar [Good night, sweet dreams]. Bilim Çocuk E-Dergi, 193, 26–29. https://bilimcocuk.tubitak.gov.tr/wp-content/uploads/sites/157/2025/09/28043779-967e-4985-aebd-0e3c2e6fb1be.pdf
24. Gierl, M. J., Bulut, O., Guo, Q., & Zhang, X. (2017). Developing, analyzing, and using distractors for multiple choice tests in education: A comprehensive review. Review of Educational Research, 87(6), 1082–1116. https://doi.org/10.3102/0034654317726529
25. Gooskens, C., & van Heuven, V. J. (2017). Measuring cross-linguistic intelligibility in the Germanic, Romance and Slavic language groups. Speech Communication, 89, 25–36. https://doi.org/10.1016/j.specom.2017.02.008
26. Henk, W. A. (1981). Effects of modified deletion strategies and scoring procedures on cloze test performance. Journal of Reading Behavior, 13(4), 347–357. https://doi.org/10.1080/10862968109547423
27. Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179–185. https://doi.org/10.1007/BF02289447
28. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
29. Jensen, J. L., McDaniel, M. A., Woodard, S. M., & Kummer, T. A. (2014). Teaching to the test…or testing to teach: Exams requiring higher-order thinking skills encourage greater conceptual understanding. Educational Psychology Review, 26(2), 307–329. https://doi.org/10.1007/s10648-013-9248-9
30. Keenan, J. M., & Betjemann, R. S. (2006). Comprehending the Gray Oral Reading Test without reading it: Why comprehension tests should not include passage-independent items. Scientific Studies of Reading, 10(4), 363–380. https://doi.org/10.1207/s1532799xssr1004_2
31. Kendeou, P., McMaster, K. L., & Christ, T. J. (2016). Reading comprehension: Core components and processes. Policy Insights from the Behavioral and Brain Sciences, 3(1), 62–69. https://doi.org/10.1177/2372732215624707
32. Kızılaslan Tunçer, B., & Erden, G. (2015). Boşluk doldurma testlerinin ilkokul 4. sınıf öğrencilerinin okuduğunu anlama düzeylerini belirlemede kullanılabilirliği [Usability of cloze tests in determining 4th-grade primary students’ reading comprehension levels, Special issue on XIV. International Participation Symposium of Primary School Teacher Education (21–23 May 2015)]. Bartın University Journal of Faculty of Education, 4(Special Issue), 318–324. https://doi.org/10.14686/BUEFAD.2015USOSOzelsayi13219
33. Kleijn, S., Pander Maat, H., & Sanders, T. (2019). Cloze testing for comprehension assessment: The HyTeC-cloze. Language Testing, 36(4), 553–572. https://doi.org/10.1177/0265532219840382
34. Kozak, S., & Recchia, H. (2019). Reading and the development of social understanding: Implications for the literacy classroom. The Reading Teacher, 72(5), 569–577. https://doi.org/10.1002/trtr.1760
35. Kubiszyn, T., & Borich, G. (2013). Educational testing and measurement: Classroom application and practice. John Wiley & Sons.
36. Kuder, G. F., & Richardson, M. W. (1937). The theory of the estimation of test reliability. Psychometrika, 2(3), 151–160.
37. Little, J. L., & Bjork, E. L. (2015). Optimizing multiple choice tests as tools for learning. Memory & Cognition, 43(1), 14–26. https://doi.org/10.3758/s13421-014-0452-8
38. Liu, X., Zhang, L., Yu, S., Bai, Z., Qi, T., Mao, H., Zhen, Z., Dong, Q., & Liu, L. (2024). The effects of age and reading experience on the lifespan neurodevelopment for reading comprehension. Journal of Cognitive Neuroscience, 36(2), 239–260. https://doi.org/10.1162/jocn_a_02086
39. Lonigan, C. J., & Burgess, S. R. (2017). Dimensionality of reading skills with elementary-school-age children. Scientific Studies of Reading, 21(3), 239–253. https://doi.org/10.1080/10888438.2017.1285918
40. Luchkina, T., Ionin, T., Lysenko, N., Stoops, A., & Suvorkina, N. (2021). Evaluating the Russian language proficiency of bilingual and second language learners of Russian. Languages, 6(2), 83. https://doi.org/10.3390/languages6020083
41. McDonald, R. P. (2013). Test theory: A unified treatment. Psychology Press.
42. Monrad, S. U., Bibler Zaidi, N. L., Grob, K. L., Kurtz, J. B., Tai, A. W., Hortsch, M., Gruppen, L. D., & Santen, S. A. (2021). What faculty write versus what students see? Perspectives on multiple choice questions using Bloom’s taxonomy. Medical Teacher, 43(5), 575–582. https://doi.org/10.1080/0142159X.2021.1879376
43. Muijselaar, M. M. L., Swart, N. M., Steenbeek-Planting, E. G., Droop, M., Verhoeven, L., & de Jong, P. F. (2017). The dimensions of reading comprehension in Dutch children: Is differentiation by text and question type necessary? Journal of Educational Psychology, 109(1), 70–83. https://doi.org/10.1037/edu0000120
44. Ono, K., Sumita, K., & Seijii, M. (1994). Abstract generation based on rhetorical structure extraction. In Proceedings of the 15th Conference on Computational Linguistics (Vol. 1, pp. 344–348). Association for Computational Linguistics.
45. Ozuru, Y., Best, R., Bell, C., Witherspoon, A., & McNamara, D. S. (2007). Influence of question format and text availability on the assessment of expository text comprehension. Cognition and Instruction, 25(4), 399–438. https://doi.org/10.1080/07370000701632371
46. Pino, M. C., & Mazza, M. (2016). The use of ‘literary fiction’ to promote mentalizing ability. PLOS ONE, 11(8), e0160254. https://doi.org/10.1371/journal.pone.0160254
47. Pretorius, E. J. (2002). Reading ability and academic performance in South Africa: Are we fiddling while Rome is burning? Language Matters, 33(1), 169–196. https://doi.org/10.1080/10228190208566183
48. Rankin, E. F., & Culhane, J. W. (1969). Comparable cloze and multiple choice comprehension test scores. Journal of Reading, 13(3), 193–198. http://www.jstor.org/stable/40017267
49. Rupp, A. A., Ferne, T., & Choi, H. (2006). How assessing reading comprehension with multiple choice questions shapes the construct: A cognitive processing perspective. Language Testing, 23(4), 441–474. https://doi.org/10.1191/0265532206lt337oa
50. Tighe, E. L., & Schatschneider, C. (2016). Examining the relationships of component reading skills to reading comprehension in frustrated adult readers. Journal of Learning Disabilities, 49, 395–409. https://doi.org/10.1177/0022219414555415
51. Ülper, H. (2010). Okuma ve anlamlandırma becerilerinin kazandırılması [Teaching reading and meaning-making skills]. Nobel Yayın Dağıtım.
52. Ulusoy, M. (2009). Boşluk tamamlama testinin okuma düzeyini ve okunabilirliği ölçmede kullanılması [Using cloze test to measure students’ readıng levels and readabılıty of texts]. Türk Eğitim Bilimleri Dergisi, 7(1), 105-126.
53. Vayre, E., & Vonthron, A. (2019). Relational and psychological factors affecting exam participation and student achievement in online college courses. Internet and Higher Education, 43, 100671. https://doi.org/10.1016/j.iheduc.2018.07.001
54. Verenna, A. M. A., Noble, K. A., Pearson, H. E., & Miller, S. M. (2018). Role of comprehension on performance at higher levels of Bloom’s taxonomy: Findings from assessments of healthcare professional students. Anatomical Sciences Education, 11(5), 433–444. https://doi.org/10.1002/ase.1768
55. Vlachos, F., & Papadimitriou, A. (2015). Effect of age and gender on children’s reading performance: The possible neural underpinnings. Cogent Psychology, 2(1), 1045224. https://doi.org/10.1080/23311908.2015.1045224
56. Wait, S. S. (1987). Textbook readability and the predictive value of the Dale–Chall, comprehensive assessment program and cloze [Unpublished doctoral dissertation]. The Florida State University, USA.
57. Wissman, K. T., Zamary, A., & Rawson, K. A. (2018). When does practice testing promote transfer on deductive reasoning tasks? Journal of Applied Research in Memory and Cognition, 7(3), 398–411. https://doi.org/10.1016/j.jarmac.2018.03.002
58. Yu, L., Yu, J. J., & Tong, X. (2023). Social–emotional skills correlate with reading ability among typically developing readers: A meta-analysis. Education Sciences, 13(2), 220. https://doi.org/10.3390/educsci13020220