YDS AS A BENCHMARK IN TURKEY: THE DIFFICULTY LEVELS OF READING COMPREHENSION QUESTIONS

Year 2019, Issue: 80, 95 - 112, 26.12.2019

Cüneyt Demir

Abstract

The ability to speak a language covers all skills, namely reading, writing, speaking, and listening, together with a reasonable degree of performance and competence knowledge. Specializing over a single skill by overlooking the others may hinder speaker proficiency. In other words, all skills equally weigh; hence, the tests that aim to measure language proficiency should be able to gauge all skills in a good balance. In spite of that reality, YDS (Foreign Language Exam) in Turkey scores test-takers only through reading questions, and accordingly, this test may not be a good benchmark for Turkey because reading questions do not seem to have a stable difficulty level. The exam is composed of different kinds of questions which are vocabulary, grammar, comprehension, dialogue, and reading. The reading questions take up the biggest portion in the exam -20 out of 80 questions-; therefore, they highly determine the test-takers’ scores. This study investigated eleven YDS tests held in the last 5 years. In total eleven YDS tests (220 reading questions) were analysed. Coleman-Liau Index was used to measure how difficult each test is. This study aims to reveal whether reading questions of YDS have a similar difficulty level by calculating the Coleman-Liau Index. This study is of great importance because YDS is a crucial benchmark test in Turkey that determines the English proficiency level of those who take it, and any discrepancy in difficulty levels may negatively affect the reliability of the test. The results showed that the reading questions are in a large range of difficulty, and do not follow a stable track of difficulty.

Keywords

YDS, Benchmark, Difficulty, Reading, Coleman.

References

• Akpinar, K. D., & Cakildere, B. (2013). Washback effects of high-stakes language tests of Turkey (KPDS and ÜDS) on productive and receptive skills of academic personnel. Journal of Language and Linguistic Studies, 9(2), 81-94.
• Aşkaroğlu, V. (2013). 2013 YDS ile 2012 KPDS ve ÜDS Sınavlarının Karşılaştırılması. Karadeniz Uluslararası Dergi, 162-173.
• Carlson, S. E., Seipel, B., & McMaster, K. (2014). Development of a new reading comprehension assessment: Identifying comprehension differences among readers. Learning and Individual Differences, 31, 40-53.
• Deacon, S. H., Kieffer, M. J., & Laroche, A. (2014). The relation between morphological awareness and reading comprehension:Evidence frommediation and longitudinal models. Scientific Studies of Reading, 18, 432–451, DOI:10.1080/10888438.20 14.926907.
• Frosini, G., Lazzerini, B., & Marcelloni, F. (1998). Performing automatic exams. Computers & Education, 281-300.
• Gür, Ö. (2013). Ölçme, değerlendirme ve kamu personeli dil sinavı (KPDS) – Bu sinav neyi ölçüyor? Sakarya University Journal of Education, 23-32.
• Kıray, G. (2015). Macro-structure Analysis of Reading Comprehension Paragraphs of KPDS and YDS Exams within Years 2003-2013. Hasan Ali Yücel Eğitim Fakültesi Dergisi, 219-233.
• Lam, R. (2015). Language assessment training in Hong Kong: Implications for language assessment literacy. Language Testing, 32(2), 169–197.
• Nagy, W. E. (2007). Metalinguistic awareness and the vocabulary-comprehension connection. In R. K. Wagner,A. E. Muse, & K. R. (Eds.), Vocabulary acquisition: Implications for reading comprehension (pp. 52-77). New York: Guilford Press.
• Nation, K. (1997). Assessing reading difficulties: the validity and utility of current measures of reading skill. British Journal of Educational Psychology, 67(3): 359-370 doi.org/10.1111/j.2044-8279.1997.tb01250.x.
• OSYM. (2016). www.osym.gov.tr. Retrieved 2 3, 2019, from YDS Çıkmış Sorular: http:// www.osym.gov.tr/TR,15073/yds-cikmis-sorular.html
• Pérez, E. V., Santos, L. M., Pérez, M. J., de Castro Fernández, J. P., & Martín, R. G. (2012). Automatic Classification of Question Difficulty Automatic Classification of Question Difficulty. Frontiers in Education Conference (pp. 1-5). IEEE.
• Perfetti, C. A. (1999). Comprehending written language: A blueprint of the reader. In C. M. Brown, & P. (. Hagoort, The neurocognition of language (pp. 167-208). Oxford: Oxford University Press.
• Perfetti, C. A., Landi, N., & Oakhill, J. (2005). The acquisition of reading comprehension skills. In C. H. M. J. Snowling, The science of reading: A handbook (pp. 227- 247). London: Blackwell.
• Perfetti, C. A., Landi, N., & Oakhill, J. (2005). The acquisition of reading comprehension skills. In M. J. Snowling, & C. H. (Eds.), The science of reading: A handbook (pp. 227-247). London: Blackwell.
• Piontek, M. E. (2008). Best practicesfor designing and grading exams. Occasional Paper, 24, 1-12.
• Readable Blog. (2017, 5 22). Retrieved 1 2, 2019, from Readability and the Coleman- Liau Index: https://readable.com/blog/the-coleman-liau-index/
• Susanti, Y., Nishikawa, H., Tokunaga, T., & Hiroyuki, O. (2016). Item difficulty analysis of english vocabulary questions. The 8th International Conference on Computer Supported Education, pp. 267-274.
• Tan, Y. T., & Othman, A. R. (2013). The relationship between complexity (taxonomy) and difficulty. The 20th National Symposium on Mathematical Sciences (pp. 596- 603). AIP Publishing LLC.
• Trapman, M., Gelderen, A., Schooten, E., & Hulstijn, J. (2016). Reading Comprehension Level and Development in Native and Language Minority Adolescent Low Achievers: Roles of Linguistic and Metacognitive Knowledge and Fluency. Reading & Writing Quarterly, 239-257, DOI: 10.1080/10573569.2016.118354 1.
• Van Gelderen, A., Schoonen, R., De Glopper, K., Hulstijn, J., Snellings, P., Simis, A., et al. (2004). Linguistic knowledge, processing speed and metacognitive knowledge in first and second language reading comprehension: A componential analysis. Journal of Educational Psychology, 96, 19–30. DOI:10.1037/0022- 0663.96.1.19.
• Van Gelderen, A., Schoonen, R., Stoel, R. D., De Glopper, K., & Hulstijn, J. (2007). Development of adolescent reading comprehension in language 1 and language 2: A longitudinal analysis of constituent components. Journal of Educational Psychology, 99, 477–491. DOI:10.1037/0022-0663.99.3.477.
• Wigfield, A., Gladstone, J. R., & Turci, L. (2016). Beyond cognition: Reading motivation and reading comprehension. Child development perspectives, 10(3), 190-195.
• Zimmaro, D. M. (2016, 11 1). Writing Good Multiple-Choice Exams. Retrieved 2 10, 2019, from https://facultyinnovate.utexas.edu/sites/default/files/writing-good- multiple-choice-exams-fic-120116.pdf