Yıl 2017, Cilt 34 , Sayı 1, Sayfalar 49 - 73 2018-12-17

Striking the Balance between Validity and Reliability of a Listening Test in Turkish as a Second Language

Emel Tozlu [1] , Aylin Ünaldı [2]


Evidence on the efficacy of an assessment tool is necessary in order to justify the decisions we make based on the scores from it. Validity evidence can be collected from several sources such as the stages before and after test administration. In the present research study, validity evidence of several types on a Turkish as a Second Language (TSL) Academic Listening Test is presented in order to establish the efficacy of it. This paper presents cognitive, contextual and scoring validity (reliability) evidence from the first and second versions of the test and investigates whether the modifications made after the first administration have had a positive effect on the quality of the test. The study concludes that although the changes made in the first version of the test strengthened the validity claims in terms of cognitive and contextual requirements, the reliability scores of the test worsened in the second version. This reminded us that although it is necessary to build the foundations of a test firmly by operationalizing the necessary contextual features and cognitive processes, this will not thoroughly guarantee the technical quality of the items. Scoring validity should be established carefully as well. This study exemplifies a thorough attempt in establishing the validity of a TSL test from multiple perspectives and aims to be an exemplary study for further test development in TSL.
Cognitive validity, contextual validity, scoring validity, assessment of listening in Turkish as a second language
  • ALTE, (2011). Manual for language test development and examining. Downloaded from https://www.alte.org/resources/Documents/ManualLanguageTest-Alte2011_EN.pdf
  • Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.
  • Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.
  • Borsboom, D., Mellenbergh, G. J., & Heerden, J. V. (2004). The concept of validity. Psychological Review, 111(4), 1061-1071.
  • Council of Europe, (2001). Common European framework of reference for languages: Learning, teaching and assessment. Cambridge: Cambridge University Press.
  • EALTA guidelines for good practice in language testing and assessment. Downloaded from http://www.ealta.eu.org/documents/archive/guidelines/English.pdf
  • Elliott, M., & Wilson, J. (2013). Context validity. In A. Geranpayeh & L. Taylor (Eds.), Examining listening: Research and practice in assessing second language listening, Studies in language testing, 35 (pp.152-241). Cambridge: Cambridge University Press.
  • Field, J. (2013). Cognitive Validity. In A. Geranpayeh & L. Taylor (Eds.), Examining listening: Research and practice in assessing second language listening, Studies in language testing, 35 (pp.77-151). Cambridge: Cambridge University Press.
  • Geranpayeh, A. (2013). Scoring validity. In A. Geranpayeh & L. Taylor (Eds.), Examining listening: Research and practice in assessing second language listening. Studies in language testing, 35 (pp.242-272). Cambridge: Cambridge University Press.
  • Kane, M. T. (2013) Validating the Interpretations and Uses of Test Scores. Journal of Educational Measurement 50(1), 1–73.
  • Khalifa, H., & Weir, C. J. (2009). Examining reading: research and practice in assessing second language reading, Studies in language testing, 29. Cambridge: UCLES/Cambridge University Press.
  • Knoch, U., & Elder, C. (2013). A framework for validating post-entry language assessments (PELAs). Papers in Language Testing and Assessment, 2(2), 48- 66.
  • McNamara, T. (2000). Language testing. Oxford: Oxford University Press.
  • Messick, S. A. (1989). Validity. In R. L. Linn (Ed) Educational measurement 13-103. New York. American Council on Education. Mac Millian Publishing Company.
  • Richards, J. C. (1983). Listening comprehension: Approach, design, procedure. TESOL quarterly, 17(2), 219-240.
  • Tozlu, E. (2017). The development of a listening test for learners of Turkish as a foreign language (Unpublished master thesis). Boğaziçi University, İstanbul, Turkey.
  • Trim, J. L. M. (2009). Breakthrough. Retrieved from https://www.coe.int/t/dg4/linguistic/Source/FinalBreakthrough%20specification_6Nov01.rtf
  • Van Ek, J., & Trim, J. L. M. (1991a). Threshold 1990. Cambridge: Cambridge University Press.
  • Van Ek, J., & Trim, J. L. M. (1991b). Waystage 1990. Cambridge: Cambridge University Press.
  • Van Ek, J., & Trim, J. L. M. (2001). Vantage. Cambridge: Cambridge University Press.
  • Walt, J. L., & Steyn, F. (2008). The validation of language tests. Stellenbosch Papers in Linguistics, 38, 191-204.
  • Weir, C. J. (1993). Understanding and developing language tests. New York & Toronto: Prentice-Hall.
  • Weir, C. J. (2005). Language testing and validation. Hampshire: Palgrave McMillan.
  • Young, J.W., So, Y., & Ockey, G.J. (2013). Guidelines for best test development practices to ensure validity and fairness for international English language proficiency assessments. Educational Testing Service. https://www.ets.org/s/about/pdf/best_practices_ensure_validity_fairness_english_language_assessments.pdf
Birincil Dil en
Bölüm Makaleler
Yazarlar

Yazar: Emel Tozlu

Yazar: Aylin Ünaldı

Tarihler

Yayımlanma Tarihi : 17 Aralık 2018

Bibtex @araştırma makalesi { buje513585, journal = {Boğaziçi Üniversitesi Eğitim Dergisi}, issn = {1300-9567}, address = {}, publisher = {Boğaziçi Üniversitesi}, year = {2018}, volume = {34}, pages = {49 - 73}, doi = {}, title = {Striking the Balance between Validity and Reliability of a Listening Test in Turkish as a Second Language}, key = {cite}, author = {Tozlu, Emel and Ünaldı, Aylin} }
APA Tozlu, E , Ünaldı, A . (2018). Striking the Balance between Validity and Reliability of a Listening Test in Turkish as a Second Language. Boğaziçi Üniversitesi Eğitim Dergisi , 34 (1) , 49-73 . Retrieved from https://dergipark.org.tr/tr/pub/buje/issue/42588/513585
MLA Tozlu, E , Ünaldı, A . "Striking the Balance between Validity and Reliability of a Listening Test in Turkish as a Second Language". Boğaziçi Üniversitesi Eğitim Dergisi 34 (2018 ): 49-73 <https://dergipark.org.tr/tr/pub/buje/issue/42588/513585>
Chicago Tozlu, E , Ünaldı, A . "Striking the Balance between Validity and Reliability of a Listening Test in Turkish as a Second Language". Boğaziçi Üniversitesi Eğitim Dergisi 34 (2018 ): 49-73
RIS TY - JOUR T1 - Striking the Balance between Validity and Reliability of a Listening Test in Turkish as a Second Language AU - Emel Tozlu , Aylin Ünaldı Y1 - 2018 PY - 2018 N1 - DO - T2 - Boğaziçi Üniversitesi Eğitim Dergisi JF - Journal JO - JOR SP - 49 EP - 73 VL - 34 IS - 1 SN - 1300-9567- M3 - UR - Y2 - 2018 ER -
EndNote %0 Boğaziçi Üniversitesi Eğitim Dergisi Striking the Balance between Validity and Reliability of a Listening Test in Turkish as a Second Language %A Emel Tozlu , Aylin Ünaldı %T Striking the Balance between Validity and Reliability of a Listening Test in Turkish as a Second Language %D 2018 %J Boğaziçi Üniversitesi Eğitim Dergisi %P 1300-9567- %V 34 %N 1 %R %U
ISNAD Tozlu, Emel , Ünaldı, Aylin . "Striking the Balance between Validity and Reliability of a Listening Test in Turkish as a Second Language". Boğaziçi Üniversitesi Eğitim Dergisi 34 / 1 (Aralık 2018): 49-73 .
AMA Tozlu E , Ünaldı A . Striking the Balance between Validity and Reliability of a Listening Test in Turkish as a Second Language. Boğaziçi Üniversitesi Eğitim Dergisi. 2018; 34(1): 49-73.
Vancouver Tozlu E , Ünaldı A . Striking the Balance between Validity and Reliability of a Listening Test in Turkish as a Second Language. Boğaziçi Üniversitesi Eğitim Dergisi. 2018; 34(1): 73-49.