Face Validity Study of a Small-Scale Test in a Tertiary-Level Intensive EFL Program

Year 2017, , 395 - 410, 30.06.2017

Emrah Cinkara , Özlem Özen Tosun

https://doi.org/10.14686/buefad.281870

Cited By: 2

Abstract

The assessment of language abilities play a
crucial role in designing tests. In order to create a positive and motivating
learning environment for test takers, one of the most important factors is the
administration of tests with high levels of face validity as well as test taker
cognizance of the abilities necessary for exam success. Therefore, this study
examined the overlap in test takers’ and language instructors’ perceptions of
the abilities being tested as well as test developers’ intentions in preparing
exam items. The study was conducted at Gaziantep University Higher School of
Foreign Languages with 38 test takers, seven language instructors, and four
test designers. Data was collected by means of a response sheet on test takers’
perceptions of tests after being modified in accordance with the test to be
analysed. The results revealed a moderate level of agreement between test
designers and language instructors while a lower agreement percentage was
obtained with test takers.

Keywords

Face validity, small-scale tests, expert opinion, test taker perception

Üniversite İngilizce Hazırlık Sınıfında Küçük Ölçekli Bir Sınav için Görünüş Geçerliği Çalışması

Abstract

Dil becerilerinin ölçülmesi ve değerlendirilmesi
eğitim öğretim çalışmaları kapsamında önemli bir alandır ve eğitim öğretim
süreçlerini de şekillendiren önemli sonuçlara sahiptir. Sınava girecek
öğrenciler için olumlu ve motive edici bir öğrenme ve sınav ortamı oluşturmak
hem sınav görünüş geçerliği kavramının temeli hem de sınavı hazırlayanların
amaçlarından bir tanesidir. Bu amaçla, bu çalışmada sınava giren öğrencilerin,
sınavı hazırlayan ve derse giren öğretim elemanlarının sınavdaki maddelerin
hazırlanma amaçları ile ilgili görüşleri araştırılmıştır. Bu çalışmada
Gaziantep Üniversitesi Yabancı Diller Yüksekokulu İngilizce hazırlık sınıfında
öğrenim gören 38 öğrenci, aynı birimden yedi öğretim elemanı ve dört sınav
hazırlama komisyonu üyesi yer almıştır. Cevap kodlama formu ve bire bir görüşme
yöntemleri kullanılarak nicel ve nitel veri toplanmıştır. Araştırma sonuçlarına
göre sınav hazırlayanlar ile derse giren öğretim elemanlarının sınav sorularının
amaçlarını eşleştirmede orta düzeyde bir uyuma sahip olduğu, öğrencilerde ise
düşük seviyeli bir uyum olduğu belirlenmiştir.

Keywords

Kapsam geçerlik, küçük ölçekli sınavlar, uzman görüşü, sınava giren öğrenci algısı

References

• Barnett, L. M., Ridgers, N. D., Zask, A., & Salmon, J. (2015). Face validity and reliability of a pictorial instrument for assessing fundamental movement skill perceived competence in young children. Journal of Science and Medicine in Sport, 18, 98–102. https://doi.org/10.1016/j.jsams.2013.12.004
• Brown, A. (1993). The role of test-taker feedback in the test development process: test- takers’ reactions to a tape-mediated test of proficiency in spoken Japanese. Language Testing, 10, 277-303.
• Brown, HD, & Abeywickrama, P. (2010). Language assessment: Principles and classroom practices (2nd ed.). White Plains, NY: Pearson Education.
• Brown, J. D. & Hudson, T. (2002). Criterion-referenced language testing. Cambridge: Cambridge University Press.
• Chan, D., & Schmitt, N. (1997). Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82(1), 143–159. https://doi.org/10.1037/0021-9010.82.1.143
• Davies, A, Brown, A, Elder, C, Hill, K, Lumley, T, & McNamara, T. (1999). Dictionary of language testing. Cambridge: Cambridge University Press.
• Elder, C. (2007). OET-IELTS benchmarking study report. Australia: Language Testing Research Centre, University of Melbourne.
• Grand, J. A., Ryan, A. M., Schmitt, N., Hmurovic, J. (2010). How Far Does Stereotype Threat Reach? The Potential Detriment of Face Validity in Cognitive Ability Testing. 24(1), 1-28. http://doi.org/10.1080/08959285.2010.518184
• He, L., & Shi, L. (2008). ESL test takers’ perceptions and experiences of standardized English writing tests. Assessing Writing, 13(2), 130-149.
• Hill, K. (1998). The effect of test-taker characteristics on reactions to and performance on an oral English proficiency test. In A.J. Kunnan (Ed.) Validation in language assessment (pp. 299-323). Mahwah, NJ: Lawrence Erlbaum.
• Holden, R. R. (2010). Face Validity. The Corsini Encyclopedia of Psychology: John Wiley & Sons, Inc.
• Hughes, A. (2003). Testing for language teachers. Cambridge: Cambridge University Press.
• Kafer, K. L., & Hunter, M. (1997). On testing the face validity of planning/problem-solving tasks in a normal population. Journal of the International Neuropsychological Society, 3(2), 108–119.
• Kim, H., & Elder, C. (2015). Interrogating the construct of aviation English : Feedback from test takers in Korea. Language Testing, 32(2), 129–149. http://doi.org/10.1177/0265532214544394
• Kelecioğlu, H., & Şahin, S. G. (2014). Geçmişten Günümüze Geçerlik. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 5(2), 1–11.
• Knoch, U. (2014). Using subject specialist to validate an ESP rating scale: The case of the International Civil Aviation Organization (ICAO) rating scale. English for Specific Purposes, 33, 77–86.
• MacLellan, E. (2001). Assessment for learning: The differing perceptions of tutors and test takers. Assessment & Evaluation in Higher Education, 26(4), 307–318.
• Nevo, B. (1985). Face Validity Revisited. Journal of Educational Measurement, 22(4), 287–293. https://doi.org/10.1111/j.1745-3984.1985.tb01065.x
• Sato, T., & Ikeda, N. (2015). Test-taker perception of what test items measure: a potential impact of face validity on student learning. Language Testing in Asia, 5(10). http://doi.org/10.1186/s40468-015-0019-z
• Scouller, K. M., & Prosser, M. (1994). Students' experiences in studying for multiple choice question examinations. Studies in Higher Education, 19(3), 267-279.
• Schmitt, N. (2002). Do reactions to tests produce changes in the construct measured? Multivariate Behavioral Research, 37(1), 105–126. http://doi.org/10.1207/S15327906MBR3701
• Shohamy, E. (2001). Democratic assessment as an alternative. Language Testing, 18(4), 373-391.
• Paker, T. (2013). The backwash effect of the test items in the achievement exams in preparatory classes. Procedia-Social and Behavioral Sciences, 70, 1463–1471.http://doi.org/10.1016/j.sbspro.2013.01.212
• Teemant, A. (2010). ESL student perspectives on university classroom testing practices. Journal of the Scholarship of Teaching and Learning, 10(3), 89-105.
• Tsaia, Y., & Tsou, C.-H. (2009). A standardised English language proficiency test as the graduation benchmark: Student perspectives in higher education. Assessment in Education: Principles,Policy & Practice, 16(3), 319-330.
• Watkins, D., Dahlin, B. O., & Ekholm, M. (2005). Awareness of the backwash effect of assessment : A phenomenographic study of the views of Hong Kong and Swedish lecturers. Instructional Science, (33), 283–309. http://doi.org/10.1007/s11251-005-3002-8
• Xie, Q. (2011). Is test taker perception of assessment related to construct validity? International Journal of Testing, 11, 324–348. http://doi.org/10.1080/15305058.2011.589018
• Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.