TEOG (TEPSE) English Test: Content Validity and Teachers’ Views

Abstract

One of the most important dynamics of the educational setting all over the world is examinations, and some of those are English tests. In Turkey, English tests for the students preparing for the high schools were included in the national exams with the Level Determination Exam (SBS) in 2008 for the first time, and Transition Examination from Primary to Secondary Education (TEPSE) replaced them in 2013. The effects of these exams have been investigated; however, to the best knowledge of the author, other principles of language assessment such as validity and reliability have not been investigated based on the analysis of documents. This study, therefore, aimed to investigate content validity of the English tests in TEPSE, which were conducted between 2016 and 2017. Data collection procedure included the analyses of TEPSE English tests, the coursebook, the curriculum of 8th grade and semi-structured interviews held with 21 English language teachers. The analyses of the frequently used items in the coursebook and table of specifications were used to examine the content validity. The results revealed that despite some factors, TEPSE English tests between 2016 and 2017 have content validity based on the alignment between the coursebook and TEPSE English tests.

Keywords

• Content Validity,English Test,TEPSE

TEOG İngilizce Sınavı: İçerik Geçerliliği ve Öğretmen Görüşleri

Abstract

Tüm dünyada eğitimin en önemli dinamiklerinden bir tanesi sınavlardır ve bunların bazıları İngilizce sınavlarını da içermektedir. Türkiye’de, liseye hazırlanan öğrenciler için uygulanan İngilizce sınavlarına ilk kez 2008 yılında SBS’de yer verilmiş ve 2013 yılında bu sınav TEOG ile yer değiştirmiştir. Bu sınavların etkileri incelenmiştir fakat -bilindiği kadarıyla- geçerlilik ve güvenirlik gibi dil değerlendirmesinin diğer ilkeleri araştırılmamıştır. Bu yüzden, bu çalışma 2016-2017 döneminde uygulanan TEOG İngilizce testlerinin içerik geçerliliğine ilişkin öğretmen görüşlerinin yanı sıra bu dönemlerde yapılan testlerin içerik geçerliliğini araştırmayı hedeflemiştir. Veri toplama yöntemi, 2016-2017 döneminde yapılan TEOG İngilizce sınavlarının analizini, ders kitabının analizini, 2016-2017 8. Sınıf müfredatını ve 21 İngilizce öğretmeniyle yapılmış olan yarı yapılandırılmış görüşmeyi içermektedir. Ders kitabında sık kullanılan yapıların analizi ve belirtke tablosu içerik geçerliliğini değerlendirmek için kullanılmıştır. Sonuçlar, bazı uyumsuzluklar, eşit olmayan dağılımlar ve dört dil becerisinin ihmaline rağmen 2016-2017 döneminde yapılan TEOG İngilizce sınavlarının ders kitabı ile uyumu açısından içerik geçerliliğinin olduğunu ortaya koymuştur.

Keywords

• İçerik Geçerliliği,İngilizce Test,TEOG

References

1. Abella, R., Urrutia, J., & Shneyderman, A. (2003, February). An examination of the validity of English language assessment test scores in a LEP student population. Paper presented at the annual meeting of the National Association for Bilingual Education, New Orleans, LA. Retrieved from http://oer.dadeschools.net/nabe03paper.pdf
2. Akın, G. (2016). Evaluation of national foreign language test in Turkey. Asian Journal of Educational Research, 4(3), 11-21. Retrieved from http://www.multidisciplinaryjournals.com/wp-content/uploads/2016/04/FULL-PAPER-EVALUATION-OF-NATIONAL-FOREIGN-LANGUAGE-TEST-IN-TURKEY.pdf
3. Aksan, M. M. (2001). Instructors’ perceptions of the content validity of the English language exams at Niğde unıversity (Unpublished Master‘s Thesis). Bilkent University, Ankara, Turkey. Retrieved from http://www.thesis.bilkent.edu.tr/0001838.pdf.
4. Al-Adawi, S. S. A., & Al-Balushi, A. A. K. (2016). Investigating content and face validity of English language placement test designed by Colleges of Applied Sciences. English Language Teaching, 9(1), 107-121. doi:10.5539/elt.v9n1p107
5. Alderson, C. J., & Kremmel, B. (2013). Re-examining the content validation of a grammar test: The (im)possibility of distinguishing vocabulary and structural knowledge. Language Testing, 30(4), 535-556. doi:10.1177/0265532213489568.
6. Antalya Provincial Directorate of National Education (2016). 2016-2017 eğitim-öğretim yılı örnek yıllık plan [2016-2017 educational year sample annual plan]. Retrieved from http://ilbap.meb.k12.tr/icerikler/2016-2017-egitim-ogretim-yili-ornek-yillik-planlari_2705897.html
7. Biemiller, A. (2003). Vocabulary: Needed if more children are to read well. Reading Psychology, 24(3-4), 323-335. doi: 10.1080/02702710390227297
8. Brown, H. D. (2004). Language assessment: Principles and classroom practices. Pearson Education.

9. Brown, H. D., & Abeywickrama, P. (2010). Language assessment: Principles and classroom practices (2nd ed.). Pearson Education.
10. Chakwera, E. W. J. (2004). Content validity of independently constructed curriculum-based examinations (Unpublished Doctoral Dissertation). Graduate School of the University of Massachusetts, Amherst, U.S.A.
11. Cheng, L., & Fox, J. (2017). Assessment in language classroom: Teachers supporting student learning. Palgrave.
12. Christensen, L. B., Johnson, B. R.,& Turner, A. L. (2015). Research methods, design, and analysis (12th ed.). Pearson.
13. Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). SAGE Publications.
14. Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4th ed.). Pearson Education.
15. Davidson, F., & Lynch, B. K. (2002). Testcraft- A teacher’s guide to writing and using language test specifications. Yale University Press.
16. Ekbatani, G. (2011). Measurement and evaluation in post-secondary ESL. Routledge.
17. Fathony, F. (2017). An analysis on content validity of English summative test items of second grade students at Mtsn Malijambe in the academic year 2015/2016 (Unpublished Master’s Thesis). The State Islamic Instıtue, Surakarta, Indonesia.
18. Fraenkel, R. J., Wallen, E. N., & Hyun, H. H. (2012). How to design and evaluate research in education (8th ed.). McGraw-Hill.
19. Griffee, T. D. (2012). An introduction to second language research methods: Design and data. California, CA: TESL-EJ Publications.
20. Gorsuch, G., & Griffee, D. T. (2018). Second language testing for student evaluation and classroom research. Information Age Publishing Inc.
21. Gömleksiz, M. N., &Aslan, S. (2017). Secondary school students’ views about the effectiveness of TEOG exam on English language learning: A qualitative study. International Journal of Social Science, 11(64), 1-14. doi: 10.9761/JASSS7375.
22. Haiyan, W., & Fuqin, Z. (2005). A validation of CET for testing candidates’ communicative competence and a proposal of a university-based communicative test. CELEA Journal, 28(2), 26-31. Retrieved from http://www.celea.org.cn/teic/60/60-26.pdf
23. Hughes, A. (2003). Testing for language teachers (2nd ed.). Cambridge University Press
24. Ing, L. M., Musah, M. B., Al-Hudawi, V. H. S., Tahir, L. M., & Kamil, M. N. (2015). Validity of teacher made assessment: A table of specification approach. Asian Social Science, 11(5), 193-200. doi:10.5539/ass.v11n5p193.
25. Jaturapitakkul, N. (2013). Students' perceptions of traditional English language testing in Thailand. Academic Journal of Interdisciplinary Studies. 2(3), 445-452. doi:10.5901/ajis.2013.v2n3p445.
26. Kang, M., & Chang, H. (2014). Ensuring validity of practical English certification of local office of education in Korea. Pan-Pacific Association of Applied Linguistics, 18(2), 111-122. Retrieved from http://paal.kr/download/18_2/18-2-7_Mun-koo_Kang_and_Hyung-ji_Chang.pdf
27. Kılıçkaya, F. (2016). Washback effects of a high-stakes exam on lower secondary school English teachers’ practices in the classroom. Lublin Studies in Modern Languages and Literature, 40(1), 116-134. doi:10.17951/lsmll.2016.40.1.116.
28. Külekçi, E. (2016). A concise analysis of the Foreign Language Examination (YDS) in Turkey and its possible washback effects. International Online Journal of Education and Teaching (IOJET), 3(4). 303-315. Retrieved from: http://iojet.org/index.php/IOJET/article/view/141/143
29. Lochmiller, C. R., & Lester, J. N. (2017). An introduction to educational research: Connecting methods to practice. Sage Publications.
30. Mackey, A., & Gas, M. S. (Eds.). (2012). Research methods in second language acquisition: A practical guide. Blackwell Publishing.
31. Mart, M. T. (2014). Determination of tutors’ views on the common exams applied in secondary schools within the student passing process from primary education to high school education (TEOG) (Unpublished Master’s Thesis). Zirve University, Gaziantep, Turkey. Available from Council of Higher Education Thesis Centre.
32. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis. (2nd ed.). Sage.
33. MoNE. (2011). Sınavla öğrenci alacak ortaöğretim kurumlarına ilişkin merkezî sınav başvuru ve uygulama kılavuzu [Guidelines for applicants regarding to secondary education institutions which admits the students based on the central exams]. Retrieved from http://mevzuat.meb.gov.tr/html/608_0.html
34. MoNE. (2016). 2016-2017 eğitim ve öğretim yılı merkezi sistem ortak sınavları [TEPSE in 2016-2017 educational year]. Retrieved from: http://ttkb.meb.gov.tr/www/2016-2017-egitim-ve-ogretim-yili-merkezi-sistem-ortak-sinavlari/icerik/281.
35. MoNE. (2016b). 8. Sınıf 1. dönem İngilizce dersi merkezi ortak sınavı [English exam questions for grade 8 in the first semester]. Retrieved from https://odsgm.meb.gov.tr/meb_iys_dosyalar/2016_11/24014749_ingilizce_a.pdf.
36. MoNE. (2017). 8. Sınıf 2. dönem İngilizce dersi merkezi ortak sınavı [English exam questions for grade 8 in the second semester]. Retrieved from http://odsgm.meb.gov.tr/meb_iys_dosyalar/2017_04/27131529_YNGYLYZCE_A.pdf
37. MoNE (2018). Sınavla öğrenci alacak ortaöğretim kurumlarına ilişkin merkezî sınav başvuru ve uygulama kılavuzu [Guidelines for applicants regarding to secondary education institutions which admits the students based on the central exams]. Retrieved from: https://www.meb.gov.tr/sinavlar/dokumanlar/2018/MERKEZI_SINAV_BASVURU_VE_UYGULAMA_KILAVUZU.pdf
38. Newman, I., Frye, B., Blumenfeld, G., & Newman, C. (1973). An introduction to the basic concepts of measurement and evaluation. University of Akron.
39. Newman, I., Lim, J., & Pineda, F. (2013). Content validity using a mixed method approach: Its application and development through the use of a table of specifications methodology. Journal of Mixed Method Research, 7(3), 243-260. doi: 10.1177/1558689813476922.
40. Nicholson, S. J. (2015). Evaluating the TOEIC® in South Korea: Practicality, reliability and validity. International Journal of Education, 7(1), 221-233. doi: 10.5296/ije.v7i1.7148.
41. Ökmen, B., & Kılıç, A. (2016). The effect of language teaching methods on academic success in Turkey. Journal of Education and Training Studies, 4(10), 193-199. doi: 10.11114/jets.v4i10.1767.
42. Pterneas, V. (2009).Word frequency counter [Software]. Available from https://www.softpedia.com/get/Office-tools/Other-Office-Tools/Word-Frequency-Counter.shtml
43. Razmjoo, S, & Tabrizi, H. (2010). A content analysis of the TEFL M.A entrance examinations (case study: Major courses). Journal of Pan-Pacific Association of Applied Linguistics, 14(1), 159-170. Retrieved from http://files.eric.ed.gov/fulltext/EJ920511.pdf
44. Siddiek, G. A. (2010). The impact of test content validity on language teaching and learning. Asian Social Science, 6(12), 133-143. Retrieved from http://www.ccsenet.org/journal/index.php/ass/article/view/6817/6305
45. Sims, M. J. (2015). A valid and reliable English proficiency exam: A model from a university language program in Taiwan. English as a Global Language Education, 1(2), 91-125. doi: 10.6294/EaGLE.2015.0102.04.
46. Solano-Flores, G. (2016). Assessing English language learners: Theory and practice. Routledge.
47. TUIK. (2016). 2015 well-being index for provinces. Retrieved from: http://www.tuik.gov.tr/PreHaberBultenleri.do?id=24561.
48. Vural. A. (2017). The assessment of consistency of 8th grade English course curriculum to the questions in TEOG English exam according to teachers’ opinions (Unpublished Master’s Thesis). Gaziantep University, Gaziantep, Turkey. Retrieved from Council of Higher Education Thesis Centre.
49. Weiping, G., & Juan, L. (2005). Test analysis of college students’ communicative competence in English. Asian EFL Journal, 7(2). Retrieved from http://www.asian-efl-journal.com/June_05_wg.pdf
50. Wolf, M. K., Farnsworth, T., & Herman, J. (2008). Validity issues in assessing English language learners’ language proficiency. Educational Assessment, 13(2-3), 80-107. doi:10.1080/10627190802394222.