The Development of a Scale to Evaluate Foreign Language Skills at Preparatory Schools

Öz

The aim of the present study is to develop a valid and reliable scale evaluating the effectiveness of language preparatory programs in the acquisition of language skills. In the development of Foreign Language Skills Scale (FLSS) in this study, research sample consisted of 326 preparatory school students for the exploratory factor analysis (EFA) and 350 preparatory school students for the confirmatory factor analysis (CFA). Based on the data obtained from the first sample, an EFA was carried out on the FLSS. EFA has identified that 27 items of the scale have factor loads between 0.519 and 0.729, while they explain 65.376% of the total variance and are distributed under five factors. These factors are named as writing skill, speaking skill, listening skill, core skills, and reading skill. A CFA was applied on the data obtained from the second sample that consisted of 350 students. As a result of the CFA, it was confirmed that the FLSS consisted of 27 items and five factors. For all the items in the scale, item-subscale, item-test correlation coefficients and mean differences between the upper and the lower 27% of the participants were calculated, and it is determined that each item is consistent with not only the subscale it is under but also the whole test. In addition, the Cronbach’s Alpha reliability coefficients of the total scale’s and five sub-scales’ internal consistency is quite high. The FLSS is expected to offer a comprehensive evaluation of the acquisition of four language skills in foreign language teaching programs.

Anahtar Kelimeler

• Scale development
• English language teaching
• Teaching language skills

References

1. Akpur, U., Alcı, B., & Karataş, H. (2016). Evaluation of the curriculum of English preparatory classes at Yildiz Technical University using CIPP model. Educational Research and Reviews, 11(7), 466-473.
2. Balcı, A. (2009). Sosyal bilimlerde araştırma: yöntem, teknik ve ilkeler. Ankara: PegemA Yayıncılık.
3. Balcı, Ö., Durak Üğüten, S., & Çolak, F. (2018). Zorunlu İngilizce hazirlik programinin değerlendirilmesi: Necmettin Erbakan Üniversitesi yabancı diller yüksekokulu örneği [The evaluation of compulsory preparatory program: The case of Necmettin Erbakan University school of foreign languages]. Kuramsal Eğitim Bilim Dergisi, 11(4), 860-893.
4. Baykul, Y. (2000). Eğitimde ve psikolojide ölçme: klasik test teorisi ve uygulaması. Ankara: ÖSYM Yayınları.
5. Bayram, N. (2013). Yapısal eşitlik modellemesine giriş (2. baskı). Bursa: Ezgi Kitabevi.
6. Black, P, Harrison, C, Lee, C., Marshall, B., & Wiliam, D. (2004) Working inside the Black Box: assessment for learning in the classroom. Phi Delta Kappan, 86(1), 8-21.
7. Büyüköztürk, Ş. (2002). Sosyal bilimler için veri analizi el kitabı. Ankara: PegemA Yayıncılık.
8. Comrey, A. L., & Lee, H. B. (1992). A first course in factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.

9. Coşkun, A. (2013). An investigation of the effectiveness of the modular general English language teaching preparatory program at a Turkish university. South African Journal of Education, 33(3). 1-18.
10. Council of Europe (CoE). (2001). Common European framework of reference for languages: Learning, teaching, assessment. Strasbourg, France: Council of Europe. Cambridge University Press. Retrieved from https://www.coe.int/en/web/common european framework reference languages
11. Ekşi, G. Y. (2017). Designing curriculum for second and foreign language studies. In A. Sarıçoban (Ed.), ELT methodology. Anı Yayıncılık: Ankara.
12. Erarslan, A. (2019). Progressive vs modular system in preparatory school English language teaching program: A case of system change at a state university in Turkey. Dil ve Dilbilimi Çalışmaları Dergisi, 15(1), 83-97.
13. Golob, T. F. (2003). Structural equation modeling for travel behavior research. Transportation Research, 37(1), 1-25.
14. Gorsuch, R. L. (1983). Factor Analysis. Hillsdale: Lawrence Erlbaum Associates.
15. Güllü, A. S. (2007). An evaluation of English program at Kozan Vocational School of Çukurova University: Students’ point of view (Unpublished master’s thesis). Çukurova University, The Graduate School of Social Sciences, English Language Teaching Department, Adana.
16. İşcan, S. (2017) The efficacy of modular EFL syllabus in prep classes. International Journal of Managament and Applied Science, 3(8). 91-94.
17. Karataş, H & Fer, S. (2009). Evaluation of English curriculum at Yıldız Technical University using CIPP model. Education and Science, 34, 47-60.
18. Karcı-Aktaş, C. & Gündoğdu, K. (2020) An extensive evaluation study of the English preparatory curriculum of a foreign language school. Pegem Eğitim ve Öğretim Dergisi, 10(1), 169-214.
19. Keser, A. D., & Köse, G. D. (2019). Determining exit criteria for English language proficiency in preparatory programs at Turkish universities. The Online Journal of Quality in Higher Education, 6(2), 45-49.
20. Kiely, R. & Rea-Dickins, P. (2005). Program evaluation in language education. Pelgrave Macmillan.
21. Kline, P. (1994). An easy guide to factor analysis. New York: Routledge.
22. Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 22(140), 1-55.
23. North, B. (2007). The CEFR illustrative descriptor scales. The Modern Language Journal, 91(4), 656- 659.
24. Öner, G. & Mede, E. (2015). Evaluation of A1 level program at an English preparatory school in a Turkish university: a case study. International Association of Research in Foreign Language Education and Applied Linguistics. ELT Research Journal, 4(3), 204-226.
25. Özkanal, Ü. (2009). The Evaluation of English preparatory program of Eskisehir Osmangazi University Foreign Languages Department and a model proposal (Unpublished doctoral thesis). Anadolu University, Eskişehir.
26. Özkanal, Ü., & Hakan, A. G. (2010). Effectiveness of university English preparatory programs: Eskisehir Osmangazi University foreign languages department English preparatory program. Journal of Language Teaching and Research, 1(3), 295-305.
27. Paker, T. (2013). The backwash effect of the test items in the achievement exams in preparatory classes. Procedia-Social and Behavioral Sciences, 70, 1463-1471.
28. Peacock, M. (2009). The evaluation of foreign-language-teacher education programmes. Language Teaching Research, 13(3), 259-78.
29. Pohlmann, J. T. (2004). Use and interpretation of factor analysis in the journal of educational research: 1992-2002. The Journal of Educational Research, 98(1), 14-23.
30. Sağlam, D., & Akdemir, E. (2018). İngilizce hazırlık öğretim programına ilişkin öğrenci görüşleri [Opinions of students on the curriculum of English preparatory program]. Yükseköğretim ve Bilim dergisi/Journal of Higher Education and Science, 8(2), 401-409.
31. Scherer, R. F., Wiebe, F. A., Luther, D. C. & Adams, J. S. (1988). Dimensionality of coping: factor stability using the ways of coping questionnaire, Psychological Reports, 62(3), 763-770. PubMed PMID: 3406294.
32. Schumacker, R. E., & Lomax, R. G. (2004). A beginner’s guide to structural equation modeling (2nd ed.). NJ: Lawrence Erlbaum Associates, Mahwah.
33. Tekin, M. (2015). Evaluation of a preparatory school program at a public university in Turkey. Uluslararası Sosyal Araştırmalar Dergisi. The Journal of International Social Research, 8(36), 718-733.
34. The Higher Education Council (2016). Yükseköğretim Kurumlarında Yabancı Dil Öğretimi ve Yabancı Dille Öğretim Yapılmasında Uyulacak Esaslara İlişkin Yönetmelik [Regulation of the Higher Education Institutions on foreign language teaching and rules to be obeyed]. Retrieved from https://www.mevzuat.gov.tr/Metin.Aspx?MevzuatKod=7.5.21475&MevzuatIliski=0&sourceXmlSearch=Y%C3%BCksek%C3%B6%C4%9Fretim%20Kurumlar%C4%B1nda%20Yabanc%C4%B1%20Dil%20%C3%96%C4%9Fretimi%20ve%20Yabanc%C4%B1%20Dille%20%C3%96%C4%9Fretim%20Yap%C4%B1lmas%C4%B1nda%20Uyulacak%20Esasl
35. Tunç, F. (2010). Evaluation of an English Language Teaching Program at a Public University Using CIPP model (Unpublished masters’ thesis). Ankara: Middle East Technical University. Retrieved from http://etd.lib.metu.edu.tr/upload/12611570/index.pdf
36. Uysal, D. (2019) Problems and solutions concerning English language preparatory curriculum at higher education in view of ELT instructors. International Journal of Contemporary Educational Research, 6(2), 452-467.
37. Wang, Y., Lin, H., & Luarn, P. (2006). Predicting consumer intention to use mobile service. Information Systems Journal, 16(2), 157-179.
38. Yap, B.W., & Khong, K.W. (2006). Examining the effects of customer service management (CSM) on perceived business performance via structural equation modelling. Applied Stochastic Models in Business and Industry, 22, 587-605.
39. Yılmaz, F. (2009). English language needs analysis of university students at a voluntary program. The Journal of Social Sciences Research, 4(1), 148-166.