A Study on Rubric Development to Evaluate Writing Skill in Teaching Turkish as a Foreign Language

Abstract

Bu çalışmada, yabancı dil olarak Türkçe öğretiminde yazma becerisini değerlendirmede geçerli ve güvenilir bir ölçek geliştirmek amaçlanmıştır. Kurumlarda yazma sınavlarının puanlanmasında yaşanan sorunlar ve standart bir dereceli puanlama anahtarının alanda bulunmayışı bu çalışmanın çıkış noktası olmuştur. Araştırma Bartın Üniversitesi Dil Eğitim Merkezinde eğitim vermekte olan öğretmenler üzerinden yürütülmüştür. Veri toplamak için hazırlanan uzman değerlendirme formları ilgili alanyazın ve konu ile ilgili benzer ölçekler incelenerek geliştirilmiştir. Ayrıca veri toplama sürecinde uzmanlara yöneltilen görüşme soruları sonucunda elde edilen nitel bulgulardan yararlanılmıştır. 30 maddeden oluşan anket form ile alanında uzman 30 öğretmenden veri toplanmıştır. Elde edilen veriler analize tabi tutulmuştur. Araştırmada ölçeğin kapsam geçerliliğini incelemek için uzmanların görüşleri alındıktan sonra anket formlarının frekans dağılımına bakılarak %49,75’i değerlendirme sorularına katıldığını, %40,25’i tamamen katıldığını ifade ederken %8,46’sı katılmadığını, %1,54’ü hiç katılmadığı şeklinde sonuçlar elde edilmiştir. Ölçeğin güvenirliği, puanlayıcıların verdikleri puanlar arasındaki uyumun tutarlığı incelenerek yapılmıştır. Güvenirlik tespiti için Kendall W uyuşum kat sayısı yöntemi kullanılmıştır. Analiz sonucunda geliştirilen ölçeklerin uyum katsayısı A1 seviyesi için .700, A2 seviyesi için .801, B1-B2 seviyeleri için .828, C1 seviyesi için .807 olarak bulunmuştur. Sonuçlar doğrultusunda tutarlılık katsayısı değerlerinin 1’e yaklaşması geliştirilen DPA’ların güvenirliğine kanıt oluşturmaktadır. Yapılan analizler sonucunda A1, A2-, B1-B2, C1 seviyelerine uygun geliştirilen dereceli puanlama anahtarlarının geçerli ve güvenilir ölçme araçları olduğu yazma sınavlarını değerlendirmede kullanılabileceği söylenebilmektedir.

Keywords

• Writing
• Assessment and Evaluation
• Scale
• Rubric
• Writing, Assessment and Evaluation, Scale, Rubric

References

1. Aktaş, M., & Buyer, D. (2018). Analytical Rubric Development for Evaluating the Written Story: Validity and Reliability Study. Journal of Mersin University Faculty of Education, 14 (2), 597-610.
2. Ari, G. (2010). Evaluation of Storytelling Texts Written by Sixth and Seventh Grade Students. Turkology Studies, 43-75.
3. Arslan, M., & Klicic, E. (2015). Problems Encountered in the Development of Writing Skills in Teaching Turkish as a Foreign Language: The Case of Bosnia and Herzegovina. Kırıkkale University Journal of Social Sciences, 5 (2), 169-182.
4. Atılgan, H., Kan, A., & Doğan, N. (2007). Measurement and Evaluation in Education. Ankara: Memoir Publishing.
5. Europe, K. (2013). European common recommendations framework for languages: learning, teaching and assessment. Germany: TELC GmbH.
6. Azizoğlu, N. İ., Demirtaş Tolaman, T., & İdi Orange, F. (2019). Academic Writing Skills in Teaching Turkish as a Foreign Language; Encountered Problems and Solution Suggestions. International Journal of Turkish Teaching as a Foreign Language, 2(1),7-22. Barın, E. (2004). Principles in Teaching Turkish to Foreigners. Hacettepe University Turkic Studies (HUTAD), (1), 19-30.
7. Barin, E. (2009). The importance of dictation and written expression in teaching Turkish to Turkish children abroad. TDAY Belleten, 1.21-32.
8. Bıkmaz Bilgen, Ö., & Doğan, N. (2017). Comparison of Inter-rater Confidence Determination Techniques. Journal of Measurement and Evaluation in Education and Psychology, 8 (1) , 63-78.

9. Boylu, E. (2015). Levels of Use of Language Learning Strategies by Learners of Turkish as a Foreign Language. Master Thesis. Ankara: Gazi University Institute of Educational Sciences.
10. Boylu, E. (2019). Assessment and evaluation practices and setting standards in teaching Turkish to foreigners. Doctoral Thesis. Çanakkale: Çanakkale Onsekiz Mart University, Institute of Educational Sciences, Department of Turkish and Social Sciences Education.
11. Büyükikiz, K. (2012). Development of the Writing Self-Efficacy Scale for Foreigners Learning Turkish as a Second Language: A Study of Validity and Reliability. Journal of Mustafa Kemal University Institute of Social Sciences, 9(18), 69-80.
12. Büyüköztürk, Çakmak, Akgün, Karadeniz, & Demirel. (2019). Scientific Research Methods. Ankara: Pegem Academy.
13. Buyukozturk, S. (2007). Manual of data analysis for social sciences. Ankara: Pegem Publishing.
14. Cakir, I. (2010). Why Is It Difficult to Acquire Writing Skills in Foreign Language Teaching? Erciyes University Journal of Social Sciences Institute, 1 (28), 1-12.
15. Cakmak Akata, A. (2009). A Study on the Functionality of the Assessment and Evaluation Process of the Turkish Curriculum (Example of Tekirdağ Province). Bolu: Abant İzzet Baysal University, (Unpublished, Master's Thesis.
16. Çelik, H., Baykal, N. B., & Kılıç Memur, H. N. (2020). Qualitative Data Analysis and Its Basic Principles. Journal of Qualitative Research in Education – Journal of Qualitative Research in Education, 8(1), 379-406.
17. Cetin, B. (2002). The Relationship of Scores Estimated from the Formal Features of Composition in Composition Type Exams with the Scores Obtained from Scoring with Key and General Impression. Ankara: Hacettepe University Institute of Social Sciences, (Unpublished) Master's Thesis.
18. Demir, B. P. (2020). Development of Analytical Rubric for Evaluation of Academic Writing Skills. Amasya University Journal of Education Faculty, 9 (2), 318-327.
19. Deniz, H., & Demir, S. (2019). Development of the Writing Anxiety Scale: Reliability and Validity Study. International Journal of Turkish Literature, Culture and Education, 8(2),1034-1051.
20. Deniz, H., & Demir, S. (2019). Development of the Writing Anxiety Scale: Reliability and Validity Study. International Journal of Turkish Literature, Culture and Education (TEKE), 8 (2).
21. Eran, İ., & Kan, İ. (2004). Reliability and validity in scales. Journal of Uludag University Faculty of Medicine, 30(3), 211-216.
22. Gedik, D. (2009). Teaching Turkish to Foreigners: The Case of Ankara University TÖMER and Gazi University TÖMER. Bolu: Abant İzzet Baysal University Social Sciences Institute, Unpublished Master's Thesis.
23. Gedik, E. (2017). Assessment and Evaluation in Teaching Turkish as a Foreign Language. Istanbul: Istanbul Arel University, Institute of Social Sciences, Master Thesis.
24. Goodrich, H. G. (1997). Understanding Rubrics. Educational Leadership, 54(4).
25. Gocer, A. (2007). Investigation of Textbooks Used in Teaching Turkish as a Foreign Language in Terms of Assessment and Evaluation. Language Journal, 137.30-46.
26. Gocer, A. (2011). Evaluation of students' written expression studies by Turkish teachers. Ondokuz Mayıs University Journal of the Faculty of Education, 30(2), 71- 97.
27. Göçer, A. (2019). The Use of Feedback as an Important Practice Technique Affecting the Functionality of Assessment and Evaluation in Turkish Education. Kırıkkale University Journal of Social Sciences, 9 (1), 111-126.
28. Göçer, A., & Mongolian, S. (2011). Related to Teaching Turkish as a Foreign Language. International Periodical for The Languages, Literature and History of Turkish or Turkic, 6(3),797-810.
29. Güdek, B., & Öztürk, D. (2016). The Effect of Self-Assessment Practices in Cello Teaching on Students' Performance and Attitudes. Turkish Studies, 11(3), 1149-1162.
30. Gunay, V. D. (2007). Text Information. Istanbul: Multilingual Publications.
31. Gunduz, S. (2003). The Art of Writing Stories and Novels. Istanbul: Toroslu Library.
32. Sun, F. (2007). Turkish Teaching and Mental Structuring. Ankara: Nobel Publishing House. Gurses, R. (2006). The Place of Assessment and Evaluation in Language Teaching. Turkish Language Studies Yearbook-Belleten, 54, 131-154.
33. Hizarcioglu, B. O. (2013). Investigation of the Consistency of Using a Rater in the Problem-Solving Process. Bolu: Abant İzzet Baysal University, Institute of Educational Sciences, Master Thesis.
34. Howell, D. C. (2002). Statistical methods for psychology (5th ed.). Pacific Grove CA: Duxbury.
35. Isisak, K. U., & Demirel, O. (2010). Using the Common European Framework of Reference for Languages in the Development of Speaking Skills. Education and Science, 35,156.
36. Kahveci, N., & Şentürk, B. (2021). A Case Study on the Evaluation of Writing Skill in Teaching Turkish as a Foreign Language. International Journal of Education, Technology and Science, 1(4),170–183.
37. Kalfa, M. (2014). Developing Basic Level Students' Writing Skills with Educational Games in Teaching Turkish to Foreigners. Hacettepe University Turkic Studies Studies, 11(85).
38. Karagöz, Y. (2019). SPSS - AMOS - META Applied Statistical Analysis. Ankara: I Nobel Academic Publishing Education Consultancy Tic. Ltd. Sti.
39. Karasar, N. (2003). Scientific Research Method. Ankara: Nobel Publication Distribution.
40. Korkmaz, Y. (2009). The Effect of Rubric Use Education in Science Teaching on Teachers' Views and Practices Regarding Measurement and Evaluation. Konya: Selcuk University, Institute of Science, Department of Primary Education, Science Education Program, Master Thesis.
41. Kürşat Yorgancı, O., & Baş, B. (2021). A Rubric Development Study for B1 Level Writing Skills for Learners of Turkish as a Foreign Language. Literacy Education Studies, 9 (1) , 67-80. DOI: 10.35233/oyea.934684.Maden, S., Dincel, Ö., & Maden, A. (2015). Learning Turksih as a Foreign Language Learners' Writing Anxiety. International Journal of Turkish Literature, Culture and Education (TEKE), 4 (2), DOI: 10.7884/teke.488.
42. Maltepe, S. (2006). Evaluation of Writing Processes and Products in Turkish Lessons in terms of Creative Writing Approach. Ankara: Ankara University/Institute of Educational Sciences, Doctoral Thesis (Unpublished).
43. Mancar, S. A. (2019). Comparison of Inter-rater Reliability Techniques in Performance Based Case Determination. Ankara: Ankara University, Institute of Educational Sciences, Master Thesis.
44. Melanlioglu, D. (2016). Metacognitive Rubric for Listening Skills for Foreign Learners. Erzincan University Journal of Education Faculty, 18(2),1206-1229.
45. Moskal, B. M., & Leydens, J. A. (2000). Scoring Rubric Development: Validity and Reliability. Practical Assessment, Reseach & Evaluation, 7(10). 71-81.
46. Öncü, H. (1994). Measurement and Evaluation in Education. Matser Basım San. and Tic. Ltd. Sti.
47. Ozbay, M. (2009). Listening Education as a Language Skill. Pegem Academy Publishing.
48. Ozbay, M. (2011). Writing Training. Pegem Academy.
49. Özbay, M., & Melanlıoğlu, D. (2009). Evaluation of Idioms in Turkish Education in terms of Teaching and Learning Process. National Education, 38(181), 8 - 19.
50. Ozcelik, D. (1992). Quantification and consideration. OSYM.
51. Ozcelik, D. A. (2016). Quantification and consideration. Pegem Academy.
52. Ozenc, M. (2013). Determination of Alternative Assessment and Evaluation Knowledge Levels of Classroom Teachers. Journal of Dicle University Ziya Gökalp Education Faculty, 21,157-178.
53. Ozkan, U. B. (2019). Analysis Method for Educational Science Research. Pegem Academy.
54. Öztürk, D., & Güdek, B. (2016). Development of Rubric for Cello Performance Evaluation. Journal of Academic Music Research, 2(3),6.
55. Brilliant, B., & Dogan, N. (2014). From Rubric and Rubric Obtained. Hacettepe University Faculty of Education Journal, 29(2), 189-197.
56. Razon, N. (1982). Reading Difficulties. International Journal of Education and Science, 19-29. Richard, J. C. (1983). Listening comprehension: Approach, design, procedure. TESOL Quarterly, 17 (2), 219-240.
57. Selvikavak, E. (2006). An Application on Developing Paragraph Writing Skills of Advanced Students in Teaching Turkish as a Foreign Language. Ankara: Ankara University, Institute of Social Sciences, Linguistics Department, Master Thesis.
58. Sever, S. (2015). Turkish teaching and full learning. Ankara: Memoir Publishing.
59. Sonmez, V. (2003). Introduction to the Teaching Profession. Ankara: Memoir Publishing.
60. Şen, Ü., & Boylu, E. (2017). Writing Anxiety Scale for Learners of Turkish as a Foreign Language. International Journal of Turkish Literature, Culture and Education, 6(2), 1122-1132.
61. Şencan, H. (2005). Reliability and Validity in Social Behavioral Measurements. Ankara: Seçkin Publishing.
62. Sengul, M. (2011). Primary Education II. Evaluation of the Effectiveness of Teaching and Assessment-Evaluation Approaches for Writing Skills in Secondary Turkish Teaching. Elazig: Fırat University, Institute of Educational Sciences, Department of Turkish Education, Doctoral Thesis.
63. Simsek, P. (2017). In Teaching Turkish as a Foreign Language, the Effects of Students' Mother Languages and Their Language Families on their Turkish Writing Skills. Ankara: Gazi University, Institute of Educational Sciences, (Unpublished) Doctoral Thesis.
64. Takil, N. B. (2014). The Effect of Communicative Approach on Writing Skills in Teaching Turkish as a Foreign Language. Ankara: Gazi University, Institute of Educational Sciences, Doctoral Thesis.
65. Tekin, H. (2003). Measurement and Evaluation in Education. Ankara: Judicial Publications: 2003.
66. Temizkan, M. (2003). An Evaluation of Turkish Teachers' Studies in the Framework of Written Expression Activity. Hatay: Mustafa Kemal University Institute of Social Sciences, Unpublished Master's Thesis.
67. Tiryaki, E. N. (2013). Writing Education in Teaching Turkish as a Foreign Language. Native language Education Journal, 1(1), 38-44.
68. Tok, M. (2013). The Need for Academic Writing in Teaching Turkish as a Foreign Language. Journal of Mustafa Kemal University Institute of Social Sciences, 10(23), 1-25.
69. Tok, M., & Potur, O. (2015). Trends of academic studies in the field of writing education (2010 - 2014) . Journal of Mother Tongue Education, 3 (4), 1-25.
70. Turgut, F., & Baykul, Y. (2010). Measurement and Evaluation in Education. Ankara: Pegem Academy.
71. Ulper, H. (2008). A Comparative Study on the Validity of Different Assessment (Grading) Approaches Used in Measuring Writing Skills (Grading) in the Context of the Purposes of Writing Essays. Ankara: 1st National Congress of Measurement and Evaluation in Education and Psychology.
72. Unsal, G. (2008). Writing Teaching. Language Journal, (142), 46-60. DOI: 10.1501/Dilder_0000000102.
73. Wach, E., & Ward, R. (2013). Learning about Qualitative Document Analysis. Institute of Development Studies.
74. Yıldırım, A., & Şimşek, H. (2011). Qualitative research methods in the social sciences. Ankara: Seçkin Publishing.
75. Yorgancı, O. K., & Baş, B. (2021). A Rubric Development Study for B1 Level Writing Skills for Learners of Turkish as a Foreign Language. Literacy Education Research, 9 (1).
76. Zorbaz, K. Z. (2013). Scoring of written expression. Abant İzzet Baysal University Journal of Education Faculty, 13(1), 179 - 193.