Evaluation of the Professional Development Program for Sec-ondary Math Teachers on Item Writing Related to Higher Order Thinking Skills

Abstract

The fact that Turkey’s achievement in large-scale international examinations is not at the expected level is associated with such factors as students, teachers, schools, curricula and assessment approaches. The literature links student-related causes mainly to their lack of higher-order thinking skills; and teacher-related causes to incompetency in writing items to measure these skills, and not participating in professional development activities aimed at measurement of the skills tested in these exams. For these reasons, the present study designed, implemented and evaluated a professional development (PD) program to improve secondary math teachers’ skills of writing higher-order items in compliance with Cognitive Diagnostic Models. The study was conducted with Guskey’s ve levels of evaluating teachers’ PD in the holistic single-case design. The program was attended by 100 participants from 20 differ- ent cities of Turkey. During the study, both quantitative and qualitative data concerning the periods before, during and after the program implementations were obtained through pre and post evaluation questionnaires and worksheets. The results indicated that the PD program was satisfactory; however, it needed improvement in terms of the total time of the program, time allocated for activities and monitoring and supporting teachers’ work at their institutions of service. In addition, it was revealed that the context should not be structured in writing higher- order items of mathematics, which involves multiple qualities and mathematical competence.

Keywords

• Professional development,program evaluation,item writing,higher-order thinking skills,PISA,TIMMS

References

1. Akyüz, G. (2006). Investigation of the effect of teacher and class characteristics on mathematics achievement in Turkey and European Union countries. Elementary Education Online, 5(2), 75-86.
2. Aslan, C. (2011). Effects of teaching applications for developing question asking skills on question forming skills of prospective teachers. Education and Science, 36(160), 236-249.
3. Aydın, A., Selvitopu, A., & Kaya, M. (2018). Resources invested in education and PISA 2015 results: a comparative study. Elementary Education Online, 17(3), 1283-1301. doi:10.17051/ilkonline.2018.466346
4. Aydın, A., Sarıer, Y., & Uysal, Ş. (2012). The comparative assessment of the results of PISA mathematical literacy in terms of socio-economic and socio-cultural variables. Education and Science, 37(164), 20-30.
5. Aydın, A., Erdağ, C., & Taş, N. (2011). A comparative evaluation of PISA 2003-2006 results in reading literacy skills: An example of top-five OECD countries and Turkey. Educational Sciences: Theory and Practice, 11(2), 665-673.
6. Aygün, B., Baran-Bulut, D., & İpek, A.S. (2016). The analysis of the primary school mathematics exam questions according to the math taxonomy. Turkish Journal of Computer and Mathematics Education, 7(1), 62-88.
7. Bayrakçı, M. (2009). In-service teacher training in Japan and Turkey: A comparative analysis of institutions and practices. Australian Journal of Teacher Education, 34(1), 10-22. doi:10.14221/ajte.2009v34n1.2
8. Baysen, E. (2006). The levels of teacher questions and student answers. Kastamonu Education Journal, 14(1), 21-28.

9. Bümen, N. T., et al. (2012). Türkiye bağlamında öğretmenlerin mesleki gelişimi: sorunlar ve öneriler. Milli Eğitim Dergisi, 41(194), 31-50.
10. Çakan, M. (2004). Comparison of elementary and secondary school teachers in terms of their assessment practices and perceptions toward their qualification levels. Ankara University Journal of Faculty of Educational Sciences, 37(2), 99-114. doi:10.1501/Egifak_0000000101
11. Çalık, B. & Aksu, M. (2018). A systematic review of teachers’ questioning in Turkey between 2000-2018. Elementary Education Online, 17(3), 1548-1565. doi:10.17051/ilkonline.2018.466389
12. Çiftçi, E. (2008). Türkiye’de Milli Eğitim Bakanlığı tarafından müzik öğretmenlerine verilen hizmetiçi eğitimin incelenmesi ve müzik öğretmenlerinin hizmet içi eğitim ihtiyaçlarının belirlenmesi. (Unpublished PhD dissertation). Gazi University, Ankara.
13. Çobanoğlu, R. & Kasapoğlu, K. (2010). The whys and hows of Finnish success at PISA. H. U. Journal of Education, 39, 121-131.
14. Demirel, Ö. (2012). Eğitimde program geliştirme: Kuramdan uygulamaya. 18th ed. Ankara: Pegem Akademi.
15. Demirkol, M. (2010). İlköğretim okullarında öğretmenlere yönelik okul-temelli hizmet içi eğitim etkinliklerinin değerlendirilmesi. Milli Eğitim Dergisi, 188, 158-173.
16. Fitzpatrick, J.L., Sanders, J.R., & Worthen, B.R. (2004). Program evaluation: alternative approaches and practical guidelines. Boston: Allyn and Bacon.
17. Guskey, T.R. (2000). Evaluating professional development. California: Corwin Press.
18. Guskey, T.R. (2002). Does it make a difference? Evaluating professional development. Educational Leadership, 59(6), 45–51.
19. Güler, H.K. (2013). Türk öğrencilerin PISA’da karşılaştıkları güçlüklerin analizi Uludağ Üniversitesi Eğitim Fakültesi Dergisi, 26(2), 501-522.
20. Güler, G., Özdemir, E., and Dikici, R. (2012). İlköğretim matematik öğretmenlerinin sınav soruları ile SBS matematik sorularının Bloom taksonomosine göre karşılaştırmalı analizi. Erzincan Üniversitesi Eğitim Fakültesi Dergisi, 14(1), 41-60.
21. Güner, N., Sezer, R., & Akkuş İspir, O. (2013). İlköğretim ikinci kademe öğretmenlerinin TIMSS hakkındaki görüşleri. Pamukkale Üniversitesi Eğitim Fakültesi Dergisi, 33(1), 11-29.
22. İnceçam, B., Demir, E., & Demir, E. (2018). Competencies of middle school teachers to prepare open-ended items used in open-ended test for in-classroom assessment. Elementary Education Online, 17(4), 1912-1927. doi:10.17051/ilkonline.2019.506900
23. İpek, A.S. & Özdemir, E. (2019). Ortaokul matematik öğretmenlerinin yazılı sınav sorularının incelenmesi. Akdeniz Eğitim Araştırmaları Dergisi, 13(27), 244-262. doi:10.29329/mjer.2019.185.11
24. Karaman, M. & Bindak, R. (2017). İlköğretim matematik öğretmenlerinin sınav soruları ile TEOG matematik sorularının yenilenmiş Bloom Taksonomisi’ne göre analizi. Current Research in Education, 3(2), 51-65.
25. Karip, E. (2017). Türkiye’nin TIMSS 2015 performansı üzerine değerlendirme ve öneriler (TEDMEM Analiz Dizisi 5) Ankara: Türk Eğitim Derneği Yayınları. Retrieved from: https://tedmem.org/download/turkiyenin-timss-2015-performansi-uzerine-degerlendirme-oneriler?wpdmdl=2515&refresh=5d18a91b948fa1561897243
26. Kazu, İ. Y. & Kerimgil, S. (2008). Yeni atanan öğretmenlerin hizmetiçi eğitime ilişkin görüşleri (Elazığ ili örneği). Gazi Üniversitesi Endüstriyel Sanatlar Eğitim Fakültesi Dergisi, 23, 14-30.
27. Lewis, A. & Smith, D. (1993). Defining higher order thinking. Theory into practice, 32 (3), 131-137. doi:10.1080/00405849309543588
28. MoNE (2016a). Uluslararası öğrenci değerlendirme programı PISA 2015 ulusal raporu Ankara: Ölçme Değerlendirme ve Sınav Hizmetleri Genel Müdürlüğü. Retrieved from: http://pisa.meb.gov.tr/?page_id=22
29. MoNE (2016b). TIMMS 2015 ulusal matematik ve fen bilimleri ön raporu 4. ve 8. sınıflar. Ankara: Ölçme, Değerlendirme ve Sınav Hizmetleri Genel Müdürlüğü. Retrieved from: https://odsgm.meb.gov.tr/meb_iys_dosyalar/2017_06/23161945_timss_2015_on_raporu.pdf
30. MoNE (2014). TIMSS 2011 ulusal matematik ve fen raporu: 8. sınıflar. Ankara: Yenilik ve Eğitim Teknolojileri Genel Müdürlüğü. Retrieved from: http://odsgm.meb.gov.tr/test/analizler/docs/timss/TIMSS-2011-8-Sinif%20Raporu.pdf
31. MoNE (2010a). PISA 2006 projesi ulusal nihai rapor. Ankara: Eğitimi Araştırma Geliştirme Dairesi Başkanlığı. Retrieved from: http://pisa.meb.gov.tr/wp-content/uploads/2013/07/PISA2006-Ulusal-Nihai-Rapor.pdf
32. MoNE (2010b). Uluslararası öğrenci değerlendirme programı PISA 2009 ulusal ön raporu. Ankara: Eğitimi Araştırma Geliştirme Dairesi Başkanlığı. Retrieved from: http://pisa.meb.gov.tr/wp-content/uploads/2013/07/PISA-2009-Ulusal-On-Rapor.pdf
33. MoNE (2005). PISA 2003 projesi ulusal nihai rapor. Ankara: Eğitimi Araştırma ve Geliştirme Dairesi Başkanlığı. Retrieved from: http://pisa.meb.gov.tr/wp-content/uploads/2013/07/PISA-2003-Ulusal-Nihai-Rapor.pdf
34. Mullis, I.V.S., et al. (2015). TIMSS 2015 international results in mathematics. IEA, TIMMS and PIRLS International Study Center, Lynch School of Education, Boston College. Retrieved from: timss2015.org/download-center
35. OECD (2016). PISA 2015 results (Volume I): Excellence and equity in education. Paris: PISA, OECD Publishing. Retrieved from: http://dx.doi.org/10.1787/9789264266490-en
36. Opfer, V.D. & Pedder, D. (2011). Conceptualizing teacher professional learning. Review of Educational Research, 81(3), 376-407. doi:10.3102/0034654311413609
37. Ornstein, A.C. & Hunkins, F.P. (2009). Curriculum: foundations, principles and issues. US: Pearson.
38. Özenç, B. & Arslanhan, S. (2010). PISA 2009 Sonuçlarına İlişkin Bir Değerlendirme Ankara: TEPAV. Retrieved from: https://www.tepav.org.tr/upload/files/1292255907-8.PISA_2009_Sonuclarina_Iliskin_Bir_Degerlendirme.pdf
39. Özer, B. (2004). In-service training of teachers in Turkey at the beginning of the 2000s. Journal of In-Service Education, 30(1), 89-100. doi:10.1080/13674580400200238
40. Peak, L., et al. (1996). Pursuing excellence: a study of U.S. eighth-grade mathematics and science teaching, learning, curriculum, and achievement in international context. Initial findings from the third international mathematics and science study. Washington, DC.: Office of Educational Research and Improvement (ED). Retrieved from: https://files.eric.ed.gov/fulltext/ED400209.pdf
41. Sahin, A. & Kulm, G. (2008). Sixth grades mathematics teachers’ intentions and use of probing, guiding and factual questions. Journal of Mathematics Teacher Education, 11(3), 221-241. doi:10.1007/s10857-008-9071-2
42. Smith, C.L., et al. (2003). How teachers change: A study of Professional development in adult education (Report 25). Cambridge, MA: National Center for the Study of Adult Learning and Literacy. Retrieved from: https://files.eric.ed.gov/fulltext/ED508607.pdf
43. Stein, M.K., et al. (2000). Implementing standards-based mathematics instructions: a casebook for professional development. New York: Teachers College.
44. Stein, M.K. & Smith, M.S. (1998). Mathematical Tasks as a Framework for Reflection: From Research to Practice. Mathematics Teaching in the Middle School, 3(January 1998), 268–75.
45. Stufflebeam, D.L., Madaus, G.F., & Kellaghan, T., Eds. (2000). Evaluation models: viewpoints on educational and human services evaluation. 2nd ed. Boston: Kluwer Academic Publishers.
46. Şata, M. (2016). Türk eğitim sistemi’nde sınıf içi ile geniş ölçekli ölçme ve değerlendirmeye genel bir bakış. Current Research in Education, 2(1), 53-60.
47. Uslu, Ö. (2013). A new model proposal for evaluating teachers' professional development programs. H. U. Journal of Education, Special Issue (1), 359-374.
48. Vlaardingerbroek, B. & Taylor, T.G. (2003). Teacher education variables as correlates of primary science ratings in thirteen TIMSS systems. International Journal of Educational Development, 23, 429-438. doi:10.1016/S0738-0593(03)00014-2
49. Yatağan, M. (2014) Fen ve teknoloji öğretim programının öğrenci ve öğretmen özelliklerine göre değerlendirilmesi: TIMSS 2007 ve TIMSS 2011 verileri ile bir durum analizi. (Unpublished PhD thesis). Gazi University, Ankara.
50. Yıldırım, A. & Şimşek, H. (2013). Sosyal Bilimlerde Nitel Araştırma Yöntemleri Ankara: Seçkin Yayıncılık.
51. Yin, R.K. (2003). Case study research: Design and methods.3rd ed. Thousand Oaks, CA: Sage.
52. Yurdakul, B., et al. (2014). Evaluation of the professional development program on web based content development. Educational Sciences: Theory and Practice, 14(4) 1427-1437. doi: 10.12738/estp.2014.4.2131
53. Yücel, C., Karadağ, E. & Turan, S. (2013). TIMSS 2011 ulusal ön değerlendirme raporu. Eskişehir Osmangazi Üniversitesi Eğitim Fakültesi Eğitimde Politika Analizi Raporlar Serisi I. Retrieved from: https://www.researchgate.net/publication/317429398_TIMSS_2011_Ulusal_On_Degerlendirme_Raporu