Understanding by Design (UbD) Ünite Planı Değerlendirmeye ve Geri Bildirim Sağlamaya Yönelik Analitik Rubrik Geliştirme Çalışması

Year 2024, Volume: 25 Issue: 2, 807 - 829, 31.08.2024

Emel Güneş Savul , Selçuk Doğan , Ümran Yazıcılar Nalbantoğlu

https://doi.org/10.17679/inuefd.1446457

Abstract

Çalışmada Understanding by Design (UbD) (Anlamaya Dayalı Tasarım) ünite planlarını değerlendirmek ve planı geliştiren kişilere etkili geri bildirim sağlama amacıyla analitik bir rubrik geliştirmek amaçlanmıştır. Geliştirilen rubriğin geçerlik ve güvenirlik çalışmaları tamamlanmıştır. Uzman görüşleri ve literatür taraması ile belirlenen ölçütlere UbD’yle uyumlu olacak şekilde üç kategoride göstergeler yazılmıştır. Hazırlanan ölçütler ve kategoriler uzman görüşleri alınarak düzenlenmiştir. Farklı sınıf seviyeleri ve farklı dersler kapsamında hazırlanmış dört UbD ünite planı, UbD uzmanları/değerlendiricileri tarafından rubrik yardımıyla değerlendirmesi istenmiştir. Değerlendiriciler arasında iç tutarlığa bakmak amacıyla hesaplanan Fleiss Kappa tutarlılık katsayısı planlar için sırasıyla .42, .41, .39 ve .36 çıkmıştır. Bu hesaplamalar orta derecede tutarlılığı işaret etmektedir. Bu amaçla uyuşmanın yaşanmadığı ölçütler değerlendirici görüşleri ile birlikte tekrar revize edilerek iç tutarlık çalışmaları yenilenmiştir. Yeniden hesaplanan Fleiss Kappa tutarlılık katsayısı sırasıyla .71, .72, .75 ve .73 çıkmıştır. Bu hesaplamalar önemli derecede uyuşmaya işaret etmektedir. Geliştirilen analitik rubrik, farklı ders, seviye ve nitelikteki UbD ünite planlarını değerlendirmede kullanılabilecek geçerli ve güvenilir bir araç olarak sunulmuştur. Bu çalışma, Türkçe literatürde ilk kez ortaya çıkan bir analitik rubrik ile UbD ünite planlarının etkili bir şekilde değerlendirilmesine katkı sağlamaktadır.

Keywords

Analitik rubrik, Understanding by Design (UbD), Öğretmen geribildirimi, Rubrik geliştirme

References

• Akbaş, O., & Duman, S. N. (2018). Tasarım yoluyla anlama (Understanding by Design) bağlamında eğitim programlarında sondan başa doğru tasarım ve büyük fikre odaklanma. [Bildiri özeti]. V.International Multidisciplinary Congress of Eurasia (IMCOFE), Roma.
• Angell, K. (2015). The Application of reliability and validity measures to assess the effectiveness of an undergraduate citation rubric. Behavioral and Social Sciences Librarian, 34(1), 2–15. https://doi.org/10.1080/01639269.2015.996481.
• Arter, J., & McTighe, J. (2001). Scoring rubrics in the classroom: using performance criteria for assessing and ımproving student performance. Corwin Press. Benjamin, S. (2007). The quality rubric: A Systematic approach for ımplementing quality principles and tools in classrooms and schools. Asq Pr.
• Berberoğlu, G. (2006). Sınıf içi ölçme değerlendirme teknikleri. Morpa Kültür Yayınları.
• Brookhart, S. M. (1999). The art and science of classroom assessment: The missing part of pedagogy. ASHE-ERIC Higher Education Report, 27(1).
• Bryant, C., Maarouf, S., Burcham, J. G., & Greer, D. (2017). The examination of a teacher candidate assessment rubric: A confirmatory factor analysis. Teaching and Teacher Education, 57, 79–96. https://doi.org/10.1016/j.tate.2016.03.012.
• Buyruk, A. A., Erdoğan, P., Deveci, C. Ç., & Toy, B. Y. (2018). Motivasyon modeli ile zenginleştirilmiş anlamaya dayalı öğretim tasarımının öğrencilerin İngilizce konuşma becerisine ve motivasyonlarına etkisi: tasarım tabanlı bir araştırma. Yükseköğretim ve Bilim Dergisi, (1), 82-94.
• Carbaugh B., Marzano R. & Toth, M. (2017). The Marzano focused teacher evaluation model. Learning Sciences International.
• Centre for Development and Enterprise (2015). Teacher evaluation lessons from other countries. CDE Publishing.
• Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. https://doi.org/10.1177/001316446002000104.
• Close, K., Amrein-Beardsley, A., & Collins, C. (2020). Putting teacher evaluation systems on the map: An overview of states' teacher evaluation systems post–every student succeeds act. Education Policy Analysis Archives, 28, 58. https://doi.org/10.14507/epaa.28.5252.
• Crawforth, K. (2001). Measuring the interrater reliability of a data collection instrument developed to evaluate anesthetic outcomes. [Doktora tezi, Wayne State University].
• Danielson, C., & McGreal, T. (2000). Teacher evaluation to enhance professional practice. Association for Supervision and Curriculum Development.
• Danielson, C. (2013). The framework for teaching evaluation instruments. Princeton: The Danielson Group. Darling-Hammond, L., Amrein-Beardsley, A., Haertel, E., & Rothstein, J. (2012). Evaluating teacher evaluation. Phi Delta Kappan, 93(6), 8-15. https://doi.org/10.1177/003172171209300603.
• De Boer I. de Vegt F. Pluk H. &Latijnhouwers M. (2021). Rubrics – a tool for feedback and assessment viewed from different perspectives: Enhancing learning and assessment quality (IAMSE Manuals). Springer.
• DeVellis, R. F., &Thorpe, C. T. (2021). Scale development: Theory and applications. Applied social research method. SAGE Publications.
• Dreyer Leon, S., &Thomas, L. (2015). Collaboration, rubrics, and teacher evaluation. In rubric nation critical ınquiries on the ımpact of rubrics in education. Information Age Publishing.
• Doğan, S., &Altun. S. (2013). Teachers' perceptıons on the effectıveness of currıculum mappıng: the case of Turkey'. Journal Of Educational and Instructional Studies In The World, 3(4), 50-60.
• Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 378–382. https://doi.org/10.1037/h0031619.
• Gaertner, Holger. (2014). “Effects of student feedback as a method of self-evaluating the quality of teaching.” Studies in Educational Evaluation, 42, 91–99, https://doi.org/10.1016/j.stueduc.2014.04.003.
• Gordon, S., & McGhee, M. (2019). The power of formati evaluation of teaching. In M. Derrington ve J. Brandon (Eds.), Differentiated teacher evaluation and professional learning: policies and practices for promoting career growth. Palgrave Macmillan.
• Gül, M., Altun, S., &Toy, B. Y. (2021). Anlamaya dayalı tasarım uygulamasının (UbD) öğrencilerin bilişsel ve duyuşsal gelişimlerine etkisi: T.C. İnkılap Tarihi ve Atatürkçülük dersi üzerine bir çalışma. Yüzüncü Yıl Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 51, 123–152.
• Hasselquist, L., & Bertolini, K. (2018). Developing effective rubrics. NACTA Journal, 62(4), 379–380.
• Hellrung, K., & Hartig, J. O. (2013). Understanding and using feedback – A review of empirical studies concerning feedback from external evaluations to teacher. Educational Research Review, 9, 174–190.
• Holtzapple, E. (2003). Criterion-related validity evidence for a standards-based teacher evaluation system. J Pers Eval Edu, 17, 207–219.
• Jonsson, A. & Svingby, G. (2007) The use of scoring rubrics: Reliability, validity and educational consequences. Educational Researcher Review, 2(2), 130-144.
• Kane, T., Kerr, K., & Pianta, R. (2014). Designing teacher evaluation systems. Josey-Bass.
• Kimball, S. (2001). Innovations in teacher evaluation: case studies of two school districts with teacher evaluation systems based on the framework for teaching (Publication No. 3033269) [Doktora Tezi, University of Wisconsin-Madison]. ProQuest Dissertations and Theses Global.
• Lynda, T., Matt, M., & Ken, S. (2021). Making classroom observations matter. Educational Leadership, 78(7), 56-62.
• Marzano, R. J. (2012). The two purposes of teacher evaluation. Educational Leadership, 70(3), 14–19.
• Marzano, R., & Toth, M. (2013). Teacher evaluation that makes a difference. ASCD.
• Marshall, K. (2015). Getting teacher evaluation rubrics right. In rubric nation critical ınquiries on the ımpact of rubrics in education. Informatiın Age Publishing.
• Matthews, M. (2023). Learn how to create powerful rubrics for authentic assessments. Toodle.
• Mertler, C. A. (2001). Designing scoring rubrics for your classroom. Practical Assessment, Research and Evaluation, 7(25), 25. https://doi.org/10.7275/gcy8-0w24.
• Messick, S. (1992). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2),13–23. https://doi.org/10.3102/0013189x023002013.
• McTighe, J., & Wiggins, G. (2015). Solving 25 problems in unit design: How do I refine my units to enhance student learning? (ASCD Arias). ASCD.
• McTighe, J., & Frontier, T. (2022). How to provide better feedback through rubrics. ASCD, 79(7), 17–23.
• Moskal, B. M., & Leydens, J. A. (2000). Scoring rubric development: validity and reliability. Practical Assessment, Research and Evaluation, 7(10), 10. https://doi.org/10.7275/q7rm-gg74.
• Organization for Economic Co-operation and Development. (2009a), Creating effective teaching and learning environments: First results from TALIS. OECD Publishing.
• Organization for Economic Co-operation and Development. (2009b). Teacher evaluation a conceptual framework and examples of country practices. OECD Publishing.
• Organization for Economic Co-operation and Development. (2013). Teachers for the 21st century using evaluation to improve teaching. OECD Publishing.
• Panadero, E., & Jönsson, A. (2013). The use of scoring rubrics for formative assessment purposes revisited: A review. Educational Research Review, 9, 129–144. https://doi.org/10.1016/j.edurev.2013.01.002.
• Popham, W. J. (2010). Classroom assessment: what teachers need to know (6th ed.). Pearson College Div; Columbus, OH.
• Roblyer, M. D., & Wiencke, W. R. (2003b). Design and use of a rubric to assess and encourage interactive qualities in distance courses. American Journal of Distance Education, 17(2), 77–98. https://doi.org/10.1207/s15389286ajde1702_2.
• Stellmack, M. A., Konheim-Kalkstein, Y. L., Manor, J. E., Massey, A. R., & Schmitz, J. (2009). An assessment of reliability and validity of a rubric for grading APA-Style introductions. Teaching of Psychology, 36(2), 102–107. https://doi.org/10.1080/00986280902739776.
• Shulman, L. (1986). Those who understand knowledge growth in teaching. Educational Researcher, 15(2), 4-14. doi: 10.3102/0013189x015002004.
• Stronge, J. (2018). Qualities of effective teachers (3rd ed.). ASCD.
• Taggart, G. L., Phifer, S. J., Nixon, J., & Wood, M. (2007). Rubrics: a handbook for construction and use.
• Thorndike, R. M., & Thorndike-Christ, T. (2009) Measurement and evaluation in psychology and education. Pearson.
• Uluçinar, U. (2021). Findings of qualitative studies on understanding by design: A Meta-Synthesis. International Journal of Curriculum and Instructional Studies, 11(2), 167-194.
• Wiggins, G. (1991). Standards, not standardization: Evoking quality student work. Educational Leadership. 48(5), 18-25.
• Wiggins, G., & McTighe, J. (2005). Understanding by design (2nd Expanded). Assn. For Supervision and Curriculum Development.
• Wiggins, G., & McTighe, J. (2011). The understanding by design guide to creating high-quality units (1st ed.). ASCD.
• Wiggins, G. P., & McTighe, J. (2012). The understanding by design guide to advanced concepts in creating and reviewing units. ASCD.
• Yurtseven, N., Doğan, S., & Altun, S. (2013). UbD (Understanding by Design) modeline göre hazırlanmış farklılaştırılmış fen ve teknoloji dersi öğretim planı: Türkiye örneği. Siirt Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 1, 1-20.