Designing Web-Based “Measurement and Evaluation” Learning Modules for Teachers’ Needs

Abstract

The challenging aspects of scheduling face-to-face in-service training programs are a barrier to make needed training widespread and accessible. Due to the limitations, the necessity has arisen to design interactive and updatable digital learning materials for teachers that have been prepared by experts and that can be accessed whenever desired. This design and development research presents a design of digital learning materials that has been drawn up on the basis of scientific processes and in the knowledge of teachers’ inadequacies as regards the topic of measurement and evaluation. The learning materials produced in this research, Web-based Measurement and Evaluation Learning Modules (W-MELM), are based on the critical aspects of item writing and test development and the needs of teachers based on the literature. The ADDIE model was taken as the basis of W-MELM in the design process. Five experts were consulted in leading to the revision of the modules to make them ready for operation. The evaluation stage consisted of using a Likert-type data collection tool of 18 items to collect teachers’ (n=50) opinions regarding W-MELM. As a result of analysis, the teacher’s views acknowledged that W-MELM that had been designed met their needs and was an effective learning tool.

Keywords

• measurement
• evaluation
• assessment literacy
• ADDIE
• teacher training
• distance education

Öğretmen İhtiyaçlarına Dönük Web Tabanlı “Ölçme ve Değerlendirme” Eğitim Seti Tasarımı

Abstract

Ağırlıklı puanlama ile ilgili geçmiş çalışmalar incelendiğinde, genellikle ağırlıklandırılmamış puanlarla olan korelasyonların incelendiği, buna karşın ağırlıklandırmanın öğrencilerin geçme – kalma oranlarına olan etkisinin araştırılmadığı görülmüştür. Bu çalışmada öğretmen yapımı çoktan seçmeli 34 maddelik bir başarı testinin 431 kişilik bir gruba öğrenme yönetim sistemi aracılığıyla uygulanmıştır. Daha sonra ağırlıklandırılmamış (1 – 0) ve madde güçlüğüne göre ağırlıklandırılmış (Qj – 0) puanlara göre madde ve test istatistiklerinin, öğrencilerin dersten geçme ve kalma durumlarının karşılaştırılması amaçlanmıştır. Aynı zamanda ağırlıklandırılmış puanların 100’lük puan sistemine çevrilmesine yönelik bir öneri de sunulmuştur. Veri analizi sonucunda 1 – 0 ve Qj – 0 yöntemlerine göre elde edilen McDonald’s Omega iç tutarlık katsayıları sırasıyla .725 ve .721 olarak elde edilmiştir. İki yönteme göre elde edilen öğrenci puanları arasında ise Pearson momentler çarpım korelasyon katsayısı .916 ve Spearman sıra farkları korelasyon katsayısı .926 olarak bulunmuştur. Aynı zamanda sırasıyla 50 ve 60 puana göre ölçüt dayanaklı bir değerlendirme yapıldığında, her iki yönteme göre dersten başarılı ve başarısız sayılan öğrenci sayıları belirlenmiştir. Buna göre Qj – 0 puanlama yöntemine göre daha çok öğrencinin dersten başarısız sayılacağı bulunmuş, ancak buna karşın bu yöntemin bireyler arasındaki farklılıkları daha iyi ortaya koyabileceği anlaşılmıştır.

Keywords

• ölçme
• değerlendirme
• ölçme ve değerlendirme okuryazarlığı
• ADDIE
• öğretmen eğitimi
• uzaktan eğitim

References

1. Ally, M. (2004). Foundations of educational theory for online learning. Theory and practice of online learning, 2, 15-44.
2. Alkharusi, H., Kazem, A. M., & Al-Musawai, A. (2011). Knowledge, skills, and attitudes of preservice and inservice teachers in educational measurement. Asia-Pacific Journal of Teacher Education, 39(2), 113–123. https://doi.org/10.1080/1359866X.2011.560649
3. American Federation of Teachers, National Council on Measurement in Education, N. E. A. (1990). Standards for teacher competence in educational assessment of students.
4. Anderson, L. W., & Krathwohl, D. R. (2010). Bloom’un eğitim hedefleri ile ilgili sınıflamasının güncelleştirilmiş biçimi [A revision of Bloom's taxonomy of educational objectives], (Durmuş Ali Özçelik, trans.). Ankara: Pegem Academy Publishing.
5. Bayat, S., & Şentürk, Ş. (2015). Fizik, kimya, biyoloji ortaöğretim alan öğretmenlerinin alternatif ölçme değerlendirme tekniklerine ilişkin görüşleri [Physics, Chemistry, Biology Teachers' Views on Alternative Assessment and Evaluation Techniques in Secondary School]. Amasya Üniversitesi Eğitim Fakültesi Dergisi, 4(1), 118-135.
6. Benzer, A., & Eldem, E. (2013). Türkçe ve edebiyat öğretmenlerinin ölçme ve değerlendirme araçları hakkında bilgi düzeyleri [Level of the Information About Turkish and Literature Teachers’ Measurement and Assessment Materials]. Kastamonu Eğitim Dergisi, 21(2), 649-664.
7. Brookhart, S. M. (2011). Educational assessment knowledge and skills for teachers. Educational Measurement: Issues and Practice, 30(1), 3–12. https://doi.org/10.1111/j.1745-3992.2010.00195.x
8. Brookhart, S. M., Moss, C. M., & Long, B. A. (2010). Teacher inquiry into formative assessment practices in remedial reading classrooms. Assessment in Education: Principles, Policy and Practice, 17(1), 41–58. https://doi.org/10.1080/09695940903565545

9. Büyüköztürk, Ş., Kılıç Çakmak, E., Akgün, Ö., Karadeniz, Ş., & Demirel, F. (2014). Bilimsel araştırma yöntemleri [Scientific research methods] (17th ed.). Ankara: Pegem Academy Publishing.
10. Clark, R. C., & Mayer, R. E. (2003). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. Pfeiffer.
11. Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Holt, Rinehart and Winston, 6277 Sea Harbor Drive, Orlando, FL.
12. Çakan, M. (2004). Öğretmenlerin ölçme-değerlendirme uygulamaları ve yeterlik düzeyleri: İlk ve ortaöğretim [Comparison of elementary and secondary school teachers in terms of their assessment practices and perceptions toward their qualification levels]. Ankara Üniversitesi Eğitim Bilimleri Fakültesi Dergisi, 37(2), 99-114.
13. Çağıltay, K., & Göktaş, Y. (Eds.). (2013). Öğretim teknolojilerinin temelleri: Teoriler, araştırmalar, eğilimler [Fundamentals of instructional technology: Theories, research, trends]. Ankara: Pegem Academy Publishing.
14. Çelen, F. K., Çelik, A., & Seferoğlu, S. S. (2013). Analysis of teachers’ approaches to distance education. Procedia-Social and Behavioral Sciences, 83, 388-392.
15. DeLuca, C., & Bellara, A. (2013). The current state of assessment education. Journal of Teacher Education, 64(4), 356–372. https://doi.org/10.1177/0022487113488144
16. DeLuca, C., LaPointe-McEwan, D., & Luhanga, U. (2016). Teacher assessment literacy: a review of international standards and measures. Educational Assessment, Evaluation and Accountability, 28(3), 251–272. https://doi.org/10.1007/s11092-015-9233-6
17. Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8.
18. Ferdig, R. E., Baumgartner, E., Hartshorne, R., Kaplan-Rakowski, R., & Mouza, C. (2020). Teaching, technology, and teacher education during the covid-19 pandemic: Stories from the field. Waynesville, NC, USA: Association for the Advancement of Computing in Education (AACE).
19. Fulcher, G. (2012). Assessment literacy for the language classroom. Language Assessment Quarterly, 9(2), 113–132. https://doi.org/10.1080/15434303.2011.642041
20. Gaitas, S., & Alves Martins, M. (2017). Teacher perceived difficulty in implementing differentiated instructional strategies in primary school. International Journal of Inclusive Education, 21(5), 544-556.
21. Gagné, R. M., & Briggs, L. J. (1979). Principles of instructional design. Kruse, New York.
22. Gotch, C. M., & French, B. F. (2014). A systematic review of assessment literacy measures. Educational Measurement: Issues and Practice, 33(2), 14–18. https://doi.org/10.1111/emip.12030
23. Gültekin, S. (2014). Testlerde kullanılacak madde türleri, hazırlama ilkeleri ve puanlaması [Test item, developing principles and scoring. N. Demirtaşlı (Ed.),In Eğitimde Ölçme ve Değerlendirme [Measurement and Evaluation in Education] (2nd ed.) (ss. 171-251). Ankara: Edge Academy.
24. Hamurcu, H. (2018). Comparative examination of the primary school science curricula in Turkey (Curricula of 1992, 2001, 2005, 2013 and 2017). Journal of Education and Learning, 7(2), 261-279.
25. Herbenhahn, B. R. (1988). Introduction to theories of learning. (3. Baskı). New Jersey: Prentice Hall.
26. Hooper, S., & Rieber, L. P. (1995). Teaching with technology. In A. C. Ornstein (Ed.), Teaching: Theory into practice (pp. 154-170). Needham Heights, MA: Allyn and Bacon.
27. Mayer, R. E. (2005). The Cambridge handbook of multimedia learning. New York: Cambridge University Press.
28. Marzano, R. J. (2006). Classroom assessment & grading that work. ASCD.
29. Mertler, C. A. (1999). Assessing student performance: A descriptive study of the classroom assessment practices of Ohio teachers. Education, 120(2), 285. https://www.questia.com/read/1G1-59644154/assessing-student-performance-a-descriptive-study
30. Mertler, C. A. (2003, October). Preservice versus inservice teachers’ assessment literacy: Does classroom experience make a difference? Annual Meeting of the Mid-Western Educational Research Association.
31. Mertler, C. A. (2009). Teachers’ assessment knowledge and their perceptions of the impact of classroom assessment professional development. Improving Schools, 12(2), 101–113. https://doi.org/10.1177/1365480209105575
32. Mertler, C. A., & Campbell, C. (2005). Measuring teachers’ knowledge & application of classroom assessment concepts: development of the “Assessment Literacy Inventory.” Annual Meeting of the American Educational Research Association.
33. Milli Eğitim Bakanlığı. (1995). Milli eğitim bakanlığı hizmet içi eğitim yönetmeliği [Ministry of National Education in-service training regulation]. Resmi gazete: 8.04.1995/22252.
34. Milli Eğitim Bakanlığı. (2018). Millî eğitim istatistikleri, örgün eğitim 2017-2018 [National education statistics formal education 2017-2018]. Ankara: Turkish Statistical Institute.
35. Murchan, D., Gerry, S., Vula, E., Bajgora, A. G., & Balidemaj, V. (2013). Formative assessment.
36. Plake, B. S., Impara, J. C., & Fager, J. J. (1993). Assessment Competencies of Teachers: A National Survey. Educational Measurement: Issues and Practice, 12(4), 10–12. https://doi.org/10.1111/j.1745-3992.1993.tb00548.x
37. Peterson, C. (2003). Bringing ADDIE to life: Instructional design at its best. Journal of Educational Multimedia and Hypermedia, 12(3), 227-241.
38. Popham, W. J. (2005). Seeking redemption for our psychometric sins. Educational Measurement: Issues and Practice, 22(1), 45–48. https://doi.org/10.1111/j.1745-3992.2003.tb00117.x
39. Quilter, S. M., & Gallini, J. K. (2000). Teachers’ assessment literacy and attitudes. Teacher Educator, 36(2), 115–131. https://doi.org/10.1080/08878730009555257
40. Reigeluth, C. M. (1983). Instructional design: What is it and why is it. Instructional-design theories and models: An overview of their current status, 1, 3-36.
41. Richey, R. C., & Klein, J. D. (2008). Research on design and development. In J.M. Spector, M.D. Merrill, J.van Merrienboer, & M.P. Driscoll (Eds.), Handbook of research on educational communications and technology (pp.748-757). New York: Routledge.
42. Richey, R. C., Klein, J. D., & Nelson, W. A. (2004). Developmental research: studies of instructional design and development. In D.H. Jonassen (Ed.), Handbook of Research on Educational Communications and Technology (pp. 1099-1130). Manwah, NJ: Lawrence Erlbaun Associates.
43. Sezer, B., Karaoğlan Yılmaz, F. G., & Yılmaz, R. (2013). Integrating technology into classroom: the learner-centered instructional design. International Journal on New Trends in Education & Their Implications, 4(4).134-144.
44. Şenel, S. (2018a). Programlama konusu “kazanımları” ile “sınıf içi ölçme süreçlerinin” bilişsel düzeylerinin karşılaştırılması [Comparison of programming “acquisitions” and “in-class measurement activities” in terms of cognitive level]. International Journal of Computers in Education, 1(2), 1-20.
45. Şenel, S. (2018b). Öğretmenlerin kaynaştırma uygulamalarındaki ölçme ve değerlendirme yeterlikleri ve problemleri [Problems and insufficiencies of teachers’ in educational measurement and evaluation of inclusive education]. 6th International Congress on Measurement and Evaluation in Education and Psychology (September 5-8, 2018). Prizren, Kosova.
46. Şenel, S., Pekdağ B., & Günaydın, S. (2018). Kimya öğretmenlerinin ölçme ve değerlendirme sürecinde yaşadıkları problemler ve yetersizlikler [Chemistry teachers’ problems and insufficiencies in educational measurement and evaluation]. Necatibey Eğitim Fakültesi Elektronik Fen ve Matematik Eğitimi Dergisi, 12(1). 419-441.
47. Taşlıbeyaz, E., Karaman, S., & Göktaş, Y. (2014). Öğretmenlerin uzaktan hizmet içi eğitim deneyimlerinin incelenmesi [Examining the experiences of teachers received in service training through distance education]. Ege Eğitim Dergisi, 15(1), 139-160.
48. Thorndike, R. M., & Tracy Thorndike, C. (2014). Measurement and evaluation in psychology and education (8th ed.). Pearson Education Limited. https://doi.org/10.2307/2282039
49. Topkaya, Y., & Yılar, B. (2016). Sosyal bilgiler öğretmenlerinin alternatif ölçme ve değerlendirme teknikleri hakkındaki görüşleri [Opinions of social studies teachers about alternative assessment and evaluation techniques]. Erzincan Üniversitesi Eğitim Fakültesi Dergisi, 18(1), 593-610.
50. Uçar, R., & İpek, C. (2006). İlköğretim okullarında görev yapan yönetici ve öğretmenlerin MEB hizmet içi eğitim uygulamalarına ilişkin görüşleri [Inservice education needs of class teachers teaching at elementary schools]. Yüzüncü Yıl Üniversitesi Eğitim Fakültesi Dergisi, 3(1), 34-53.
51. Wang, F., & Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments. Educational Technology Research and Development, 53(4), 5–23.
52. Wang, T. H., Wang, K. H., Wang, W. L., Huang, S. C., & Chen, S. Y. (2004). Web-based Assessment and Test Analyses (WATA) system: development and evaluation. Journal of Computer Assisted Learning, 20(1), 59–71. https://doi.org/10.1111/j.1365-2729.2004.00066.x
53. Wang, T. H., Wang, K. H., Wang, W. L., Huang, S. C., & Chen, S. Y. (2008). Designing a Web-based assessment environment for improving pre-service teacher assessment literacy. Computers and Education, 51(1), 448–462. https://doi.org/10.1016/j.compedu.2007.06.010
54. Xu, Y., & Brown, G. T. L. (2016). Teacher assessment literacy in practice: A reconceptualization. Teaching and Teacher Education, 58, 149–162. https://doi.org/10.1016/j.tate.2016.05.010
55. Yalın, H. İ. (2000). Öğretim teknolojileri ve materyal geliştirme [Instructional technologies and material development]. Ankara: Nobel Academy Publishing.