A study to develop a valid and reliable achievement test for the subject of moon phases and eclipses in astronomy education

Abstract

In this study, an achievement test was developed for the subject of the phases of the Moon and eclipses in the astronomy course of the science teaching programme of universities. In order to develop an achievement test, firstly the purpose of the research was determined. Then, the subtopics of the phases of the Moon and eclipses in the science teacher education programme were formed. The questions of the test were prepared according to the sub-topics. The test, which initially consisted of 28 items, was reduced to 23 items due to content validity after expert opinion. For the pilot application, the achievement test was applied to 101 pre-service science teachers from the third and fourth grades who had received astronomy education in Malatya and Elazığ. After the application, item analyses of the test were performed with Microsoft Excel, SPSS and TAP programs. In data analysis, item difficulty and item discrimination indices were analysed. After the analysis, 3 more items were removed from the test and the achievement test was finalised with 20 questions. The KR-20 reliability coefficient of the test was calculated as 0.717. In addition, the average difficulty value of the test was 0.568 and the general discrimination index value was 0.44. From the data obtained, it was concluded that the test was highly reliable and discriminative. The Moon phases and eclipses achievement test can be used to determine the prior knowledge of the students, can be used within the scope of formative assessment activities during the teaching process, and can also be used after teaching.

Keywords

• Astronomy education
• Moon phases and eclipses
• Achievement test
• Science education

Astronomi eğitiminde Ay’ın evreleri ve tutulmalar konusu için geçerli ve güvenilir bir başarı testi geliştirme çalışması

Öz

Özet: Bu çalışmada üniversitelerin fen bilgisi öğretmenliği, astronomi dersinin Ay’ın evreleri ve tutulmalar konusuna yönelik bir başarı testi geliştirilmiştir. Başarı testi geliştirmek için öncelikle araştırmanın amacı belirlenmiştir. Ardından fen bilgisi öğretmenliği öğretim programı Ay’ın evreleri ve tutulmalar konusuna yönelik alt konu başlıkları oluşturulmuştur. Testin soruları alt konu başlıklarına göre hazırlanmıştır. Başlangıçta 28 maddeden oluşan test uzman görüşü sonrası kapsam geçerliği nedeniyle 23 maddeye düşürülmüştür. Pilot uygulama için başarı testi Malatya ve Elazığ’daki astronomi eğitimi almış üçüncü ve dördüncü sınıftan 101 fen bilgisi öğretmen adayına uygulanmıştır. Uygulama sonrası testin madde analizleri Microsoft Excel, SPSS ve TAP programları ile yapılmıştır. Veri analizinde madde güçlük ve madde ayırt edicilik indekslerine bakılmıştır. Analiz sonrasında 3 madde daha testten çıkarılmış ve başarı testi 20 sorudan oluşan son halini almıştır. Testin KR-20 güvenirlik katsayısı 0.717 olarak hesaplanmıştır. Ayrıca testin ortalama güçlük değeri 0.568 ve genel ayırt edicilik indeks değeri 0.44 olarak hesaplanmıştır. Elde edilen verilerden testin yüksek düzeyde güvenilir ve ayırt edici olduğu sonucuna ulaşılmıştır. Ay’ın evreleri ve tutulmalar başarı testi öğrencilerin ön bilgilerini belirlemede, öğretim sürecinde biçimlendirici değerlendirme faaliyetleri kapsamında kullanılabileceği gibi öğretim sonrasında da kullanılabilir.

Anahtar Kelimeler

• Astronomi eğitimi
• Ay’ın evreleri ve tutulmalar
• Başarı testi
• Fen eğitimi

Kaynakça

1. Álvarez, M., Galperin, D., & Quinteros, C. (2018). Indagación de las concepciones de estudiantes primarios y secundarios sobre los fenómenos astronómicos cotidianos. In: En Papini, M., (Ed.), Las Ciencias de la Naturaleza y la Matemática en el Aula: Nuevos Desafíos y Paradigmas. Tandil, Argentina: National University of the Center of the Province of Buenos Aires. p129-142.
2. Bacon, D. R. (2003). Assessing learning outcomes: A comparison of multiple-choice and short-answer questions in a marketing context. Journal of Marketing Education, 25(1), 31-36. https://doi.org/10.1177/0273475302250570
3. Bandalos, D. L. (2018). Measurement theory and applications for the social sciences (1st ed.). Guilford Publications.
4. Baxter, J. (1989). Children’s understanding of familiar astronomical events. International Journal of Science Education, 11(5), 502-513. https://doi.org/10.1080/0950069890110503
5. Bayraktar, S. (2009). Pre-service primary teachers’ ideas about lunar phases. Journal of Turkish Science Education, 6(2), 12-23.
6. Bisard, W., Aron, R., Francek, M., & Nelson, B. (1994). Assessing selected physical science and earth science misconceptions of middle school through university preservice teachers. Journal of College Science Teaching, 24, 38-42.
7. Black, P., Harrison, C., Lee, C., Marshal, B., & Wiliam, D. (2003). Assessment for learning: Putting it into practice. McGraw-Hill Education.
8. Brown, J. D. (2005). Testing in language programs: A comprehensive guide to English language assessment. Prentice-Hall.

9. Brualdi Timmins, A. C. (1998). Classroom questions. Practical Assessment, Research, and Evaluation, 6(1). 1-3. https://doi.org/10.7275/05rc-jd18
10. Büyüköztürk, Ş. (2014). Sosyal bilimler için veri analizi el kitabı (19. baskı). Pegem Akademi.
11. Callison, P., & Wright, E. (1993). The effect of teaching strategies using models on pre-service elementary teachers’ conceptions about Earth- Sun-Moon relationships. Annual Meeting of the National Association for Research in Science Teaching, Atlanta, Georgia.
12. Camino, N. (1995). Ideas previas y cambio conceptual en astronomía. Preconceptions and conceptual change in astronomy. A study with primary school teachers about day and night, seasons and lunar phases. Enseñanza de las Ciencias, 13(1), 81-96.
13. Comins, N. F. (2001). Heavenly errors: Misconceptions about the real nature of the universe. New York: Columbia University Press.
14. Creswell, J. W., & Creswell, J. D. (2014). Research desing: qualitative, quantitative and mixed methods approaches (Vol. 54). United State of America: Sage Publications.
15. Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Holt, Rinehart and Winston, 6277 Sea Harbor Drive, Orlando, FL 32887.
16. Currie, M., & Chiramanee, T. (2010). The effect of the multiple-choice item format on the measurement of knowledge of language structure. Language Testing, 27(4), 471-491. https://doi.org/10.1177/0265532209356790
17. Danaia, L., & McKinnon, D. H. (2007). Common alternative astronomical conceptions encountered in junior secondary science classes: Why is this so? Astronomy Education Review, 2(6), 32-53. https://doi.org/10.3847/AER2007017
18. Demirel, F. G. (2007). İlköğretim 5. sınıf fen ve teknoloji dersinin “Dünya, Güneş ve Ay” ünitesinde işbirlikli öğrenme yönteminin öğrenci başarılarına ve derse olan tutumlarına etkisi. Yayımlanmamış Yüksek Lisans Tezi, Gazi Üniversitesi, Eğitim Bilimleri Enstitüsü, Ankara.
19. Dibattista, D., & Kurzawa, L. (2011). Examination of the quality of multiple-choice items on classroom tests. Canadian Journal for the Scholarship of Teaching and Learning, 2(2), 4. https://doi.org/10.5206/cjsotl-rcacea.2011.2.4.
20. Dicovskiy, E., Iglesias, M., Karaseur, F., Gangui, A., Cabrera, J., & Godoy, E. (2012). El problema de la posición del observador y el movimiento tridimensional en la explicación de las fases de la Luna en docentes de primaria en formación. In: Actas de las III Jornadas de Enseñanza e Investigación Educativa en el campo de las Ciencias Exactas y Naturales. La Plata, Argentina: National University of La Plata. p219-230.
21. Douglas, M., Wilson, J., & Ennis, S. (2012). Multiple-choice question tests: a convenient, flexible and effective learning tool? A case study. Innovations in Education and Teaching International, 49(2), 111-121. https://doi.org/10.1080/14703297.2012.677596
22. Düşkün, İ. (2011). Güneş-Dünya-Ay modeli geliştirilmesi ve fen bilgisi öğretmen adaylarının astronomi eğitimindeki akademik başarılarına etkisi. Yayımlanmamış Yüksek Lisans Tezi, İnönü Üniversitesi, Eğitim Bilimleri Enstitüsü, Malatya.
23. Erdoğan, S. (2010). Dünya, Güneş ve Ay konusunun ilköğretim 5. sınıf öğrencilerine bilimsel tartışma odaklı yöntem ile öğretilmesinin öğrencilerin başarılarına, tutumlarına ve tartışmaya katılma istekleri üzerine etkisinin incelenmesi. Yayımlanmamış Yüksek Lisans Tezi. Uşak Üniversitesi, Sosyal Bilimler Enstitüsü, Uşak.
24. Fraenkel, J. K., & Wallen, N. E. (1996). How to design and evaluate research in education (3. edition). New York: McGraw-Hill Inc.
25. Galperin, D., Prieto, L., & Heredia, L. (2018). Concepciones de docentes sobre las causas de los fenómenos astronómicos cotidianos. In: Papini, M., (Ed.), Las Ciencias de la Naturaleza y la Matemática en el Aula: Nuevos Desafíos y Paradigmas. Tandil: Universidad Nacional del Centro de la Provincia de Buenos Aires. p116-128.
26. Galperin, D., & Raviolo, A. (2019). Reference frames and astronomy teaching: the development of a topocentric approach to the lunar phases. Science Education International, 30(1), 28-37.
27. Glass, L. A., & Sinha, N. (2013). Multiple-choice questioning is an efficient instructional methodology that may be widely implemented in academic courses to improve exam performance. Current Directions in Psychological Science, 22(6), 471-477. https://doi.org/10.1177/0963721413495870
28. Golvardi Yazdi, M., Haghighat Shoar, M., Sobhani, Gh., Vafi Sani, F., Khoshkholgh, R., Mousavi Bazaz, N., & Mansourzadeh, A. (2021). Factors affecting students’ guesswork in multiple-choice questions and corrective strategies. Medical Education Bulletin, 2(4), 341-349. https://doi.org/10.22034/MEB.2021.312176.1032
29. Güler, M. (2017). The effect of goal orientation on student achievement. In E. Karadag (Ed.), The factors effecting student achievement (pp. 291–307). Springer. https://doi.org/10.1007/978-3-319-56083-0_18.
30. Haladyna, T. M. (2004). Developing and validating multiple-choice test items. Routledge. Ibbett, L. N., & Wheldon, J. B. (2016). The incidence of clueing in multiple choice test bank questions in accounting: Some evidence from Australia. e-Journal of Business Education and Scholarship of Teaching, 10(1), 20-35.
31. Kan, A. (2009). Ölçme araçlarında bulunması gereken nitelikler, Hakan Atılgan (Ed.). Eğitimde ölçme ve değerlendirme. Anı Yayıncılık.
32. Karaman, P. (2020). Temel kavramlar. İçinde F. Orçan (Ed.), Sosyal bilimlerde istatistik Excel ve SPSS uygulamaları (ss. 27-52). Ankara: Anı Yayıncılık.
33. Kanlı, U. (2014). A study on identifying the misconceptions of pre-service and in-service teachers about basic astronomy concepts. Eurasia Journal of Mathematics, Science and Technology Education, 10(5), 471–479. https://doi.org/10.12973/eurasia.2014.1120a
34. Karamustafaoğlu, S., & Tutar, M. (2016). 6. sınıf Dünya’mız, Ay ve yaşam kaynağımız Güneş ünitesine yönelik bir başarı testi geliştirme. Eğitim Bilimlerinde Yenilikler ve Nitelik Arayışı (ss. 303-320), Ö. Demirel ve S. Dinçer (Ed.), Pegem Akademi Yayınları, Ankara.
35. Little, J. L., & Bjork, E. L. (2015). Optimizing multiple-choice tests as tools for learning. Memory & Cognition, 43(1), 14–26. https://doi.org/10.3758/s13421-014-0452-8
36. Moore, P. (2000). The data book of astronomy. Institute of Physics Publishing.
37. Navarro Pastor, M. (2009). Aprendizaje y enseñanza de astronomía diurna en primaria (Doctoral dissertation, Universitat d'Alacant/Universidad de Alicante).
38. Netemeyer, R. G., Bearden, W. O., & Sharma, S. (2003). Scaling Procedures: Issues and Applications. Sage Publications, Inc Thousand Oaks, California.
39. Nicol, D. (2007). E-assessment by design: Using multiple-choice tests to good effect. Journal of Further and Higher Education, 31(1), 53-64. https://doi.org/10.1080/03098770601167922.
40. Polat, M. (2020). Analysis of multiple-choice versus open-ended questions in language tests according to different cognitive domain levels. Novitas-ROYAL (Research on Youth and Language), 14(2), 76-96.
41. Scott, M., Stelzer, T., & Gladding, G. (2006). Evaluating multiple-choice exams in large introductory physics courses. Physical Review Special Topics-physics Education Research, 2(2), 020102. https://doi.org/10.1103/PhysRevSTPER.2.020102
42. Shuttleworth, M. (2016). Content validity. https://explorable.com/contentvalidity.html adresinden 23.08.2022 tarihinde elde edilmiştir.
43. Slepkov, A. D., & Godfrey, A. T. K. (2019). Partial credit in answer-until-correct multiple-choice tests deployed in a classroom setting. Applied Measurement in Education, 32(2), 138–150. https://doi.org/10.1080/08957347.2019.1577249.
44. Sontay, G., & Karamustafaoğlu, O. (2020). Fen bilimleri dersi “Güneş, Dünya ve Ay” ünitesine yönelik başarı testinin geliştirilmesi. Gazi Üniversitesi Gazi Eğitim Fakültesi Dergisi, 40(2), 511-551.
45. Stahly, L., Krockover, G., & Shepardson, D. (1999). Third grade students’ ideas about the lunar phases. Journal of Research in Science Teaching, 36, 159-177. https://doi.org/10.1002/(SICI)1098-2736(199902)36:2%3C159::AID-TEA4%3E3.0.CO;2-Y
46. Suskie, L. (2018). Assessing student learning: A common sense guide. John Wiley. Suzuki, M. (2003). Conversations about the moon with prospective teachers in Japan. Science Education, 87(6), 892-910. https://doi.org/10.1002/sce.10082
47. Taşcan, M., & Ünal, İ. (2020). Fen Bilgisi Öğretmenlerine Göre Ay’ın Hareketleri ve Evreleri ile Güneş, Dünya, Ay Konularının Öğretimi. Uluslararası Eğitim Bilim ve Teknoloji Dergisi, 6(2), 75-97. https://doi.org/10.47714/uebt.738066
48. Tobias, S., & Raphael, J. B. (1996). In-class examinations in college-level science: New theory, new practice. Journal of Science Education and Technology, 5(4), 311–320. https://doi.org/10.1007/BF01677127
49. Trumper, R. (2001). Assessing students’ basic astronomy conceptions from junior high school through university. Australian Science Teachers Journal, 47(1), 21-31.
50. Wells, C. S., & Wollack, J. A. (2003). An instructor’s guide to understanding test reliability. Testing and Evaluation Services, University of Wisconsin.
51. Wendler, C. L., & Walker, M. E. (2006). Practical issues in designing and maintaining multiple test forms for large-scale programs. In Downing, S. M., & Haladyna, T. M. (Eds.), Handbook of test development (pp. 445-467). Lawrence Erlbaum Associates Publishers.