The Assessment of the Fifth-Grade Students’ Science Critical Thinking Skills through Design-Based STEM Education

Abstract

Critical thinking has been one of the 21st-century skills consistently associated with students’ future career advancement as a positive student outcome of STEM education. The aim of the study is to develop and validate science critical thinking skill instruments to assess the improvement in the subject of living organisms and force and friction through design-based STEM education. In this design-based research study, the student’s modules were developed by the integrated STEM education principles involving the activities and worksheets in line with the frame of critical thinking approach. The kappa statistics for content validity, exploratory and confirmatory factor analysis for construct validity, and item and reliability analysis for the quality of items were used in the development stage of instruments. The results of these analyses endorsed the 15 two-tier item for each test of Living Things Critical Thinking (LTCT) and Measuring Force and Friction Critical Thinking (MFFCT) as unidimensional constructs to produce valid and reliable data to measure the fifth grade students' critical thinking skills in the related science content. Comparing the pre and post applications of instruments in the study group indicated that STEM modules improved the students’ science critical thinking skills such as interpretation, analysis, and inference. In this respect, developing and validating instruments to assess the integrated critical thinking skills will contribute to the empirical examination of this construct within the context of school science learning.

Keywords

• STEM education
• Science critical thinking
• Design-based
• Living things

References

1. Abdallah, M. M. S. & Wegerif, R. B. (2014). Design-based research (DBR) in educational enquiry and technological studies: A version for PhD students targeting the integration of new technologies and literacies into educational contexts. ERIC: ED546471. Retrieved 3, 2019, from http://files.eric.ed.gov/fulltext/ED546471.pdf
2. Arıcak, O. T., Avcu, A., Topçu, F., & Tutlu, M. G. (2020). Use of item response theory to validate cyberbullying sensibility scale for university students. International Journal of Assessment Tools in Education, 7(1), 18-29. Retrieved October 3, 2020, from https://dx.doi.org/10.21449/ijate.629584
3. Asysyifa, D.S., Jumadi, Wilujeng, I., & Kuswanto, H. (2019). Analysis of students critical thinking skills using partial credit models (PCM) in physics learning. International Journal of Educational Research Review, 4(2), 245 253. https://doi.org/10.24331/ijere.518068
4. Awang, Z. (2012). A handbook on structural equation modeling using AMOS (6th Ed). Universiti Teknologi Mara Press: Malaysia.
5. Baharin, N., Kamarudin, N., & Manaf, U. K. A. (2018). Integrating STEM education approach in enhancing higher order thinking skills. International Journal of Academic Research in Business and Social Sciences, 8(7), 810–822. Retrieved February 3, 2019, from http://dx.doi.org/10.6007/IJARBSS/v8-i7/4421
6. Bailin, S. (2002). Critical thinking and science education. Science & Education, 11(4), 361-375. Retrieved February 3, 2019, from http://dx.doi.org/10.1023/A:1016042608621
7. Beyer, B. K. (1995). Critical thinking. Bloomington, IN: Phi Delta Kappa Educational Foundation.
8. Brooks, G. P. & Johanson, G. A. (2003). TAP: Test analysis program. Applied Psychological Measurement, 27(4), 303-304.

9. Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). New York, NY: Guilford Press.
10. Bursal, M. (2017). SPSS ile temel veri analizleri. Ankara: Anı Yayıncılık.
11. Büyüköztürk, Ş. (2012). Sosyal bilimler için veri analizi el kitabı [Data analysis handbook for social sciences]. Ankara: Pegem Akademi.
12. Coaley, K. (2010). An introduction to psychological assessment and psychometrics. London: Sage Publications.
13. Crocker, L. & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart and Winston.
14. Çorlu, M. S., Capraro, R. M. & Capraro, M. M. (2014). Introducing STEM education: Implications for educating our teachers for the age of innovation. Education and Science, 39(171), 74-85.
15. Deng, N., Wells, C., & Hambleton, R. (2008). A confirmatory factor analytic study examining the dimensionality of educational achievement tests. NERA Conference Proceedings 2008. 31. Retrieved January 10, 2020, from https://opencommons.uconn.edu/nera_2008/31
16. Dilekli, Y. (2019). Etkinliklerle düşünme eğitimi. Ankara: Anı Yayıncılık.
17. Doğan, N. (2019). Eğitimde ölçme ve değerlendirme [Measuremnet and evaluation in education]. Ankara Pegem Akademi.
18. Doğan, H. (2020). Design, implementation, and evaluation of the fifth grade science course units with an integrated STEM education approach (Unpublished doctoral dissertation). Pamukkale University, Denizli, Turkey.
19. Duran, M., & Şendağ, S. (2012). A preliminary investigation into critical thinking skills of urban high school students: Role of an IT/STEM program. Creative Education, 3(2), 241–250. Retrieved February 3, 2019, from http://dx.doi.org/10.4236/ce.2012.32038
20. Ebel, R. L. & Frisbie, D. A. (1991). Essentials of educational measurement (5th ed.). Englewood Cliffs: Prentice-Hall.
21. Ennis, R. (1989). Critical thinking and subject specificity: Clarification and needed research. Educational Researcher, 18(3), 4 10. Retrieved March, 3, 2017 from https://doi.org/10.3102/0013189X018003004
22. Ergün, A. & Külekci, E. (2019). The effect of problem based STEM education on the perception of 5th grade students of engineering, engineers and technology. Pedagogical Research, 4(3), em0037. Retrieved March, 3, 2020 from https://doi.org/10.29333/pr/5842
23. Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction. Research findings and recommendations (ERIC Document Reproduction Service No. ED315423). Retrieved March, 3, 2017, from https://eric.ed.gov/?id=ED315423
24. Ferrando, P. J. & Lorenzo-Seva, U. (2017). Assessing the quality and appropriateness of factor solutions and factor score estimates in exploratory item factor analysis. Educational and Psychological Measurement, 1–19. https://doi.org/10.1177/0013164417719308
25. Fraenkel, J.R. & Wallen, N.E. (2008). How to design and evaluate research in education (7th ed.). New York: McGraw-Hill.
26. Gencer, A. S. & Boran, G. H. (2017). Üst düzey düşünme becerilerinin öğretimi [Teaching higher order thinking skills]. In S. Dal & M. Köse (Ed), Öğretim ilke ve yöntemleri (pp. 405-445). Ankara: Anı Yayıncılık
27. Griffard, P. B., & Wandersee, J. H. (2001). The two-tier instrument on photosynthesis: what does it diagnose? International Journal of Science Education, 23(10), 1039-1052. Retrieved March, 3, 2017, from https://doi.org/10.1080/09500690110038549
28. Hair, J. F., Black, W. C, Babin, B.J., & Anderson, R. E. (2009). Multivariate data analysis (7th ed.). Upper Saddle River, NJ: Pearson Prentice Hall.
29. Haladyna, T. M. (1994). Developing validating multiple choice test items. Lawrence Erlbaum Associates, Publishers.
30. Harjo, B., Kartowagiran, B., & Mahmudi, A. (2019). Development of critical thinking skill instruments on mathematical learning high school. International Journal of Instruction, 12(4), 149 166. Retrieved January 10, 2020, from https://doi.org/10.29333/iji.2019.12410a
31. Hattie, J. (1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9, 139–64.
32. Honey, M., Pearson, G., & Schweingruber, H. (2014). STEM integration in K-12 education: Status, prospects, and an agenda for research. Washington, DC: National Academies Press.
33. Kline, B. R. (2015). Principles and practice of structural equation modeling (4th ed.). New York, NY: Guilford Press.
34. Kuş, M. & Çakıroğlu, E., (2020). Prospective mathematics teachers‟ critical thinking processes about scientific research: Newspaper article example. Turkish Journal of Education, 9(1), 22-45. https://doi.org/10.19128/turje.605456
35. Mabruroh, F. & Suhandi, A. (2017). Construction of critical thinking skills test instrument related the concept on sound wave. IOP Conference Series: Journal of Physics: Conf. Series 812 (2017) 012056 https://doi.org/10.1088/1742-6596/812/1/012056
36. Mapeala, R. & Siew, N. M. (2015). The development and validation of a test of science critical thinking for fifth graders. Springer Plus, 4(741). DOI 10.1186/s40064-015-1535-0
37. Massachusetts Department of Education. (2006). Massachusetts science and technology/engineering curriculum framework. Retrieved January 10, 2019, from http://www.doe.mass.edu/frameworks/scitech/1006.do
38. Ministry of National Education. (2018). Elementary and middle school (3, 4, 5, 6, 7, and 8th grades) science curriculum. Ankara: Board of Education and Training.
39. Mutakinati, L., Anwari, I., & Yoshisuke, K. (2018). Analysis of students’ critical thinking skill of middle school through stem education project-based learning. Journal Pendidikan IPA Indonesia, 7(1), 54 65. Retrieved February 3, 2019, from http://journal.unnes.ac.id/index.php/jpii https://doi.org/10.15294/jpii.v7i1.10495 NGSS Lead States. (2013). Next generation science standards: For states by states. Washington, DC: The National Academies Press.
40. National Research Council. (2011). Assessing 21st Century Skills: Summary of a Workshop. Washington, DC: The National Academies Press.
41. Nunnally, J.C. & Bernstein, I.H. (1994). Psychometric Theory (3rd ed.). New York, NY: McGraw-Hill, Inc.
42. Öner, A. T., Navruz, B., Biçer, A., Peterson, C. A., Capraro, R.M., & Capraro, M.M. (2014). T-STEM academies‟ academic performance examination by education service centers: A Longitudinal Study. Turkish Journal of Education, 3(4), 40-51.
43. Oonsim., W. & Chanprasert, K. (2017). Developing critical thinking skills of grade 11 students by STEM education: Focus on electrostatic in physics. Rangsit Journal of Educational Studies, 4 (1), 54-59.
44. Patton, M.Q. (2014). Qualitative research and evaluation methods: Integrating theory and practice. Thousand Oaks, CA: Sage Publications.
45. Polit, D. & Hungler, B., P. (1999). Nursing research: Principles and methods. Philadelphia: Lippincott Company.
46. Polit, D. F., Beck, C. T., & Owen, S. V. (2007). Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health, 30(4), 459-467.
47. Rahmawati, Y., Ridwan, A., Hadinugrahaningsih, T., & Soeprijanto (2019). Developing critical and creative thinking skills through STEAM integration in chemistry learning. IOP Conference. Series: Journal of Physics: Conf. Series 1156 (2019) 012033. https://doi.org/10.1088/1742-6596/1156/1/012033
48. Reynders, G., Lantz, J., Ruder, S. M., Stanford, C. L., & Cole, R. S. (2020). Rubrics to assess critical thinking and information processing in undergraduate STEM courses. International Journal of STEM Education, 7(9). Retrieved February 3, 2019, from https://doi.org/10.1186/s40594-020. 00208-5
49. Sadhu, S. & Laksono, E. W. (2018). Development and validation of an integrated assessment for measuring critical thinking and chemical literacy in chemical Equilibrium. International Journal of Instruction, 11(3), 557-572. Retrieved February 3, 2019, from https://doi.org/10.12973/iji.2018.11338a
50. Stevens, J. P. (2002). Applied multivariate statistics for the social sciences. New Jersey: Lawrance Erlbaum Association, Inc.
51. Swartz, R. J., Costa, A. L., Beyer, B. K., Reagan, R., & Kallick, B. (2008). Thinking-based learning: Promoting quality student achievement in the 21st century. New York, NY: Teachers College Press.
52. Sya'bandari, Y., Firman, H., & Rusyat, L. (2017). The development and validation of science virtual test to assess 7th grade students' critical thinking on matter and heat topic. Journal of Science Learning, 1(1), 17-27.
53. Timmerman, M. E. & Lorenzo-Seva, U. (2011). Dimensionality assessment of ordered polytomous items with parallel analysis. Psychological Methods, 16(2), 209–220. Retrieved February 3, 2019, from https://doi.org/10.1037/a0023353
54. Tabachnick, B. G. & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Boston: Allyn and Bacon.
55. Waddell, B. (2019). Influence of STEM lessons on critical thinking (Unpublished master’s thesis). The Graduate College at the University of Nebraska, Lincoln. Retrieved February 3, 2019, from https://digitalcommons.unl.edu/teachlearnstudent/103
56. Wang, H. H., Moore, T. J., Roehrig, G. H., & Park, M. S. (2011). STEM integration: Teacher perceptions and practice. Journal of Pre-College Engineering Education Research, 1(2), 1-13. Retrieved February 3, 2019, from https://doi.org/10.5703/1288284314636
57. Whiston, S. C. (2012). Principles and applications of assessment in counseling (4th ed.). Belmont, CA: Brooks/Cole, Cengage Learning.
58. Wicaksana, Y. D., Widoretno, S., & Dwiastuti, S. (2020). The use of critical thinking aspects on module to enhance students’ academic achievement. International Journal of Instruction, 13(2), 303 314. Retrieved February 3, 2019, from https://doi.org/10.29333/iji.2020.13221a
59. Wiersma, W., & Jurs, S. G. (2005). Research methods in education: An introduction (8th ed.). Boston: Pearson/A and B.
60. Willingham, D. T. (2008). Critical thinking: Why is it so hard to teach? Arts Education Policy Review, 109(4), 21 32. Retrieved February 3, 2019, from https://doi.org/10.3200/AEPR.109.4.21-32
61. Yurdugül, H. & Bayrak, F. (2012). Content validity measures in scale development studies: Comparison of content validity index and kappa statics. Hacettepe University Journal of Education, Special Issue 2, 264-271.
62. Yalaki Y., Doğan, N., İrez, S., Doğan, N., Çakmakçı, G., Kara, B. E. (2019). Measuring nature of science views of middle school students. International Journal of Assessment Tools in Education, 6(3), 461 475. Retrieved October 3, 2019, from https://dx.doi.org/10.21449/ijate.561154