Examining Non-Science Majors' Knowledge of Scientific Practices in Evaluating Scientific Media Claims

Year 2025, Volume: 12 Issue: 1, 1 - 20

Filiz Bezci , Semra Sungur

Abstract

This study examines the knowledge of scientific practices that non-science undergraduate students need to critically evaluate scientific claims in the media. Employing a cross-sectional quantitative research design, the study gathered data through a self-report instrument involving 266 undergraduate students from non-science disciplines. A twelve-item, two-tier multiple-choice assessment was utilized, featuring questions from two research articles adapted from popular media. Findings reveal that non-science majors often struggle to understand the scientific practices necessary to evaluate scientific reports in mainstream media. Results also indicate that many participants fail to recognize the importance of controlled, randomized experimental designs in establishing cause-and-effect relationships. Additionally, findings suggest that students do not fully grasp that hypotheses are supported by evidence rather than proven and that scientific claims must be based on substantial proof. The study further shows that many participants underestimate the importance of peer review in validating scientific claims. Finally, the analysis revealed no significant gender differences in students' competencies related to critiquing these reports.

Keywords

Evaluating scientific claims, Non-science majors, Popular media, Gender

References

• Abd-El-Khalick, F., Bell, R. L., & Lederman, N. G. (2004). Inquiry as context for the nature of science: Developing a framework for classroom practice. Science Education, 88(4), 397–419. doi:10.1002/sce.10118
• Ahmed, S., & Rasul, M. E. (2022). Social media news use and COVID-19 misinformation engagement: survey study. Journal of Medical Internet Research, 24(9), e38944. doi:10.2196/38944
• Aldridge, J., & Goldman, R. (2002). Gender equity and education. In S. Dragin (Ed.), Current issues and trends in education (pp. 163–170). Boston: Allyn and Bacon.
• Blickenstaff, J. C. (2005). Women and science careers: Leaky pipeline or gender filter? Gender and Education, 17(4), 369–386. doi:10.1080/09540250500145072
• Busch, K. C., & Rajwade, A. (2024). Conceptualizing community scientific literacy: Results from a systematic literature review and a Delphi method survey of experts. Science Education, 108, 1231–1268. doi:10.1002/sce.21871
• Bybee, R. W. (2011). Scientific and engineering practices in K-12 classrooms: Understanding A Framework for K-12 Science Education. The Science Teacher, 78(9), 34–40.
• Chalmers, A. F. (1999). What is this thing called science? (3rd ed.). Indianapolis, IN: Hackett Publishing Company.
• Carli, L.L., Alawa, L., Lee, Y., Zhao, B., & Kim, E. (2016). Stereotypes about gender and science: Women ≠ scientists, Psychology of Women Quarterly, 40(2), 244–260. doi:10.1177/0361684315622645
• Cavagnetto, A. R., & Hand, B. (2012). The importance of embedding argument within science classrooms. In M. S. Khine (Ed.), Perspectives on scientific argumentation: Theory, practice and research (pp. 39–53). Springer. doi:10.1007/978-94-007-2470-9
• Cheryan, S., Ziegler, S. A., Montoya, A. K., & Jiang, L. (2017). Why are some STEM fields more gender-balanced than others? Psychological Bulletin, 143(1), 1–35. doi:10.1037/bul0000052
• Cinelli, M., Quattrociocchi, W., Galeazzi, A., Valensise, C. M., Brugnoli, E., Schmidt, A. L., Zola, P., Zollo, F., & Scala, A. (2020). The COVID-19 social media infodemic. Scientific Reports, 10(1), 16598. doi:10.1038/s41598-020-73510-5
• Dindar, H., & Yangın, S. (2007). Teachers’ perceptions about the transition process to elementary school science and technology teaching curriculum. Kastamonu Education Journal, 15(1), 185-198.
• Driver, R., Leach, J., Millar, R., & Scott, P. (1996). Young people's images of science. Buckingham: Open University Press.
• Donley, D. W. (2024). Teaching the nature of science improves scientific literacy among students not majoring in STEM. Journal of Undergraduate Neuroscience Education, 22(2), A152-A157. doi:10.59390/HRWL6927
• Duschl, R. A., & Bybee, R. W. (2014). Planning and carrying out investigations: An entry to learning and to teacher professional development around NGSS science and engineering practices. International Journal of STEM Education, 1(12), 1-9. doi:10.1186/s40594-014-0012-6
• Duschl, R. A., & Grandy, R. (2013). Two views about explicitly teaching nature of science. Science & Education, 22(9), 2109–2139. doi:10.1007/s11191-012-9539-4
• Eccles, J. (1987). Gender roles and women's achievement-related decisions. Psychology of Women Quarterly, 11(2),135–172.
• Erduran, S., & Dagher, Z. R. (2014). Reconceptualizing the nature of science for science education: Scientific knowledge, practices and other family categories. Dordrecht, Netherlands: Springer. doi:10.1007/978-94-017-9057-4
• Farhoudinia, B., Ozturkcan, S., & Kasap, N. (2024). Emotions unveiled: detecting COVID-19 fake news on social media. Humanities and Social Sciences Communications, 11(1), 1-11. doi:10.1057/s41599-024-03083-5
• Gabarron, E., Oyeyemi, S. O., & Wynn, R. (2021). COVID-19-related misinformation on social media: a systematic review. Bulletin of the World Health Organization, 99(6), 455-463. doi:10.2471/BLT.20.276782
• Gisondi, M. A., Barber, R., Faust, J. S., Raja, A., Strehlow, M. C., Westafer, L. M., & Gottlieb, M. (2022). A deadly infodemic: social media and the power of COVID-19 misinformation. Journal of medical Internet research, 24(2), e35552. doi:10.2196/35552
• Gökçe, İ. (2006). Evaluation of the İnner Valume of the Science and Technology Curriculum and Teacher’s Guide and the Problems Faced in the Practice (Balıkesir Example). (Unpublished master’s thesis). Balikesir University, Balikesir.
• Grandy, R.E. (1995). Expertise and the critical nonexpert (comment on Norris). Science Education, 79(2), 219–221.
• Guzzetti, B. J., & Wilhams, W. O. (1996). Changing the pattern of gendered discussion: Lessons from science classrooms. Journal of Adolescent and Adult Literacy, 40(1), 38-47.
• Hackett, G., & Betz, N. E. (1981). A self-efficacy approach to the career development of women. Journal of Vocational Behavior, 18(3), 326-339. doi:10.1016/0001-8791(81)90019-1
• Hanson, N. R. (1958). Patterns of discovery: An inquiry into the conceptual foundations of science. New York, NY: Cambridge University Press.
• Happer, C., & Philo, G. (2013). The role of the media in the construction of public belief and social change. Journal of Social and Political Psychology, 1(1), 321-336. doi:10.5964/jspp.v1i1.96
• Huang, C. (2013). Gender differences in academic self-efficacy: A meta-analysis. European Journal of Psychology of Education, 28, 1-35. doi: 10.1007/s10212-011-0097-y
• Irzik, G., & Nola, R. (2011). A family resemblance approach to the nature of science for science education. Science & Education, 20(7), 591–607. doi:10.1007/s11191-010-9293-4
• Korpan, C.A., Bisanz, G.L., Bisanz, J., & Henderson, J.M. (1997). Assessing literacy in science: Evaluating of scientific new briefs. Science Education, 81(5), 515-532.
• Krajcik, J. S., McNeill, K. L., & Reiser, B. J. (2008). Learning-goals-driven design model: Developing curriculum materials that align with national standards and incorporate project-based pedagogy. Science Education, 92(1), 1–32. doi:10.1002/sce.20240
• Kuhn, T. S. (1996). The structure of scientific revolutions (3rd ed.). Chicago: University of Chicago Press.doi:10.7208/chicago/9780226458144.001.0001
• Lederman, N. G. (1992). Students’ and teachers’ conceptions of the nature of science: A review of the research. Journal of Research in Science Teaching, 29(4), 331–359. doi:10.1002/tea.3660290404
• Lederman, N. G., Antink, A., & Bartos, S. (2014). Meaningful assessment of learners' understandings about scientific inquiry—The views about scientific inquiry (VASI) questionnaire. Journal of Research in Science Teaching, 51(1), 65-83. doi:10.1002/tea.21125
• Lederman, N. G., & Druger, M. (1985). Classroom factors related to changes in students' conceptions of the nature of science. Journal of Research in Science Teaching, 22(7), 649–662. doi: 10.1002/tea.3660220705
• Leung, J.S.C., Wong, A.S.L., & Yung, B.H.W. (2017). Evaluation of science in the media by non-science majors. International Journal of Science Education, Part B, 7(3), 219-236. doi:10.1080/21548455.2016.1206983
• Martin-Dunlop, C.S. (2013). Prospective elementary teachers' understanding of the nature of science and perceptions of the classroom learning environment. Research in Science Education, 43(3), 873. doi:10.1007/s11165-012-9290-5
• Martins, N., Weaver, A. J., & Lynch, T. (2018). What the public “knows” about media effects research: The influence of news frames on perceived credibility and belief change. Journal of Communication, 68(1), 98-119. doi:10.1080/21548455.2016.1206983
• Matthews, M.R. (2015). Science Teaching: The Contribution of History and Philosophy of Science (20th Anniversary Revised and Expanded Edition). New York: Routledge.
• McComas, W. F. (1998). The principal elements of the nature of science: Dispelling the myths. In W. F. McComas (Ed.), The nature of science in science education: Rationales and strategies (pp. 53–70). Dordrecht: Kluwer Academic Publishers.
• McNeill, K. L., & Krajcik, J. (2008). Scientific explanations: Characterizing and evaluating the effects of teachers' instructional practices on student learning. Journal of Research in Science Teaching, 45(1), 53-78. doi:10.1002/tea.20201
• Millar, R. (2006). Twenty-first-century science: Insights from the design and implementation of a scientific literacy approach in school science. International Journal of Science Education, 28(13), 1499–1521. doi:10.1080/09500690600718344
• Moss, D.M. (2001). Examining student conceptions of the nature of science. International Journal of Science Education, 23(8), 771-790. doi:10.1080/09500690010016030
• National Research Council. (2007). Taking Science to School: Learning and Teaching Science in Grades K-8. Washington, DC: The National Academies Press. doi:10.17226/11625
• National Research Council. (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. Washington, DC: The National Academies Press. doi:10.17226/13165
• Next Generation Science Standards Lead States. (2013). Next Generation Science Standards: For states, by states. Washington, DC: The National Academies Press. doi:10.17226/18290
• Nisbett, R.E. (1993). Rules for reasoning. Hillsdale, NJ: Lawrence Erlbaum.
• Norris, S. P., Phillips, L. M., & Korpan, C. A. (2003). University students' interpretation of media reports of science and its relationship to background knowledge, interest, and reading difficulty. Public Understanding of Science, 12(2), 123–145. doi:10.1177/09636625030122
• Okasha, S. (2002). Philosophy of science: A very short introduction. New York: Oxford University Press. doi:10.1093/actrade/9780198745587.001.0001
• Osborne, J. (2014). Teaching scientific practices: Meeting the challenge of change. Journal of Science Teacher Education, 25(2), 177–196. doi:10.1007/s10972-014-9384-1
• Osborne, J., Collins, S., Ratcliffe, M., Millar, R., & Duschl, R. (2003). What “ideas-about-science” should be taught in school science? A Delphi study of the expert community. Journal of Research in Science Teaching, 40(7), 692–720. doi:10.1002/tea.10105
• Osborne, J., & Dillon, J. (2008). Science education in Europe: Critical reflections. London, UK: Nuffield Foundation.
• Pellechia, M.G. (1997). Trends in science coverage: A content analysis of three US newspapers, Public Understanding of Science, 6(1), 49–68. doi:10.1088/0963-6625/6/1/004
• Riegle-Crumb, C., Moore, C., & Ramos-Wada, A. (2010). Who wants to have a career in science or math? Exploring adolescents' future aspirations by gender and race/ethnicity. Science Education, 96(4), 548–566. doi:10.1002/sce.20431
• Rosenthal, S. (2020). Media literacy, scientific literacy, and science videos on the internet. Frontiers in Communication, 5, 581585. doi:10.3389/fcomm.2020.581585
• Rudolph, J. L. (2024). Scientific literacy: Its real origin story and functional role in American education. Journal of Research in Science Teaching, 61(3), 519-532. doi:10.1002/tea.21890
• Sağdıç, A. & Yiğit, G. (2025). Unlocking the science in English coursebooks: A critical exploration of scientific literacy. Bartın University Journal of Faculty of Education, 14(1), 42-54. doi:10.14686/buefad.1284580
• Schneider, B., Milesi, C., Perez-Felkner, L., Brown, K., & Gutin, I. (2015). Does the gender gap in STEM majors vary by field and institutional selectivity? Teachers College Record, 117(1), 1–9.
• Shapin, S. (1995). A social history of truth: Civility and science in seventeenth-century England. London: University of Chicago Press.
• Sjöström, J. (2024). Vision III of scientific literacy and science education: an alternative vision for science education emphasising the ethico-socio-political and relational-existential. Studies in Science Education, 1-36. doi:10.1080/03057267.2024.2405229
• Shearmur, J. (2006). Karl Popper: The logic of scientific discovery. In J. Shand (Ed.), Central works of philosophy v4: Twentieth century: Moore to Popper (pp. 262-286). Newyork, NY:Routledge. doi:10.4324/9781315478937
• Strimaitis, A. M., Schellinger, J., Jones, A., Grooms, J., & Sampson, V. (2014). Development of an instrument to assess student knowledge necessary to critically evaluate scientific claims in the popular media. Journal of College Science Teaching, 43(5), 55-68.
• Tindall, T. & Hamil, B. (2004). Gender disparity in science education: The causes consequences and solutions. Education, 125(2), 282-296.
• Tytler, R. (2007). Re-imagining science education: Engaging students in science for Australia's future. Melbourne, Australia: Australian Council for Educational Research.
• Wang, M. T., & Degol, J. L. (2017). Gender gap in science, technology, engineering, and mathematics (STEM): Current knowledge, implications for practice, policy, and future directions. Educational Psychology Review, 29(1), 119–140. doi:10.1007/s10648-015-9355-x
• Wellington, J. (1991). Newspaper science, school science: Friends or enemies. International Journal of Science Education, 13(4), 363–372. doi:10.1080/0950069910130401
• Williams, M. M., & George-Jackson, C. (2014). Using and doing science: Gender, self-efficacy, and science identity of undergraduate students in STEM. Journal of Women and Minorities in Science and Engineering, 20(2), 99–126. doi:10.1615/JWomenMinorScienEng.2014004477
• Woolfolk, A. E. (1998). Educational psychology (3rd ed.). Boston: Allyn and Bacon.
• Zimmerman, C., Bisanz, G.L., & Bisanz, J. (1998). Everyday scientific literacy: Do students use information about the social context and methods of research to evaluate news briefs about science?, Alberta Journal of Educational Research, 44(2), 188–207.