Effect of Web-Based Collaborative Learning Method with Scratch on Critical Thinking Skills of 5th Grade Students

Year 2023, Volume: 10 Issue: 2, 82 - 104, 30.03.2023

Gamze Alp , Nermin Bulunuz

https://doi.org/10.17275/per.23.30.10.2

Cited By: 1

Abstract

This study aimed to investigate the effect of web-based collaborative learning method with Scratch software on the critical thinking skills of 5th grade students and to create materials in Scratch software on the biodiversity subject. This study, in which a quasi-experimental design with pre-test - post-test control group was used, was carried out with 96 fifth grade students selected using convenient sampling method. In the implementation of the study, in addition to the education and training activities on biodiversity in the control groups, the students in the experimental group created products using the web-based collaborative learning method with Scratch in science and information technologies and software courses. As a result of the t-test for the independent samples of the post-test data obtained from the critical thinking questionnaire, it was determined that the critical thinking skills of the experimental group students were higher than the control group students. It is recommended for younger individuals to use Scratch to come up with solutions by using critical thinking skills to cope with complex problems affecting the whole world such as global warming, wars, and pandemics, and to choose the right solution amongst the given ones. Teachers can get an idea about students’ opinions, conceptual errors, and their capability to express themselves, and they can reorganize the educational activities by taking these variables into consideration.

Keywords

Scratch, web-based collaborative learning, critical thinking, middle school students, science lesson, coding

Thanks

This paper is extracted from the first author’s master dissertation “The impact of Scratch program and web-assisted cooperative learning method on the level of conceptual understanding and critical thinking skills of 5th grade students” submitted to the Bursa Uludag University of Turkey.

References

• Aivaloglou, E., Hermans, F., Moreno-León, J., & Robles, G. (2017). A dataset of scratch programs: scraped, shaped and scored. In Proceedings of the 14th International Conference on Mining Software Repositories (pp. 511-514). Piscataway, NJ: IEEE
• Al-Kindi N. S., & Al-Mekhlafi A. M. (2017). The practice and challenges of implementing critical thinking skills in Omani post-basic EFL classrooms. English Language Teaching, 10(12), 116–133. https://doi.org/10.5539/elt.v10n12p116
• Ala-Mutka, K. (2011). Mapping digital competence: Towards a conceptual understanding. Sevilla: Institute for Prospective Technological Studies, 7-60.
• Ali, G., & Awan, R.N. (2021). Thinking based instructional practices and academic achievement of undergraduate science students: Exploring the Role of Critical Thinking Skills and Dispositions. Journal of Innovative Sciences, 7(1), 56–70. https://doi.org/10.17582/journal.jis/2021/7.1.56.70
• Almeida, R., & Pessoa, T. (2017, November). Scratch software in higher education: Pedagogical experience in educational science. In Computers in Education (SIIE), 2017 International Symposium on (pp. 1-5). IEEE. Portugal. https://doi.org/10.1109/SIIE.2017.8259653
• Arısoy, B., & Aybek, B. (2021). The effects of subject-based critical thinking education in mathematics on students' critical thinking skills and virtues. Eurasian Journal of Educational Research, 92, 99-119. https://eric.ed.gov/?id=EJ1294083
• Armoni, M., & Gal-Ezer, J. (2014). Early computing education: why? what? when? who?. ACM Inroads, 5(4), 54-59. https://doi.org/10.1145/2684721.2684734
• Aydın E., & Selvi M. (2020). Ortaokul öğrencilerine yönelik ekosistem, biyolojik çeşitlilik ve çevre sorunları başarı testinin geliştirilmesi [Development of ecosystem, biodiversity and environmental problems achievement test for secondary school students]. Journal of Research in Education and Society, 7(2),661-682. https://dergipark.org.tr/en/pub/etad/issue/58757/825756
• Bacon, F. (1828). Advancement of learning. Cambridge: Harvard University Press.
• Benton, L., Hoyles, C., Kalas, I., & Noss, R. (2017). Bridging primary programming and mathematics: Some findings of design research in England. Digital Experiences in Mathematics Education, 3(2), 115-138. https://doi.org/10.1007/s40751-017-0028-x
• Bolat A., Karamustafaoğlu S., & Karamustafaoğlu, O. (2020). The effect of outdoor school learning environment on student achievement in 5th grade ‘world of living’ unit: example of biodiversity museum. Karaelmas Journal of Educational Sciences, 8(1), 42-54. https://dergipark.org.tr/en/pub/kebd/issue/67224/1049153
• Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. In Proceedings of the 2012 annual meeting of the American Educational Research Association (pp. 1–25). Vancouver, Canada.
• Büyüköztürk, Ş. (2018). Eğitimde bilimsel araştırma yöntemleri [Scientific research methods in education]. Ankara: Pegem.
• Calao, L. A., Moreno-León, J., Correa, H. E., & Robles, G. (2015). Developing mathematical thinking with Scratch. In G. Conole, T. Klobučar, C. Rensing, J. Konert, & E. Lavoué (Eds.), Design for Teaching and Learning in a Networked World. Lecture Notes in Computer Science, vol 9307 (pp. 17–27). https://doi.org/10.1007/978-3-319-24258-3_2
• Can, A. (2018). SPSS ile nicel veri analizi [Quantitative data analysis with SPSS]. Ankara: Pegem.
• Chiang, F. K., & Qin, L. (2018). A Pilot study to assess the impacts of game-based construction learning, using scratch, on students’ multi-step equation-solving performance. Interactive Learning Environments, 26(6), 803-814. https://doi.org/10.1080/10494820.2017.1412990
• Chang, C. K., Tsai, Y. T., & Chin, Y. L. (2017, July). A visualization tool to support analyzing and evaluating Scratch projects. In Advanced Applied Informatics (IIAI-AAI), 2017 6th IIAI International Congress on (pp. 498-502). IEEE. Japan. https://doi.org/10.1109/IIAI-AAI.2017.83
• Cortázar, C., Nussbaum, M., Harcha, J., Alvares, D., López, F., Goñi, J., & Cabezas, V. (2021). Promoting critical thinking in an online, project-based course. Computers in Human Behavior, 119, 106705. https://doi.org/10.1016/j.chb.2021.106705
• Cottrell, S. (2017). Critical thinking skills: Effective analysis, argument and reflection. London: Bloomsbury Publ.
• Çepni, S. (2018). Araştırma ve proje çalışmalarına giriş [Introduction to research and project studies]. Trabzon: Celebler.
• Descartes, R. (2012). Rules for the direction of the mind (J. Cottingham, R. Stotoothoff, D. Murdoch Trans.). Cambridge: Cambridge University Press. (Original work published in 1701). Retrieved from https://doi.org/10.1017/CBO9780511805042.004
• Dewey, J. (2022). How we think. Chelmsford: Courier Corporation. Retrieved from https://books.google.com.tr/books?hl=tr&lr=&id=ozCcEAAAQBAJ&oi=fnd&pg=PT7&dq=How+we+think&ots=J3saq3Rubd&sig=A5MJQJRRC4sRZlQrNLCEJm8_dS8&redir_esc=y#v=onepage&q=How%20we%20think&f=false
• Dickson, B. A., Kotsopoulos, D., & Harris, L. (2022). The Use of Coding Clubs to Develop Middle-School Students’ Spatial Reasoning Abilities. Digital Experiences in Mathematics Education, 8(1), 50-69. https://doi.org/10.1007/s40751-022-00099-x
• Dohn, N. B. (2020). Students’ interest in Scratch coding in lower secondary mathematics. British Journal of Educational Technology, 51(1), 71-83. https://doi.org/10.1111/bjet.12759
• Dowd, J. E., Thompson, R. J., Jr, Schif, L. A., & Reynolds, J. A. (2018). Understanding the complex relationship between critical thinking and science reasoning among undergraduate thesis writers. CBE-Life Sciences Education, 17(1), 1-10. https://doi.org/10.1187/cbe.17-03-0052
• Du, X., & Zhang, L. (2022). Investigating EFL learners’ perceptions of critical thinking learning affFordances: Voices from Chinese university English Majors. SAGE Open, 12(2), 21582440221094584. https://doi.org/10.1177/21582440221094584
• Duncan, O. D. (2014). Introduction to structural equation models. Amsterdam: Elsevier.
• Efe, H. A., & Efe R. (2022). Ortaokul öğrencilerine yönelik biyoçeşitlilik okuryazarlık değerlendirme aracı uyarlama çalışması [A biodiversity literacy assessment instrument adaptation study for middle school students]. Journal of Computer and Education Research, 10(20), 672-692. https://doi.org/10.18009/jcer.1135421
• Eggen, P., & Kauchak, D. (2001). Educational Psychology: Windows on Classrooms. 8th. Upper Saddle River, NJ: Pearson.
• Elder, L., & Paul, R. (1994). Critical thinking: Why we must transform our teaching. Journal of Developmental Education, 18(1), 34. https://www.proquest.com/docview/1437884071?pq-origsite=gscholar&fromopenview=true
• Ennis, R.H. (1991). “Goals for a critical thinking curriculum”. In A. Costa (Eds.), Developing minds (pp. 68-72). Virginia: ASCD. Retrieved from https://files.eric.ed.gov/fulltext/ED332166.pdf
• Falloon, G. (2016). An analysis of young students' thinking when completing basic coding tasks using Scratch Jnr. on the iPad. Journal of Computer Assisted Learning, 32(6), 576-593. https://doi.org/10.1111/jcal.12155
• Fields, D., Vasudevan, V., & Kafai, Y. B. (2015). The programmers’ collective: fostering participatory culture by making music videos in a high school Scratch coding workshop. Interactive Learning Environments, 23(5), 613-633. https://doi.org/10.1080/10494820.2015.1065892
• Fikriyatii, A., Agustini, R., & Sutoyo, S. (2022). Critical thinking cycle model to promote critical thinking disposition and critical thinking skills of pre-service science teacher. Cypriot Journal of Educational Science, 17(1), 120-133. https://doi.org/10.18844/cjes.v17i1.6690
• Food and Agriculture Organization (FAO). (2019). The state of the world’s biodiversity for food and agriculture. Retrieved from http://www.fao.org/state-of-biodiversity-for-food-agriculture/en/
• Gadanidis, G., Hughes, J. M., Minniti, L., & White, B. J. (2017). Computational thinking, grade 1 students and the binomial theorem. Digital Experiences in Mathematics Education, 3(2), 77-96. https://doi.org/10.1007/s40751-016-0019-3
• Gezer, U., & Ersoy, A. F. (2021). The effect of activities based on mobile applications on academic achievement, critical thinking skills and motivation in social studies course. Anadolu Journal of Educational Sciences International, 11(2), 790-825. https://doi.org/10.18039/ajesi.921684
• Glaser, R. (1984). Education and thinking: The role of knowledge. American psychologist, 39(2), 93. https://psycnet.apa.org/doi/10.1037/0003-066X.39.2.93
• Goodsett, M., & Schmillen, H. (2022). Fostering critical thinking in first-year students through ınformation literacy ınstruction. College & Research Libraries, 83(1), 91. https://doi.org/10.5860/crl.83.1.91
• Green, S. B., & Salkind, N. J. (2005). Using SPSS for Windows and Macintosh: Analyzing and understanding data. New Jersey: Pearson.
• Griffin, P., & Care, E. (2015). The ATC21S method. In P. Griffin & E. Care (Eds.), Assessment and teaching of 21st century skills (pp. 3-33). Dordrecht: Springer. Retrieved from https://link.springer.com/content/pdf/10.1007/978-94-017-9395-7.pdf
• Harvey, D. J., Montgomery, L. N., Harvey, H., Hall, F., Gange, A. C., & Watling, D. (2020). Psychological benefits of a biodiversity-focussed outdoor learning program for primary school children. Journal of Environmental Psychology, 67, 101381. https://doi.org/10.1016/j.jenvp.2019.101381
• Haydari, V. & Costu, B. (2021). The effect of common knowledge construction model-based instruction on 5th grade students' conceptual understanding of biodiversity. Journal of Education in Science, Environment and Health (JESEH), 7(3), 182-199. https://doi.org/10.21891/jeseh.840798
• Hazaymeh, W. A., & Alomery, M. K. (2022). The effectiveness of visual mind mapping strategy for improving English language learners' critical thinking skills and reading ability. European Journal of Educational Research, 11(1), 141-150. https://doi.org/10.12973/eu-jer.11.1.141
• Hermans, F., & Aivaloglou, E. (2017). Teaching software engineering principles to k-12 students: a mooc on scratch. Proceedings of the 39th International Conference on Software Engineering: Software Engineering and Education Track (pp. 13-22). Washington DC: IEEE Press. https://doi.org/10.1109/ICSE-SEET.2017.13
• Hidayati, N., Zubaidah, S., & Amnah, S. (2022). The PBL vs. digital mind maps ıntegrated PBL: choosing between the two with a view to enhance learners’ critical thinking. Participatory Educational Research, 9(3), 330-343. https://doi.org/10.17275/per.22.69.9.3
• Hudgins, B. B., & Edelman, S. (1988). Children’s self-directed critical thinking. The Journal of Educational Research, 81(5), 262-273. https://doi.org/10.1080/00220671.1988.10885834
• Hutchison, A., Nadolny, L., & Estapa, A. (2016). Using coding apps to support literacy instruction and develop coding literacy. The Reading Teacher, 69(5), 493-503. https://doi.org/10.1002/trtr.1440
• Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services (IPBES). (2019). Global assessment report on biodiversity and ecosystem services. Retrieved from https://doi.org/10.5281/zenodo.3831673
• International Society for Technology in Education. (ISTE). (2016). National educational technology standards for students. Retrieved from https://www.iste.org/standards/iste-standards-for-students
• Israel, M., & Lash, T. (2020). From classroom lessons to exploratory learning progressions: Mathematics+ computational thinking. Interactive Learning Environments, 28(3), 362-382. https://doi.org/10.1080/10494820.2019.1674879
• Istikomah, I., Basori, B., & Budiyanto, C. (2017). The influences of problem-based learning model with fishbone diagram to students's critical thinking ability. IJIE (Indonesian Journal of Informatics Education), 1(2), 83-91. https://jurnal.uns.ac.id/ijie/article/view/11432
• Jatiningsih, N. & Dewi, N. R. (2022). Development of e-comic science ınteractive learning with Scratch (eCILS) based on problem based learning to train critical thinking skills for junior high school students. Unnes Science Education Journal, 11(2), 90-99. https://journal.unnes.ac.id/sju/index.php/usej/article/view/56448
• Jones, P., McCreery, E., Judge, B., & Eales-Reynolds, L. J. (2013). Critical thinking skills for education students. London: Thinking Matters.
• Kaçar, T., & Çakmak, Z. (2020). Sosyal bilgiler dersinde sorgulamaya dayalı öğretimin öğrencilerin ders başarısına, eleştirel düşünme becerilerine ve öğrenmenin kalıcılığına etkisi. [The effect of inquiry-based teaching on students’ course achievement, critical thinking skills and permanence of learning in social studies course]. Electronic Journal of Social Sciences, 19(76), 1651-1672. https://doi.org/10.17755/esosder.712886
• Kanbul, S., & Uzunboylu, H. (2017). Importance of coding education and robotic applications for achieving 21st-century skills in North Cyprus. International Journal of Emerging Technologies in Learning (iJET), 12(01), 130-140. https://doi.org/10.3991/ijet.v12i01.6097
• Kang, M., Heo, H., Jo, I., Shin, J., & Seo, J. (2010). Developing an educational performance ındicator for new millennium students. Journal of Research on Technology in Education, 43(2), 157-170. https://doi.org/10.1080/15391523.2010.10782567
• Kanmaz, A. (2022). Middle school teachers' critical thinking skills and awareness towards teaching critical thinking skills. International Online Journal of Education and Teaching (IOJET), 9(4). 1648- 1671. https://eric.ed.gov/?id=EJ1353469
• Karanja, L. (2021). Teaching critical thinking in a college-level writing course: A critical reflection. International Online Journal of Education and Teaching, 8(1), 229-249.
• Kazancı, K. (2014). The effect of Web 2.0 tools on critical thinking with a special emphasis on collaborative learning. (Unpublished master’s dissertation). Çukurova University, Adana, Turkey.
• Keleş, F., & Özenoğlu, H. (2017). Ortaokul öğrencileri için biyolojik çeşitlilik konusunda ders planı tasarlama [Designing a lesson plan about biological diversity for secondary school students]. Adnan Menderes University Faculty of Education Journal of Educational Sciences (EJES), 8(2), 41-65.
• Khalid, L., Bucheerei, J., & Issah, M. (2021). Pre-service teachers’ perceptions of barriers to promoting critical thinking skills in the classroom. SAGE Open, 11(3), 21582440211036094. https://doi.org/10.1177/21582440211036094
• Krathwohl D. R. (2002). A revision of Bloom's taxonomy: an overview. Theory into Practice, 41(4), 212-218. https://doi.org/10.1207/s15430421tip4104_2
• Lai, E.R., & Viering, M. (2012). Assessing 21st century skills: Integrating research findings. New Jersey: Pearson.
• Lesest, B., & Wolbers, M. H. J. (2020). Critical thinking, creativity and study results as predictors of selection for and success of excellence programs in Dutch higher education ınstitutions. European journal of higher education, 11(1), 29-43. https://doi.org/10.1080/21568235.2020.1850310
• Lombardi, L., Mednick, F. J., De Backer, F., & Lombaerts, K. (2022). Teachers' perceptions of critical thinking in primary education. International Journal of Instruction, 15(4), 1-16. https://doi.org/10.29333/iji.2022.1541a
• Ma, H., Zhao, M., Wang, H., Wan, X., Cavanaugh, T. W., & Liu, J. (2021). Promoting pupils’ computational thinking skills and self-efficacy: A problem-solving instructional approach. Educational Technology Research and Development, 69(3), 1599-1616. https://doi.org/10.1007/s11423-021-10016-5
• McDonald, S. D. (2017). Enhanced critical thinking skills through problem-solving games in secondary schools. Interdisciplinary Journal of E-Learning & Learning Objects, 13, 79-96. https://doi.org/10.28945/3711
• Miller, J. (2019). STEM education in the primary years to support mathematical thinking: Using coding to identify mathematical structures and patterns. Zdm Mathematics Education, 51(6), 915-927. https://doi.org/10.1007/s11858-019-01096-y
• Ministry of Education (MoNE) (2018a). Bilişim teknolojileri ve yazılım dersi programı [Information Technologies and Software Curriculum]. Ankara Board of Education, Ankara.
• Ministry of Education (MoNE) (2018b). Fen bilimleri dersi öğretim programı [Science course curriculum]. Ankara: Board of Education.
• Mongar, K. (2022). Alignment of the environmental science textbooks, examinations and curriculum framework to achieve the teaching objectives. Journal of Turkish Science Education, 19(1), 52-70
• Moonma, J., & Kaweera, C. (2022). A study of critical thinking skills practice in collaborative writing in EFL context. Asian Journal of Education and Training, 8(1), 8-14. https://doi.org/10.20448/edu.v8i1.3656
• More, T. (1890). Utopia. Oxford: Oxford University Press.
• Moreno-León, J., Robles, G., & Román-González, M. (2015). Dr. Scratch: Automatic analysis of scratch projects to assess and foster computational thinking. RED. Revista de Educación a Distancia, (46), 1-23.
• Mullis, I. V. S., & Martin, M. O. (2017). TIMSS 2019 assessment frameworks. TIMSS and PIRLS International Study Center. Lynch School of Education, Boston College. Retrieved from http://timssandpirls.bc.edu/timss2019/frameworks/
• Mullis, I. V., & Martin, M. O. (2019). PIRLS 2021 assessment frameworks. TIMSS and PIRLS International Study Center. Lynch School of Education, Boston College. Retrieved from https://files.eric.ed.gov/fulltext/ED606056.pdf
• Mutakinati, L., Anwari, I., & Kumano, Y. (2018). Analysis of students’ critical thinking skill of middle school through STEM education project-based learning. Jurnal Pendidikan IPA Indonesia, 7(1), 54-65. https://doi.org/10.15294/jpii.v7i1.10495
• National Research Council (NRC). (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. Washington, DC: The National Academies Press.
• National Research Council (NRC). (2015). Guide to implementing the next generation science standards. Washington, DC: The National Academies Press.
• Niemelä, P. (2017, October). All rosy in Scratch lessons: No bugs but guts with visual programming. 2017 IEEE Frontiers in Education Conference (FIE) (pp. 1-9). IEEE. USA. https://doi.org/10.1109/FIE.2017.8190612
• North Central Regional Educational Laboratory (NCREL). (2003). EnGauge 21st century skills: Digital literacy for the digital age. Napierville, IL and Los Angeles, CA: NCREL.
• Nouri, J., Zhang, L., Mannila, L., & Norén, E. (2020). Development of computational thinking, digital competence and 21st century skills when learning programming in K-9. Education Inquiry, 11(1), 1-17. https://doi.org/10.1080/20004508.2019.1627844
• Oluk, A., & Korkmaz, Ö. (2016). Comparing students' scratch skills with their computational thinking skills in terms of different variables. International Journal of Modern Education and Computer Science (IJMECS), 8(11), 1-7. https://doi.org/10.5815/ijmecs.2016.11.01
• Oluk, A., Korkmaz, Ö., & Oluk, H. A. (2018). Scratch’ın 5. sınıf öğrencilerinin algoritma geliştirme ve bilgisayarca düşünme becerilerine etkisi [The effect of using Scratch on developing skills related to algorithm development and computational thinking]. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 9(1), 54-71.
• Organisation for Economic Co-operation and Development (OECD). (2018). The future of education and skills-education, learning compass 2030. Retrieved from https://www.oecd.org/education/2030-project/about/
• Ouhıba M., (2022). The role of literature in boosting EFL university students’ critical thinking: Case of first-year students in Algeria. Arab World English Journal (AWEJ), 13(1) 477-485. https://dx.doi.org/10.24093/awej/vol13no1.31
• Partnership for 21st Century Skills (P21). (2009). P21 framework definitions. Retrieved from http://www.p21.org/storage/documents/P21_Framework_Definitions.pdf
• Paul, R., & Elder, L. (2009). The miniature guide to critical thinking concepts and tools. Lanham: Rowman & Littlefield.
• Quattrucci, J. G. (2018). Problem-Based approach to teaching advanced chemistry laboratories and developing students’ critical thinking skills. Journal of Chemical Education, 95(2), 259-266. https://doi.org/10.1021/acs.jchemed.7b00558
• Ridlo, S. (2020). Critical thinking skills reviewed from communication skills of the primary school students in STEM-based project-based learning model. Journal of Primary Education, 9(3), 311-320. https://doi.org/10.15294/jpe.v9i3.27573
• Ristanto, R., Sabrina, A., & Komala, R. (2022). Critical thinking skills of environmental changes: a biological ınstruction using guided discovery learning-argument mapping (GDL-AM). Participatory Educational Research, 9(1), 173-191. https://doi.org/10.17275/per.22.10.9.1
• Rodríguez-Martínez, J. A., González-Calero, J. A., & Sáez-López, J. M. (2020). Computational thinking and mathematics using Scratch: An experiment with sixth-grade students. Interactive Learning Environments, 28(3), 316-327. https://doi.org/10.1080/10494820.2019.1612448
• Ruggiero, R. V. (2012). Beyond feelings: a guide to critical thinking. New York: McGraw Hill.
• Sasson, I., Yehuda, I., & Malkinson, N. (2018). Fostering the skills of critical thinking and question-posing in a project-based learning environment. Thinking Skills and Creativity, 29, 203-212. https://doi.org/10.1016/j.tsc.2018.08.001
• Schaal, S., Schaal, S., & Lude, A. (2015). Digital Geogames to foster local biodiversity. International Journal for Transformative Research, 2(2), 16-29. https://doi.org/10.1515/ijtr-2015-0009
• Schönfelder, M. L., & Bogner, F. X. (2018). How to sustainably increase students’ willingness to protect pollinators. Environmental Education Research, 24(3), 461-473. https://doi.org/10.1080/13504622.2017.1283486
• Statter, D., & Armoni, M. (2020). Teaching abstraction in computer science to 7th grade students. ACM Transactions on Computing Education (TOCE), 20(1), 1-37. https://doi.org/10.1145/3372143
• Stobaugh, R. (2013). Assessing critical thinking in middle and high schools: Meeting the common core. Londra: Routledge.
• The Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services (IPBES) (2019). Global assessment report on biodiversity and ecosystem services. Retrieved from https://ipbes.net/global-assessment
• Van Laar, E., van Deursen, A. J., van Dijk, J. A., & de Haan, J. (2017). The relation between 21st-century skills and digital skills: A systematic literature review. Computers in Human Behavior, 72, 577–588. https://doi.org/10.1016/j.chb.2017.03.010
• Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L., & Wilensky, U. (2016). Defining computational thinking for mathematics and science classrooms. Journal of Science Education and Technology, 25(1), 127-147. https://doi.org/10.1007/s10956-015-9581-5
• Weng, X., Cui, Z., Ng, O. L., Jong, M. S., & Chiu, T. K. (2022a). Characterizing Students’ 4C Skills Development During Problem-based Digital Making. Journal of Science Education and Technology, 31(3), 372-385. https://doi.org/10.1007/s10956-022-09961-4
• Weng, X., Ng, O. L., Cui, Z., & Leung, S. (2022b). Creativity Development with Problem-Based Digital Making and Block-Based Programming for Science, Technology, Engineering, Arts, and Mathematics Learning in Middle School Contexts. Journal of Educational Computing Research, 0, 1-25. https://doi.org/10.1177/07356331221115661
• World Economic Forum (WEF). (2020). The global risk report. Retrieved from https://www.weforum.org/reports/the-global-risks-report-2020
• World Wide Fund for Nature (WWF). (2020). Doğanın Yok Oluşu ve Pandemilerin Yükselişi [The Extinction of Nature and the Rise of Pandemics]. Retrieved from https://www.wwf.org.tr/?9920/Doganin-Yok-Olusu-ve-Pandemilerin-Yukselisi
• Yeşilyurt, E. (2021). Critical thinking and its teaching: A conceptual overview of all dimensions and elements. The Journal of International Social Research, 14(77), 816-828.
• Zhang, L., & Nouri, J. (2019). A systematic review of learning computational thinking through Scratch in K-9. Computers & Education, 141, 103607. https://doi.org/10.1016/j.compedu.2019.103607