A validity and reliability study of the Turkish computational thinking scale

Abstract

The purpose of this study was to adapt the computational thinking scale developed by Tsai, Liang and Hsu (2021) into Turkish in order to determine the computational thinking skill levels of secondary school students according to such basic elements defined by Selby and Woollard (2013) as abstraction, decomposition, algorithmic thinking, evaluation and generalization and to do the related validity and reliability study. A total of 454 high school students (9th – 12th grade) determined with the convenient sampling method constituted the sample of the study. The original scale was made up of 19 5-point Likert-type items. Confirmatory Factor Analysis (CFA) was performed to examine the conformity of the data collected via the adapted scale to the five-factor structure of the original scale. As a result of CFA, it was seen that the factor structure of the original scale was preserved. The reliability of the scale was checked with the internal consistency coefficient for the whole scale and its factors. The Cronbach Alpha coefficients obtained were .84 and McDonald's omega coefficients obtained were .86. The scale's Turkish adaptation was found to be a valid and trustworthy measurement tool for establishing the computational thinking proficiency levels of students in high school.

Keywords

• Computational thinking
• scale adaptation
• validity
• reliability

References

1. Allsop, Y. (2019). Assessing computational thinking process using a multiple evaluation approach. International Journal of Child-Computer Interaction, 19, 30–55. https://doi. org/10.1016/j.ijcci.2018.10.004
2. Aumgri, C., & Petsangsri, S. (2019). Computational thinking for preservice teachers in Thailand. A confirmatory factor analysis. Revista ESPACIOS, 40(29).
3. Barendsen, E., Mannila, L., Demo, B., Grgurina, N., Izu, C., Mirolo, C., ... & Stupurienė, G. (2015). Concepts in K-9 computer science education. In Proceedings of the 2015 ITiCSE on working group reports (pp. 85-116).
4. Bavera, F., Daniele, M., Buffarini, F., Quintero, T., & De Dominici, C. (2020). Didáctica de las ciencias de la computación: experiencias y percepciones de docentes de educación primaria. In XXVI Congreso Argentino de Ciencias de la Computación (CACIC)(Modalidad virtual, 5 al 9 de octubre de 2020).
5. Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. In Annual American Educational Research Association Meeting, Vancouver, BC, Canada.
6. Brown, T.A. (2006). Confirmatory factor analysis for applied research. New York: Guilford Press
7. Büyüköztürk, Ş. (2013). Sosyal Bilimler için Veri Analizi El Kitabı İstatistik, Araştırma Deseni SPSS Uygulamaları ve Yorum. Ankara: PegemA Yayıncılık, Genişletilmiş 18. Baskı, Ankara.
8. Byrne, B. M. (1998). Structural equation modeling with LISREL, PRELIS, and SIMPLIS: Basic concepts, applications, and programming.

9. Ceylan, V. K. (2020). Senaryo temelli scratch öğretim programının öğrencilerin bilgi işlemsel düşünme becerilerine, problem çözme ve programlama ünitesi erişilerine etkisi. Doktora tezi, Aydın Adnan Menderes Üniversitesi Sosyal Bilimler Enstitüsü, Aydın.
10. Coşkun R, Altunışık R, Yıldırım E, Bayraktaroğlu S. (2010). Sosyal Bilimlerde Araştırma Yöntemleri SPSS Uygulamaları. Sakarya: Sakarya Yayıncılık.
11. Çetin, İ., Otu, T., & Oktaç, A. (2020). Adaption of the Computational Thinking Test into Turkish. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 11(2), 343-360.
12. Çokluk, Ö., Şekercioğlu, G., & Büyüköztürk, Ş. (2012). Sosyal bilimler için çok değişkenli istatistik: SPSS ve LISREL uygulamaları (Vol. 2). Ankara: Pegem Akademi.
13. Denning, P. J. (2007). Computing is a natural science. Communications of ACM, 50, 13–18
14. Denning, P. J. (2009). The profession of IT Beyond computational thinking. Communications of the ACM, 52(6), 28-30.
15. Dikkartın Övez, F.T. & Acar, İ. G. (2022). The effect of block-based game development activities on the geometry achievement, computational thinking skills and opinions of seventh-grade students. Journal of Educational Technology & Online Learning, 5(4), 1106-1121.
16. diSessa, A. A. (1991). An overview of Boxer. Journal of Mathematical Behavior, 10(1), 3- 15.
17. Doleck, T., Bazelais, P., Lemay, D. J., Saxena, A., & Basnet, R. B. (2017). Algorithmic thinking, cooperativity, creativity, critical thinking, and problem solving: Exploring the relationship between computational thinking skills and academic performance. Journal of Computers in Education, 4(4), 355–369. https://doi.org/10.1007/s40692-017-0090-9
18. Dolmacı, A., & Akhan, N. E. (2020). Bilişimsel Düşünme Becerileri Ölçeğinin Geliştirilmesi: Geçerlik ve Güvenirlik Çalışması. Itobiad: Journal of the Human & Social Science Researches 9(3).
19. Ertugrul-Akyol, B. (2019). Development of Computational Thinking Scale: Validity and Reliability Study. International Journal of Educational Methodology, 5(3), 421-432.
20. Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological methods, 4(3), 272.
21. Fornell C, Larcker DF. (1981). Evaluating structural equation models with un observable variables and measurement error. Journal of Marketing Research, 18(1), 39-50
22. Fraillon, J., Ainley, J., Schulz, W., Duckworth, D., and Friedman, T. (2019). IEA international computer and information literacy study 2018 assessment framework (p. 74). Springer Nature.
23. Gözüm, S. ve Aksayan, S. (2003). Kültürlerarası ölçek uyarlaması için rehber II: psikometrik özellikler ve kültürlerarası karşılaştırma. Hemşirelikte Araştırma Geliştirme Dergisi, 5, 1-25
24. Grover, S., & Pea, R. (2013). Computational thinking in K–12: A review of the state of the field. Educational researcher, 42(1), 38-43.
25. Guzdial, M. (2008). Education: Paving the way for computational thinking. Communications of ACM, 51, 25–27. https://doi.org/10.1145/1378704.1378713
26. Güngör, D. (2016). Psikolojide Ölçme Araçlarının Geliştirilmesi ve Uyarlanması Kılavuzu. Türk psikoloji yazıları, 19(38), 104-112.
27. Gülbahar, Y., Kert, S. B., & Kalelioğlu, F. (2019). Bilgi işlemsel düşünme becerisine yönelik öz yeterlik algısı ölçeği: Geçerlik ve güvenirlik çalışması. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 10(1), 1-29.
28. Hambleton, R. K., & Bollwark, J. (1991). Adapting Tests for Use in Different Cultures: Technical Issues and Methods.
29. Hambleton, R. K., & Kanjee, A. (1993). Enhancing the Validity of Cross-Cultural Studies: Improvements in Instrument Translation Methods.
30. Hsu, C. Y., Liang, J. C., & Tsai, M. J. (2020). Probing the structural relationships between teachers’ beliefs about game-based teaching and their perceptions of technological pedagogical and content knowledge of games. Technology, Pedagogy and Education, 29(3), 297-309.
31. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modeling: a multidisciplinary journal, 6(1), 1-55.
32. İliç¸, U., Haseski, H., & Tuğtekin, U. (2018). Publication trends over 10 years of computational thinking research. Informatics in Education, 18(2), 297
33. Karalar, H., & Alpaslan, M. M. (2021). Assessment of Eighth Grade Students' Domain-General Computational Thinking Skills. International Journal of Computer Science Education in Schools, 5(1), 35-47.
34. Kong, S.-C., Abelson, H., & Lai, M. (2019). Introduction to computational thinking education. In S.-C. Kong & H. Abelson (Eds.), Computational thinking education (pp. 1-10). Springer Singapore
35. Korkmaz, Ö., Çakir, R., & Özden, M. Y. (2017). A validity and reliability study of the computational thinking scales (CTS). Computers in human behavior, 72, 558-569.
36. Kukul, V., & Karatas, S. (2019). Computational thinking self-efficacy scale: Development, validity and reliability. Informatics in Education, 18(1), 151-164.
37. Kutlu, Ö., Doğan, C. D. & Karakaya, İ. (2014). Ölçme ve değerlendirme: performansa ve portfolyoya dayalı durum belirleme. Ankara: Pegem Akademi.
38. Lu, J. J., & Fletcher, G. H. (2009). Thinking about computational thinking. ACM SIGCSE Bulletin, 41(1), 260-264.
39. MEB TTK. (2018). Ortaöğretim Kurumları Haftalık Ders Çizelgesi. Ankara: Talim ve Terbiye Kurulu Başkanlığı.
40. MEB. (2018). Matematik Öğretim Programı. http://mufredat.meb.gov.tr/Programlar.aspx. Erişim Tarihi: 9/01/2023.
41. Nunnally, B. H. & Bernstein, J.C. (1994). Psychometric theory (3rd ed.). London, UK: McGraw-Hill
42. Papert, S. (1980). Teaching children to be mathematicians us. teaching about mathematics (No. 249). memo.
43. Polya, G. (1945). How to solve it; a new aspect of mathematical method. Princeton University Press.
44. Raykov T. (1998). Coefficient alpha and composite reliability with ınterrelated nonhomogeneous ıtems. Applied Psychological Measurement, 22(4), 375-385.
45. Román-González, M., Pérez-González, J. C., & Jiménez-Fernández, C. (2017). Which cognitive abilities underlie computational thinking? Criterion validity of the Computational Thinking Test. Computers in human behavior, 72, 678-691.
46. Selby, C., & Woollard, J. (2013). Computational thinking: The developing definition. https://eprints.soton.ac.uk/356481/
47. Sherin, B., diSessa, A. A., & Hammer, D. (1993). Dynaturtle revisited: Learning physics through collaborative design of a computer model. Interactive Learning Environments, 3(2), 91-118.
48. Sneider, C., Stephenson, C., Schafer, B., & Flick, L. (2014). Exploring the science framework and NGSS: Computational thinking in the science classroom. Science Scope, 38(3), 10-15.
49. Şencan, H. (2005). Sosyal ve davranışsal ölçümlerde güvenilirlik ve geçerlilik. Ankara: Seçkin Yayınevi
50. Tang, X., Yin, Y., Lin, Q., Hadad, R., & Zhai, X. (2020). Assessing computational thinking: A systematic review of empirical studies. Computers & Education, 148, 103798. https://doi.org/10.1016/j.compedu.2019.103798
51. Tsai, M.-J. (1999). The impact of strategic and cooperative learning on Taiwanese eighth graders’ computer achievement, attitudes, and anxiety (Unpublished Doctoral Dissertation). The University of Texas at Austin, Austin, TX, USA.
52. Tsai, M.-J., Wang, C.-Y., & Hsu, P.-F. (2019). Developing the Computer Programming Self-Efficacy Scale (CPSES) for computer literacy education. Journal of Educational Computing Research, 56(8), 1345–1360. https://doi.org/10.1177/0735633117746747
53. Tsai, M. J., Liang, J. C., & Hsu, C. Y. (2021). The computational thinking scale for computer literacy education. Journal of Educational Computing Research, 59(4), 579-602.
54. Tsai, M. J., Liang, J. C., Lee, S. W. Y., & Hsu, C. Y. (2022). Structural validation for the developmental model of computational thinking. Journal of Educational Computing Research, 60(1), 56-73.
55. Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L., & Wilensky, U. (2016). Defining computational thinking for mathematics and science classrooms. Journal of Science Education and Technology, 25(1), 127–147. https://doi.org/10. 1007/s10956-015-9581-5
56. Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49, 33–36.
57. Wing, J. M. (2008). Computational thinking and thinking about computing. Philosophical Transactions of the Royal Society A – Mathematical Physical and Engineering Sciences, 366(1881), 3717–3725.
58. Wing, J. M. (2011). Research notebook: Computational thinking – what and why? The link. Carneige Mellon.
59. Yadav, A., Mayfield, C., Zhou, N., Hambrusch, S., & Korb, J. T. (2014). Computational thinking in elementary and secondary teacher education. ACM Transactions on Computing Education (TOCE), 14(1), 1–16. https://doi.org/10.1145/2576872
60. Yağcı, M. (2019). Bilişimsel düşünme becerilerini incelemek için geçerli ve güvenilir bir araç. Eğitim ve Bilgi Teknolojileri, 24 (1), 929-951