Developing a game-based test to assess middle school sixth-grade students’ algorithmic thinking skills

Öz

This study was carried out to develop a test to assess algorithmic thinking skills. To this end, the twelve steps suggested by Downing (2006) were adopted. Throughout the test development, 24 middle school sixth-grade students and eight experts in different areas took part as needed in the tasks on the project. The test was given to 252 students attending the sixth grade who were selected through purposeful sampling. The content validity of the test was ensured by means of obtaining expert opinion, whereas the construct validity was ensured by performing an independent sample t-test on the difference between the lower and upper groups. As a result, the algorithmic thinking skills assessment test was finalized with 22 main items and 2 sample items, totalling 24 items. The KR-20 reliability analysis proved a quite reliable test based on the reliability coefficient of 0.83. As mentioned earlier, the independent sample t-test was applied to the difference of lower and upper groups for construct validation of the test. It was seen that the test items are significant in discriminating the students in the lower and upper groups (p<0.01).

Anahtar Kelimeler

• Algorithmic thinking
• Test
• Middle school
• Game
• Embodied cognition

References

1. Altakrouri, B., & Schrader, A. (2012, September). Towards dynamic natural interaction ensembles. In the 26th BCS Conference on Human Computer Interaction 26 (pp. 1-10). http://dx.doi.org/10.14236/ewic/HCI2012.0
2. Anderson, M.L. (2003). Embodied cognition: A field guide. Artificial intelligence, 149(1), 91-130. https://doi.org/10.1016/S0004-3702(03)00054-7
3. Apostolellis, P., Stewart, M., Frisina, C., & Kafura, D. (2014). RaBit EscAPE: A board game for computational thinking. In Proceedings of the 2014 conference on Interaction design and children (pp. 349-352). http://dx.doi.org/10.1145/2593968.2610489
4. Ayala, N.A.R., Mendívil, E.G., Salinas, P., & Rios, H. (2013). Kinesthetic learning applied to mathematics using kinect. Procedia Computer Science, 25, 131 135. https://doi.org/10.1016/j.procs.2013.11.016
5. Aytekin, A., Çakır, F.S., Yücel, Y.B., & Kulaözü, İ. (2018). The place and importance of algorithms in our lives. Eurasian Journal of Social and Economic Studies, 5(7), 143 150. https://dergipark.org.tr/tr/pub/asead/issue/41013/495619
6. Başol, G. (2019). Measurement and evaluation in education. Pegem Citation Index, 001-307.
7. Baykul, Y., Gelbal, S., & Kelecioğlu, H. (2003). Measurement and evaluation in education for Anatolian teacher high schools. National Education Printing House.
8. Bayrakçeken, S. (2015). Test development. E. Karip (Ed.), In Measurement and Evaluation (s. 292-322). Pegem Academy.

9. Bellocchi, A., King, D.T., & Ritchie, S.M. (2016). Context-based assessment: Creating opportunities for resonance between classroom fields and societal fields. International Journal of Science Education, 38(8), 1304 1342. https://doi.org/10.1080/09500693.2016.1189107
10. Borkulo, S., Chytas, C., Drijvers, P., Barendsen, E., & Tolboom, J. (2021). Computational thinking in the mathematics classroom: Fostering algorithmic thinking and generalization skills using dynamic mathematics software. In The 16th Workshop in Primary and Secondary Computing Education (pp. 1 9). https://doi.org/10.1145/3481312.3481319.
11. Brown, W. (2015). Introduction to algorithmic thinking. https://raptor.martincarlisle.com/Introduction%20to%20Algorithmic%20Thinking.doc
12. Büyüköztürk, Ş., Kılıç-Çakmak, E., Akgün, Ö.E., Karadeniz, Ş., & Demirel, F. (2020). Scientific research methods. Pegem Publications.
13. Chen, K.Z., & Chi, H.H. (2022). Novice young board-game players’ experience about computational thinking. Interactive Learning Environments, 30(8), 1375-1387. https://doi.org/10.1080/10494820.2020.1722712
14. Chuechote, S., Nokkaew, A., Phongsasithorn, A., & Laosinchai, P. (2020). A neo-piagetian analysis of algorithmic thinking development through the “sorted” digital game. Contemporary Educational Technology, 12(1), 1 15. http://dx.doi.org/10.30935/cet.685959
15. Czakóová, K. (2020). Developing algorithmic thinking by educational computer games. Paper presented at the Conference eLearning and Software for Educationtional, Romania. http://dx.doi.org/10.12753/2066-026X-20-003
16. Czakóová, K., & Udvaros, J. (2021). Applications and games for the development of algorithmic thinking in favor of experiential learning. In EDULEARN21 Proceedings (pp. 6873-6879). IATED. https://doi.org/10.21125/edulearn.2021.1389
17. Çetin, B. (Ed.) (2019). Measurement and evaluation in education. Anı Publishing.
18. Debabi, W., & Bensebaa, T. (2016). Using serious game to enhance learning and teaching algorithmic. Journal of e Learning and Knowledge Society, 12(2). http://dx.doi.org/10.20368/1971-8829/1125
19. Demir, O., & Köse, İ.A. (2014). Comparison of cutoff scores determined by Angoff, Nedelsky and Ebel standard setting methods. Journal of Mersin University Faculty of Education, 10(2), 14 27. https://dergipark.org.tr/en/pub/mersinefd/issue/17394/181823?publisher=mersin
20. Doleck, T., Bazelais, P., Lemay, D.J., Saxena, A., & Basnet, R.B. (2017). Algorithmic thinking, cooperativity, creativity, critical thinking, and problem solving: Exploring the relationship between computational thinking skills and academic performance. Journal of Computers in Education, 4(4), 355-369. https://doi.org/10.1007/s40692-017-0090-9
21. Downing, S.M. (2006). Twelve steps for effective test development. Handbook of test development, 3, 25. https://doi.org/10.4324/9780203874776.ch1
22. Elshahawy, M., Aboelnaga, K., & Sharaf, N. (2020). Codaroutine: A serious game for introducing sequential programming concepts to children with autism. In 2020 IEEE Global Engineering Education Conference (EDUCON) (pp. 1862 1867). IEEE. http://dx.doi.org/10.1109/EDUCON45650.2020.9125196
23. Erümit, K.A., Karal, H., Şahin, G., Aksoy, D.A., Aksoy, A., & Benzer, A.İ. (2018). A model proposal for teaching programming: programming in seven steps. Education and Science, 44(197), 1-29. http://dx.doi.org/10.15390/EB.2018.7678
24. Evripidou, S., Amanatiadis, A., Christodoulou, K., & Chatzichristofis, S.A. (2021). Introducing algorithmic thinking and sequencing using tangible robots. IEEE Transactions on Learning Technologies, 14(1), 93 105. https://doi.org/10.1109/TLT.2021.3058060
25. Fensham, P.J., & Rennie, L.J. (2013). Towards an authentically assessed science curriculum. In Valuing assessment in science education: Pedagogy, curriculum, policy (pp. 69-100). Springer.
26. Figueiredo, M., Amante, S., Gomes, H.M.D.S.V., Gomes, M.A., Rego, B., Alves, V., & Duarte, R.P. (2021). Algorithmic thinking in early childhood education: Opportunities and supports in the Portuguese context. In EDULEARN21 Proceedings (pp. 9339-9348). IATED. https://doi.org/10.22521/edupij.2022.112.3
27. Futschek, G., & Moschitz, J. (2010). Developing algorithmic thinking by inventing and playing algorithms. Proceedings of the 2010 constructionist approaches to creative learning, thinking and education: Lessons for the 21st century (constructionism 2010), 1-10.
28. Gall, M.D., Barg, W.R., & Gall, j.P. (1996). Educational research: an introduction (6th ed.). Longman Publishing.
29. Geisinger, K.F. (2016). 21st century skills: What are they and how do we assess them?. Applied Measurement in Education, 29(4), 245 249. https://doi.org/10.1080/08957347.2016.1209207
30. Güler, N. (2015). Measurement and evaluation in education (7th ed.). Pegem Academy Publishing.
31. Gürbüz, H., Evlioğlu, B., Erol, Ç.S., Gülseçen, H., & Gülseçen, S. (2017). “What’s the weather like today?”: A computer game to develop algorithmic thinking and problem solving skills of primary school pupils. Education and Information Technologies, 22(3), 1133-1147. https://doi.org/10.1007/s10639-016-9478-9
32. Hsu, C.C., & Wang, T.I. (2018). Applying game mechanics and student-generated questions to an online puzzle-based game learning system to promote algorithmic thinking skills. Computers & Education, 121, 73 88. http://dx.doi.org/10.1016/j.compedu.2018.02.002
33. Hu, C. (2011). Computational thinking: What it might mean and what we might do about it. In Proceedings of the 16th annual joint conference on innovation and technology in computer science education (pp. 223 227). ACM. http://dx.doi.org/10.1145/1999747.1999811
34. Huizinga, J. (1955). Homo ludens: A study of the play element in culture. The Beacon Press. https://doi.org/10.2307/2087716
35. Iacolina, S.A., Lai, A., Soro, A., & Scateni, R. (2010). Natural interaction and computer graphics applications. In Eurographics italian chapter conference (pp. 141–146).
36. Jančec, L., & Vujičić, L. (2021). Project “Algorithmic Thinking Skills through Play-Based Learning for Future's Code Literates”. In 2021 44th International Convention on Information, Communication and Electronic Technology (MIPRO) (pp. 641-644). IEEE. https://doi.org/10.23919/MIPRO52101.2021.9597078
37. Kanaki, K., & Kalogiannakis, M. (2022). Assessing algorithmic thinking skills in relation to gender in early childhood. Educational Process: International Journal, 11, 44 59. https://dx.doi.org/10.22521/edupij.2022.112.3
38. Karakasis, C., & Xinogalos, S. (2020). BlocklyScript: Design and pilot evaluation of an RPG platform game for cultivating computational thinking skills to young students. Informatics in Education, 19(4), 641 668. https://doi.org/10.15388/infedu.2020.28
39. Karatay, R., & Doğan, F. (2016). Development of science process skill scale of 7th grade secondary school students. Dicle Üniversitesi Ziya Gökalp Eğitim Fakültesi Dergisi, 27, 1-8. http://dx.doi.org/10.14582/DUZGEF.548
40. Kátai, Z. (2015). The challenge of promoting algorithmic thinking of both sciences‐and humanities‐oriented learners. Journal of Computer Assisted Learning, 31(4), 287-299. https://doi.org/10.1111/jcal.12070
41. Kazimoglu, C. (2020). Enhancing confidence in using computational thinking skills via playing a serious game: A case study to increase motivation in learning computer programming. IEEE Access, 8, 221831 221851. http://dx.doi.org/10.1109/ACCESS.2020.3043278
42. Keane, T. (2012). Leading with technology: 21st century skills= 3Rs+ 4Cs. Australian Educational Leader, 34(2), 44. https://search.informit.org/doi/10.3316/informit.895597893475453
43. Khanlari, A. (2013). Effects of robotics on 21st century skills. European Scientific Journal, 9(27). https://doi.org/10.19044/esj.2013.v9n27p%25p
44. Kim, B., Park, H., & Baek, Y. (2009). Not just fun, but serious strategies: Using meta-cognitive strategies in game based learning. Computers & Education, 52(4), 800 810. https://doi.org/10.1016/j.compedu.2008.12.004
45. Kiss, G., & Arki, Z. (2017). The influence of game-based programming education on the algorithmic thinking. Procedia Social and Behavioral Sciences, 237, 613 617. http://dx.doi.org/10.1016/j.sbspro.2017.02.020
46. Kocagül-Sağlam, M. & Ünal-Çoban, G. (2019). Developing the reasoning skills test for science teachers and teacher candidates. Elementary Education Online, 17(3). https://doi.org/10.17051/ilkonline.2018.466374
47. Kosmas, P., Ioannou, A., & Retalis, S. (2018). Moving bodies to moving minds: A study of the use of motion-based games in special education. TechTrends, 62, 594-601. https://doi.org/10.1007/s11528-018-0294-5
48. Kosmas, P., & Zaphiris, P. (2023). Improving students’ learning performance through Technology Enhanced Embodied Learning: A four year investigation in classrooms. Education and Information Technologies, 1 24. https://doi.org/10.1007/s10639-022-11466-x
49. La Belle, T.J., Moll, L.C., & Weisner, T.S. (1979). Context-based educational evaluation: A participant research strategy. Educational Evaluation and Policy Analysis, 1(3), 85-94. https://doi.org/10.2307/1164160
50. Lee, T.Y., Mauriello, M.L., Ahn, J., & Bederson, B.B. (2014). CTArcade: Computational thinking with games in school age children. International Journal of Child-Computer Interaction, 2(1), 26-33. http://dx.doi.org/10.1016/j.ijcci.2014.06.003
51. Lertlapnon, T., Lueangrungudom, N., & Vittayakorn, S. (2022). Protobot: An Educational Game for Algorithmic Thinking. In 2022 14th International Conference on Information Technology and Electrical Engineering (ICITEE) (pp. 79 84). IEEE. https://doi.org/10.1109/ICITEE56407.2022.9954081
52. Li, J., Lin, Y., Sun, M., & Shadiev, R. (2020). Socially shared regulation of learning in game-based collaborative learning environments promotes algorithmic thinking, learning participation and positive learning attitudes. Interactive Learning Environments, 1 12. https://doi.org/10.1080/10494820.2020.1857783
53. Lin, S.Y., Chien, S.Y., Hsiao, C.L., Hsia, C.H., & Chao, K.M. (2020). Enhancing computational thinking capability of preschool children by game-based smart toys. Electronic Commerce Research and Applications, 44, 101011. https://doi.org/10.1016/j.elerap.2020.101011
54. McFarlane, A., Sparrowhawk, A., & Heald, Y. (2002). Report on the educational use of games. TEEM (Teachers evaluating educational multimedia). Cambridge.
55. Mezak, J., & Papak, P.P. (2018). Learning scenarios and encouraging algorithmic thinking. In 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) (pp. 0760 0765). IEEE. https://doi.org/10.23919/MIPRO.2018.8400141
56. Mumcu, H.Y., & Yıldız, S. (2018). The investigation of algorithmic thinking skills of 5th and 6th graders according to different variables. MATDER Journal of Mathematics Education, 3(2), 18-26. https://dergipark.org.tr/en/pub/med/issue/41621/453450
57. Özçelik, D.A. (2013). Assessment and Evaluation in Schools Teacher's Handbook. Pegem Academy.
58. Özden, B., & Yenice, N. (2021). Developing a scientific inquiry skills test for secondary school 7th and 8th grade students. Journal of Mersin University Faculty of Education, 17(1), 112-131. https://doi.org/10.17860/mersinefd.726360
59. Paloma, G.F. (Ed.). (2017). Embodied Cognition. Theories and applications in education science. Nova Science Publishers. https://hdl.handle.net/11386/4712878
60. Paspallis, N., Kasenides, N., & Piki, A. (2022). A Software Architecture for Developing Distributed Games that Teach Coding and Algorithmic Thinking. In 2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC) (pp. 101-110). IEEE. https://doi.org/10.1109/COMPSAC54236.2022.00023
61. Pivec, M., & Kearney, P. (2007). Games for learning and learning from games. Informatica, 31(4).
62. Prensky, M. (2003). Digital game-based learning. Computers in Entertainment (CIE), 1(1), 21-21.
63. Sarı, U., Pektaş, H.M., Şen, Ö.F., & Çelik, H. (2022). Algorithmic thinking development through physical computing activities with Arduino in STEM education. Education and Information Technologies, 1-21. https://doi.org/10.1007/s10639-022-10893-0
64. Scharf, F., Winkler, T., & Herczeg, M. (2008). Tangicons: Algorithmic reasoning in a collaborative game for children in kindergarten and first class. Paper presented at the 7th International Conference on Interaction Design and Children, USA. http://dx.doi.org/10.1145/1463689.1463762
65. Shang, J., Ma, S., Hu, R., Pei, L., & Zhang, L. (2019). Game-based learning in future school. In shaping future schools with digital technology. Springer. http://dx.doi.org/10.1007/978-981-13-9439-3_8
66. Sungkaew, K., Lungban, P., & Lamhya, S. (2022). Game development software engineering: digital educational game promoting algorithmic thinking. International Journal of Electrical and Computer Engineering (IJECE), 12(5), 5393 5404. http://dx.doi.org/10.11591/ijece.v12i5.pp5393-5404
67. Şardağ, M., & Kocakülah, A. (2016). Developing a science process skills test for eighth grade students. Journal of Sakarya University Faculty of Education, 31, 1 32. https://dergipark.org.tr/tr/pub/sakaefd/issue/24690/261073
68. Taasoobshirazi, G., & Carr, M. (2008). A review and critique of context-based physics instruction and assessment. Educational Research Review, 3(2), 155 167. https://doi.org/10.1016/j.edurev.2008.01.002
69. Thomas, J.O., Odemwingie, O.C., Saunders, Q., & Watlerd, M. (2015). Understanding the difficulties African-American middle school girls face while enacting computational algorithmic thinking in the context of game design.
70. Thomas, J.O., Rankin, Y., Minor, R., & Sun, L. (2017). Exploring the difficulties African-American middle school girls face enacting computational algorithmic thinking over three years while designing games for social change. Computer Supported Cooperative Work (CSCW), 26(4), 389-421. https://doi.org/10.1007/s10606-017-9292-y
71. Tsukamoto, H., Oomori, Y., Nagumo, H., Takemura, Y., Monden, A., & Matsumoto, K.I. (2017). Evaluating algorithmic thinking ability of primary schoolchildren who learn computer programming. In 2017 IEEE Frontiers in Education Conference (FIE) (pp. 1-8). IEEE. http://dx.doi.org/10.1109/FIE.2017.8190609
72. Turchi, T., Fogli, D., & Malizia, A. (2019). Fostering computational thinking through collaborative game-based learning. Multimedia Tools and Applications, 78(10), 13649-13673. https://doi.org/10.1007/s11042-019-7229-9
73. Turnuklu, A. (2000). A qualitative research technique that can be used effectively in educational research: Interview. Educational management in theory and practice, 24(24), 543-559.
74. Vasconcelos, J. (2007). Basic Strategy for Algorithmic Problem Solving. http://www.cs.jhu.edu/~jorgev/cs106/ProblemSolving.html
75. Wangenheim, C., Medeiros, G., Missfeldt Filho, R., Petri, G., da Cruz Pinheiro, F., Ferreira, M.N., & Hauck, J.C. (2019). SplashCode--A Board Game for Learning an Understanding of Algorithms in Middle School. Informatics in Education, 18(2), 259-280. http://dx.doi.org/10.31235/osf.io/2qbnp
76. Wilson, M. (2002). Six views of embodied cognition. Psychonomic bulletin & review, 9, 625-636. https://doi.org/10.3758/BF03196322
77. Yadav, A., Gretter, S., Good, J., McLean, T. (2017). Computational Thinking in Teacher Education. In: Rich, P., Hodges, C. (eds) Emerging Research, Practice, and Policy on Computational Thinking. Educational Communications and Technology: Issues and Innovations. Springer, Cham. https://doi.org/10.1007/978-3-319-52691-1_13
78. Yıldırım, A., & Şimşek, H. (2006). Qualitative research methods in the social sciences. Seçkin Publishing.
79. Yılmaz, E.A. (2020). By the power of games: an introduction to the science of gamification. Epsilon Publishing House. https://dergipark.org.tr/tr/pub/tk/issue/56680/784921
80. Zsakó, L., & Szlávi, P. (2012). ICT Competences: Algorithmic Thinking. Acta Didactica Napocensia, 5(2), 49-58.