Developing a game-based test to assess middle school sixth-grade students’ algorithmic thinking skills

Year 2024, Volume: 11 Issue: 1, 88 - 108, 16.03.2024

Emre Zengin , Yasemin Karal

https://doi.org/10.21449/ijate.1327082

Abstract

This study was carried out to develop a test to assess algorithmic thinking skills. To this end, the twelve steps suggested by Downing (2006) were adopted. Throughout the test development, 24 middle school sixth-grade students and eight experts in different areas took part as needed in the tasks on the project. The test was given to 252 students attending the sixth grade who were selected through purposeful sampling. The content validity of the test was ensured by means of obtaining expert opinion, whereas the construct validity was ensured by performing an independent sample t-test on the difference between the lower and upper groups. As a result, the algorithmic thinking skills assessment test was finalized with 22 main items and 2 sample items, totalling 24 items. The KR-20 reliability analysis proved a quite reliable test based on the reliability coefficient of 0.83. As mentioned earlier, the independent sample t-test was applied to the difference of lower and upper groups for construct validation of the test. It was seen that the test items are significant in discriminating the students in the lower and upper groups (p<0.01).

Keywords

Algorithmic thinking, Test, Middle school, Game, Embodied cognition

Ethical Statement

Trabzon University, 13.08.2021, E-81614018-000-704.

References

• Altakrouri, B., & Schrader, A. (2012, September). Towards dynamic natural interaction ensembles. In the 26th BCS Conference on Human Computer Interaction 26 (pp. 1-10). http://dx.doi.org/10.14236/ewic/HCI2012.0
• Anderson, M.L. (2003). Embodied cognition: A field guide. Artificial intelligence, 149(1), 91-130. https://doi.org/10.1016/S0004-3702(03)00054-7
• Apostolellis, P., Stewart, M., Frisina, C., & Kafura, D. (2014). RaBit EscAPE: A board game for computational thinking. In Proceedings of the 2014 conference on Interaction design and children (pp. 349-352). http://dx.doi.org/10.1145/2593968.2610489
• Ayala, N.A.R., Mendívil, E.G., Salinas, P., & Rios, H. (2013). Kinesthetic learning applied to mathematics using kinect. Procedia Computer Science, 25, 131 135. https://doi.org/10.1016/j.procs.2013.11.016
• Aytekin, A., Çakır, F.S., Yücel, Y.B., & Kulaözü, İ. (2018). The place and importance of algorithms in our lives. Eurasian Journal of Social and Economic Studies, 5(7), 143 150. https://dergipark.org.tr/tr/pub/asead/issue/41013/495619
• Başol, G. (2019). Measurement and evaluation in education. Pegem Citation Index, 001-307.
• Baykul, Y., Gelbal, S., & Kelecioğlu, H. (2003). Measurement and evaluation in education for Anatolian teacher high schools. National Education Printing House.
• Bayrakçeken, S. (2015). Test development. E. Karip (Ed.), In Measurement and Evaluation (s. 292-322). Pegem Academy.
• Bellocchi, A., King, D.T., & Ritchie, S.M. (2016). Context-based assessment: Creating opportunities for resonance between classroom fields and societal fields. International Journal of Science Education, 38(8), 1304 1342. https://doi.org/10.1080/09500693.2016.1189107
• Borkulo, S., Chytas, C., Drijvers, P., Barendsen, E., & Tolboom, J. (2021). Computational thinking in the mathematics classroom: Fostering algorithmic thinking and generalization skills using dynamic mathematics software. In The 16th Workshop in Primary and Secondary Computing Education (pp. 1 9). https://doi.org/10.1145/3481312.3481319.
• Brown, W. (2015). Introduction to algorithmic thinking. https://raptor.martincarlisle.com/Introduction%20to%20Algorithmic%20Thinking.doc
• Büyüköztürk, Ş., Kılıç-Çakmak, E., Akgün, Ö.E., Karadeniz, Ş., & Demirel, F. (2020). Scientific research methods. Pegem Publications.
• Chen, K.Z., & Chi, H.H. (2022). Novice young board-game players’ experience about computational thinking. Interactive Learning Environments, 30(8), 1375-1387. https://doi.org/10.1080/10494820.2020.1722712
• Chuechote, S., Nokkaew, A., Phongsasithorn, A., & Laosinchai, P. (2020). A neo-piagetian analysis of algorithmic thinking development through the “sorted” digital game. Contemporary Educational Technology, 12(1), 1 15. http://dx.doi.org/10.30935/cet.685959
• Czakóová, K. (2020). Developing algorithmic thinking by educational computer games. Paper presented at the Conference eLearning and Software for Educationtional, Romania. http://dx.doi.org/10.12753/2066-026X-20-003
• Czakóová, K., & Udvaros, J. (2021). Applications and games for the development of algorithmic thinking in favor of experiential learning. In EDULEARN21 Proceedings (pp. 6873-6879). IATED. https://doi.org/10.21125/edulearn.2021.1389
• Çetin, B. (Ed.) (2019). Measurement and evaluation in education. Anı Publishing.
• Debabi, W., & Bensebaa, T. (2016). Using serious game to enhance learning and teaching algorithmic. Journal of e Learning and Knowledge Society, 12(2). http://dx.doi.org/10.20368/1971-8829/1125
• Demir, O., & Köse, İ.A. (2014). Comparison of cutoff scores determined by Angoff, Nedelsky and Ebel standard setting methods. Journal of Mersin University Faculty of Education, 10(2), 14 27. https://dergipark.org.tr/en/pub/mersinefd/issue/17394/181823?publisher=mersin
• Doleck, T., Bazelais, P., Lemay, D.J., Saxena, A., & Basnet, R.B. (2017). Algorithmic thinking, cooperativity, creativity, critical thinking, and problem solving: Exploring the relationship between computational thinking skills and academic performance. Journal of Computers in Education, 4(4), 355-369. https://doi.org/10.1007/s40692-017-0090-9
• Downing, S.M. (2006). Twelve steps for effective test development. Handbook of test development, 3, 25. https://doi.org/10.4324/9780203874776.ch1
• Elshahawy, M., Aboelnaga, K., & Sharaf, N. (2020). Codaroutine: A serious game for introducing sequential programming concepts to children with autism. In 2020 IEEE Global Engineering Education Conference (EDUCON) (pp. 1862 1867). IEEE. http://dx.doi.org/10.1109/EDUCON45650.2020.9125196
• Erümit, K.A., Karal, H., Şahin, G., Aksoy, D.A., Aksoy, A., & Benzer, A.İ. (2018). A model proposal for teaching programming: programming in seven steps. Education and Science, 44(197), 1-29. http://dx.doi.org/10.15390/EB.2018.7678
• Evripidou, S., Amanatiadis, A., Christodoulou, K., & Chatzichristofis, S.A. (2021). Introducing algorithmic thinking and sequencing using tangible robots. IEEE Transactions on Learning Technologies, 14(1), 93 105. https://doi.org/10.1109/TLT.2021.3058060
• Fensham, P.J., & Rennie, L.J. (2013). Towards an authentically assessed science curriculum. In Valuing assessment in science education: Pedagogy, curriculum, policy (pp. 69-100). Springer.
• Figueiredo, M., Amante, S., Gomes, H.M.D.S.V., Gomes, M.A., Rego, B., Alves, V., & Duarte, R.P. (2021). Algorithmic thinking in early childhood education: Opportunities and supports in the Portuguese context. In EDULEARN21 Proceedings (pp. 9339-9348). IATED. https://doi.org/10.22521/edupij.2022.112.3
• Futschek, G., & Moschitz, J. (2010). Developing algorithmic thinking by inventing and playing algorithms. Proceedings of the 2010 constructionist approaches to creative learning, thinking and education: Lessons for the 21st century (constructionism 2010), 1-10.
• Gall, M.D., Barg, W.R., & Gall, j.P. (1996). Educational research: an introduction (6th ed.). Longman Publishing.
• Geisinger, K.F. (2016). 21st century skills: What are they and how do we assess them?. Applied Measurement in Education, 29(4), 245 249. https://doi.org/10.1080/08957347.2016.1209207
• Güler, N. (2015). Measurement and evaluation in education (7th ed.). Pegem Academy Publishing.
• Gürbüz, H., Evlioğlu, B., Erol, Ç.S., Gülseçen, H., & Gülseçen, S. (2017). “What’s the weather like today?”: A computer game to develop algorithmic thinking and problem solving skills of primary school pupils. Education and Information Technologies, 22(3), 1133-1147. https://doi.org/10.1007/s10639-016-9478-9
• Hsu, C.C., & Wang, T.I. (2018). Applying game mechanics and student-generated questions to an online puzzle-based game learning system to promote algorithmic thinking skills. Computers & Education, 121, 73 88. http://dx.doi.org/10.1016/j.compedu.2018.02.002
• Hu, C. (2011). Computational thinking: What it might mean and what we might do about it. In Proceedings of the 16th annual joint conference on innovation and technology in computer science education (pp. 223 227). ACM. http://dx.doi.org/10.1145/1999747.1999811
• Huizinga, J. (1955). Homo ludens: A study of the play element in culture. The Beacon Press. https://doi.org/10.2307/2087716
• Iacolina, S.A., Lai, A., Soro, A., & Scateni, R. (2010). Natural interaction and computer graphics applications. In Eurographics italian chapter conference (pp. 141–146).
• Jančec, L., & Vujičić, L. (2021). Project “Algorithmic Thinking Skills through Play-Based Learning for Future's Code Literates”. In 2021 44th International Convention on Information, Communication and Electronic Technology (MIPRO) (pp. 641-644). IEEE. https://doi.org/10.23919/MIPRO52101.2021.9597078
• Kanaki, K., & Kalogiannakis, M. (2022). Assessing algorithmic thinking skills in relation to gender in early childhood. Educational Process: International Journal, 11, 44 59. https://dx.doi.org/10.22521/edupij.2022.112.3
• Karakasis, C., & Xinogalos, S. (2020). BlocklyScript: Design and pilot evaluation of an RPG platform game for cultivating computational thinking skills to young students. Informatics in Education, 19(4), 641 668. https://doi.org/10.15388/infedu.2020.28
• Karatay, R., & Doğan, F. (2016). Development of science process skill scale of 7th grade secondary school students. Dicle Üniversitesi Ziya Gökalp Eğitim Fakültesi Dergisi, 27, 1-8. http://dx.doi.org/10.14582/DUZGEF.548
• Kátai, Z. (2015). The challenge of promoting algorithmic thinking of both sciences‐and humanities‐oriented learners. Journal of Computer Assisted Learning, 31(4), 287-299. https://doi.org/10.1111/jcal.12070
• Kazimoglu, C. (2020). Enhancing confidence in using computational thinking skills via playing a serious game: A case study to increase motivation in learning computer programming. IEEE Access, 8, 221831 221851. http://dx.doi.org/10.1109/ACCESS.2020.3043278
• Keane, T. (2012). Leading with technology: 21st century skills= 3Rs+ 4Cs. Australian Educational Leader, 34(2), 44. https://search.informit.org/doi/10.3316/informit.895597893475453
• Khanlari, A. (2013). Effects of robotics on 21st century skills. European Scientific Journal, 9(27). https://doi.org/10.19044/esj.2013.v9n27p%25p
• Kim, B., Park, H., & Baek, Y. (2009). Not just fun, but serious strategies: Using meta-cognitive strategies in game based learning. Computers & Education, 52(4), 800 810. https://doi.org/10.1016/j.compedu.2008.12.004
• Kiss, G., & Arki, Z. (2017). The influence of game-based programming education on the algorithmic thinking. Procedia Social and Behavioral Sciences, 237, 613 617. http://dx.doi.org/10.1016/j.sbspro.2017.02.020
• Kocagül-Sağlam, M. & Ünal-Çoban, G. (2019). Developing the reasoning skills test for science teachers and teacher candidates. Elementary Education Online, 17(3). https://doi.org/10.17051/ilkonline.2018.466374
• Kosmas, P., Ioannou, A., & Retalis, S. (2018). Moving bodies to moving minds: A study of the use of motion-based games in special education. TechTrends, 62, 594-601. https://doi.org/10.1007/s11528-018-0294-5
• Kosmas, P., & Zaphiris, P. (2023). Improving students’ learning performance through Technology Enhanced Embodied Learning: A four year investigation in classrooms. Education and Information Technologies, 1 24. https://doi.org/10.1007/s10639-022-11466-x
• La Belle, T.J., Moll, L.C., & Weisner, T.S. (1979). Context-based educational evaluation: A participant research strategy. Educational Evaluation and Policy Analysis, 1(3), 85-94. https://doi.org/10.2307/1164160
• Lee, T.Y., Mauriello, M.L., Ahn, J., & Bederson, B.B. (2014). CTArcade: Computational thinking with games in school age children. International Journal of Child-Computer Interaction, 2(1), 26-33. http://dx.doi.org/10.1016/j.ijcci.2014.06.003
• Lertlapnon, T., Lueangrungudom, N., & Vittayakorn, S. (2022). Protobot: An Educational Game for Algorithmic Thinking. In 2022 14th International Conference on Information Technology and Electrical Engineering (ICITEE) (pp. 79 84). IEEE. https://doi.org/10.1109/ICITEE56407.2022.9954081
• Li, J., Lin, Y., Sun, M., & Shadiev, R. (2020). Socially shared regulation of learning in game-based collaborative learning environments promotes algorithmic thinking, learning participation and positive learning attitudes. Interactive Learning Environments, 1 12. https://doi.org/10.1080/10494820.2020.1857783
• Lin, S.Y., Chien, S.Y., Hsiao, C.L., Hsia, C.H., & Chao, K.M. (2020). Enhancing computational thinking capability of preschool children by game-based smart toys. Electronic Commerce Research and Applications, 44, 101011. https://doi.org/10.1016/j.elerap.2020.101011
• McFarlane, A., Sparrowhawk, A., & Heald, Y. (2002). Report on the educational use of games. TEEM (Teachers evaluating educational multimedia). Cambridge.
• Mezak, J., & Papak, P.P. (2018). Learning scenarios and encouraging algorithmic thinking. In 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) (pp. 0760 0765). IEEE. https://doi.org/10.23919/MIPRO.2018.8400141
• Mumcu, H.Y., & Yıldız, S. (2018). The investigation of algorithmic thinking skills of 5th and 6th graders according to different variables. MATDER Journal of Mathematics Education, 3(2), 18-26. https://dergipark.org.tr/en/pub/med/issue/41621/453450
• Özçelik, D.A. (2013). Assessment and Evaluation in Schools Teacher's Handbook. Pegem Academy.
• Özden, B., & Yenice, N. (2021). Developing a scientific inquiry skills test for secondary school 7th and 8th grade students. Journal of Mersin University Faculty of Education, 17(1), 112-131. https://doi.org/10.17860/mersinefd.726360
• Paloma, G.F. (Ed.). (2017). Embodied Cognition. Theories and applications in education science. Nova Science Publishers. https://hdl.handle.net/11386/4712878
• Paspallis, N., Kasenides, N., & Piki, A. (2022). A Software Architecture for Developing Distributed Games that Teach Coding and Algorithmic Thinking. In 2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC) (pp. 101-110). IEEE. https://doi.org/10.1109/COMPSAC54236.2022.00023
• Pivec, M., & Kearney, P. (2007). Games for learning and learning from games. Informatica, 31(4).
• Prensky, M. (2003). Digital game-based learning. Computers in Entertainment (CIE), 1(1), 21-21.
• Sarı, U., Pektaş, H.M., Şen, Ö.F., & Çelik, H. (2022). Algorithmic thinking development through physical computing activities with Arduino in STEM education. Education and Information Technologies, 1-21. https://doi.org/10.1007/s10639-022-10893-0
• Scharf, F., Winkler, T., & Herczeg, M. (2008). Tangicons: Algorithmic reasoning in a collaborative game for children in kindergarten and first class. Paper presented at the 7th International Conference on Interaction Design and Children, USA. http://dx.doi.org/10.1145/1463689.1463762
• Shang, J., Ma, S., Hu, R., Pei, L., & Zhang, L. (2019). Game-based learning in future school. In shaping future schools with digital technology. Springer. http://dx.doi.org/10.1007/978-981-13-9439-3_8
• Sungkaew, K., Lungban, P., & Lamhya, S. (2022). Game development software engineering: digital educational game promoting algorithmic thinking. International Journal of Electrical and Computer Engineering (IJECE), 12(5), 5393 5404. http://dx.doi.org/10.11591/ijece.v12i5.pp5393-5404
• Şardağ, M., & Kocakülah, A. (2016). Developing a science process skills test for eighth grade students. Journal of Sakarya University Faculty of Education, 31, 1 32. https://dergipark.org.tr/tr/pub/sakaefd/issue/24690/261073
• Taasoobshirazi, G., & Carr, M. (2008). A review and critique of context-based physics instruction and assessment. Educational Research Review, 3(2), 155 167. https://doi.org/10.1016/j.edurev.2008.01.002
• Thomas, J.O., Odemwingie, O.C., Saunders, Q., & Watlerd, M. (2015). Understanding the difficulties African-American middle school girls face while enacting computational algorithmic thinking in the context of game design.
• Thomas, J.O., Rankin, Y., Minor, R., & Sun, L. (2017). Exploring the difficulties African-American middle school girls face enacting computational algorithmic thinking over three years while designing games for social change. Computer Supported Cooperative Work (CSCW), 26(4), 389-421. https://doi.org/10.1007/s10606-017-9292-y
• Tsukamoto, H., Oomori, Y., Nagumo, H., Takemura, Y., Monden, A., & Matsumoto, K.I. (2017). Evaluating algorithmic thinking ability of primary schoolchildren who learn computer programming. In 2017 IEEE Frontiers in Education Conference (FIE) (pp. 1-8). IEEE. http://dx.doi.org/10.1109/FIE.2017.8190609
• Turchi, T., Fogli, D., & Malizia, A. (2019). Fostering computational thinking through collaborative game-based learning. Multimedia Tools and Applications, 78(10), 13649-13673. https://doi.org/10.1007/s11042-019-7229-9
• Turnuklu, A. (2000). A qualitative research technique that can be used effectively in educational research: Interview. Educational management in theory and practice, 24(24), 543-559.
• Vasconcelos, J. (2007). Basic Strategy for Algorithmic Problem Solving. http://www.cs.jhu.edu/~jorgev/cs106/ProblemSolving.html
• Wangenheim, C., Medeiros, G., Missfeldt Filho, R., Petri, G., da Cruz Pinheiro, F., Ferreira, M.N., & Hauck, J.C. (2019). SplashCode--A Board Game for Learning an Understanding of Algorithms in Middle School. Informatics in Education, 18(2), 259-280. http://dx.doi.org/10.31235/osf.io/2qbnp
• Wilson, M. (2002). Six views of embodied cognition. Psychonomic bulletin & review, 9, 625-636. https://doi.org/10.3758/BF03196322
• Yadav, A., Gretter, S., Good, J., McLean, T. (2017). Computational Thinking in Teacher Education. In: Rich, P., Hodges, C. (eds) Emerging Research, Practice, and Policy on Computational Thinking. Educational Communications and Technology: Issues and Innovations. Springer, Cham. https://doi.org/10.1007/978-3-319-52691-1_13
• Yıldırım, A., & Şimşek, H. (2006). Qualitative research methods in the social sciences. Seçkin Publishing.
• Yılmaz, E.A. (2020). By the power of games: an introduction to the science of gamification. Epsilon Publishing House. https://dergipark.org.tr/tr/pub/tk/issue/56680/784921
• Zsakó, L., & Szlávi, P. (2012). ICT Competences: Algorithmic Thinking. Acta Didactica Napocensia, 5(2), 49-58.