Investigating different learning supports in exploratory serious games

Abstract

This study aimed to investigate effects of four versions of an exploratory serious game with four different instructional supports on novice students’ developments of knowledge in object-oriented programming. Types of instructional supports integrated into each version of the game are (a) foreshadowing and pedagogical agent support, (b) only pedagogical agent support, (c) pedagogical agent and product-goal support, and (d) pedagogical agent and peer collaboration support. The study collected both quantitative and qualitative data; a pre and a post achievement test, an online self-regulation scale and a learning object evaluation scale provided quantitative data, while a semi-structured interview with selected students provided qualitative data. In addition to four experimental groups, a control group who studied the same learning content with a teacher only was also included. The analysis of those five groups’ data (n=149) revealed that different instructional supports in an exploratory serious game have different effects on learning. Further analysis was conducted to unveil interaction between instructional supports, level of online self-regulation and value attribution to the serious game. Finally, the study discussed the use of instructional supports in exploratory serious games, and provided suggestions on further research questions,

Keywords

• Pedagogical agent
• collaboration
• foreshadowing
• product-goal example
• serious games

Supporting Institution

Bogazici University

Project Number

BAP 22D02P1

Ethical Statement

The research reported in this document has been approved by the research ethics committee of the authors' university.

Thanks

Bogazici University

Açınsayıcı Eğitsel Oyunlarda Öğrenme Destek Türlerinin İncelenmesi

Abstract

Bu çalışma, açınsayıcı bir dijital eğitsel oyun ortamının bilgisayar programlamada yeni olan öğrencilerin nesne yönelimli programlama kavramsal bilgisi geliştirmelerine etkisini dört değişik öğrenme destek senaryosunda incelemeyi amaçlamıştır. Dört farklı sürüm olarak geliştirilen eğitsel oyunda öğrenme destekleri, (a) önceden gösterme ve eğitsel kılavuz desteği, (b) sadece eğitsel kılavuz desteği, (c) eğitsel kılavuz ve öğrenme ürününe özgü destek ve (d) eğitsel kılavuz ve akran iş birliği olarak çeşitlendirilmiştir. Çalışmanın nicel verileri ön ve son başarı testleri, çevrimiçi öz-düzenleme ölçeği ve öğrenme nesnesi değerlendirme ölçeği ile toplanmış, nitel verileri ise deney grubundaki bazı öğrencilerle yapılan yarı yapılandırılmış görüşmelerle toplanmıştır. Çalışmaya aynı içeriği oyun ortamı dışında öğretmenle çalışan bir kontrol grubu da dahil edilmiştir. Dört deney ve bir kontrol grubundaki (n=149) öğrencilerden elde edilen verilerin çözümlemesi, açınsayıcı dijital oyun ortamında kullanılan öğrenme destek türlerinin öğrenmeye farklı etkisine işaret etmiştir. Çalışmada ayrıca öğrenme destek türü, çevrimiçi öz düzenleme ve öğrenme nesnesi değerlendirme değişkenlerinin etkileşimi istatistiksel olarak incelenmiştir. Çalışma açınsayıcı dijital eğitsel oyunlarda öğrenme desteklerinin kullanımı üzerine bir dizi öneri ve olası araştırma konularını tartışmıştır.

Keywords

• Eğitsel kılavuz
• iş birliği
• önden gösterme
• öğrenme ürünü örneklendirme
• ciddi oyun
• programlama

Supporting Institution

Boğaziçi Üniversitesi BAP

Project Number

BAP 22D02P1

Ethical Statement

Bu araştırma yürütülmeden önce araştırma için etik kurul onayı alınmış olup, istendiğinde ilgili belge sunulacaktır.

Thanks

Boğaziçi Üniversitesi

References

1. Abbasi, S., Kazi, H., Kazi, A. W., Khowaja, K., & Baloch, A. (2021). Gauge NYP in student’s learning performance, normalized learning gains and perceived motivation with serious games. Information, 12(3), 101. https://doi.org/10.3390/info12030101
2. Adams, D. M., Mayer, R. E., MacNamara, A., Koenig, A., & Wainess, R. (2012). Narrative games for learning: Testing the discovery and narrative hypotheses. Journal of Educational Psychology, 104(1), 235. https://doi.org/10.1037/a0025595
3. Ak, O., & Kutlu, B. (2017). Comparing 2D and 3D game‐based learning environments in terms of learning gains and student perceptions. British Journal of Educational Technology, 48(1), 129-144.
4. Akbulut, Y. (2010). Sosyal Bilimlerde SPSS Uygulamaları. İstanbul. İdeal.
5. Akkaya, A., & Akpınar, Y. (2022). Experiential serious-game design for development of knowledge of OOP and computational thinking skills, Computer Science Education, 32(4), 476-501.
6. Akpınar, Y., & Turan, M. (2012). Designing a collaborative learning game: Its validation with a turn-taking control scheme in a primary science unit. Education & Science. 37(163), 254-267.
7. Akpınar, Y., (2005). Bilgisayar Destekli Öğretim ve Uygulamalar. Ankara. Anı Yayıncılık.
8. Akpınar, Y., Kutbay, E., & Akkaya, A. (2023). Designing exploratory serious games with learning supports. Journal of Theory & Practice in Education, 19(1), 83-96. https://doi.org/10.17244/eku.1248565

9. Alfieri, L., Brooks, P. J., Aldrich, N. J., & Tenenbaum, H. R. (2011). Does discovery-based instruction enhance learning? Journal of Educational Psychology, 103(1), 1–18
10. Alpert, J. B., Young, M. G., Lala, S. V., & McGuinness, G. (2021). Medical student engagement and educational value of a remote clinical radiology learning environment. Academic Radiology, 28(1), 112-118.
11. Bainbridge, K., Shute, V., Rahimi, S., Liu, Z., Slater, S., Baker, R. S., & D'Mello, S. K. (2022). Does embedding learning supports enhance transfer during game-based learning? Learning and Instruction, 77, 101547. https://doi.org/10.1016/j.learninstruc.2021.101547
12. Brünken, R., Plass, J. L., & Leutner, D. (2003). Direct measurement of cognitive load in multimedia learning. Educational Psychologist, 38(1), 53–61.
13. Cai, Z., Mao, P., Wang, D., He, J., Chen, X., & Fan, X. (2022). Effects of scaffolding in digital game-based learning on student’s achievement: A three-level meta-analysis. Educational Psychology Review, 34(2), 537-574.
14. Çakmak, E. K. (2007). Çoklu ortamlarda dar boğaz: Aşırı bilişsel yüklenme. Gazi Eğitim Fakültesi Dergisi, 27(2), 1-24.
15. Chen, C. H., & Law, V. (2016). Scaffolding individual and collaborative game-based learning in learning performance and intrinsic motivation. Computers in Human Behavior, 55, 1201–1212.
16. Chen, Z. H., & Chen, S. Y. (2014). When educational agents meet surrogate competition: Impacts of competitive educational agents on students' motivation and performance. Computers & Education, 75, 274-281.
17. Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods approaches. CA: Sage
18. Creswell, J. W. (2013). Educational research (5th ed.). MA: Pearson.
19. Csikszentmihalyi, M. (2014). Toward a psychology of optimal experience. In M. Csikszentmihalyi, Flow and the foundations of positive psychology (pp. 209-226). Dordrecht, Springer. https://doi.org/10.1007/978-94-017-9088-8_14
20. Engeström, Y. (1987). Learning by expanding. Helsinki: Orienta-Konsultit.
21. Fiorella, L., & Mayer, R. E. (2012). Paper-based aids for learning with a computer-based game. Journal of Educational Psychology, 104(4), 1074–1082.
22. Florea, A., Gellert, A., Florea, D., & Florea, A.-C. (2016). Teaching programming by developing games in Alice. In I. Roceanu, D. Dubois, D. Beligan, F. Moldoveanu, M. I. Dascalu, I. Stanescu, & D. Barbieru (Ed.), The International Scientific Conference eLearning and Software for Education. 1, pp. 503-510. Bucharest: "Carol I" National Defence University Publishing House.
23. Furtak E. M., Seidel T., Iverson H., & Briggs D. C. (2012). Experimental and quasi-experimental studies of inquiry-based science teaching: A meta-analysis. Review of Educational Research, 82(3), 300–329.
24. Hawlitschek, A., & Joeckel, S. (2017). Increasing the effectiveness of digital educational games: The effects of a learning instruction on students’ learning, motivation and cognitive load. Computers in Human Behavior, 72, 79-86.
25. Herder, T., & Rau, M. A. (2022). Representational-competency supports in the context of an educational video game for undergraduate astronomy. Computers & Education, 190, 104602. https://doi.org/10.1016/j.compedu.2022.104602
26. Hwang, W. Y., Manabe, K., & Huang, T. H. (2023). Collaborative guessing game for EFL learning with kinesthetic recognition. Thinking Skills and Creativity, 48, 101297. https://doi.org/10.1016/j.tsc.2023.101297
27. Iten, N.,& Petko, D. (2014). Learning with serious games: Is fun playing the game a predictor of learning success? British Journal of Educational Technology, 47(1), 151-163
28. Keller, J. M. (2010). Motivational design for learning and performance. NY: Springer.
29. Kiili, K. (2005). Digital game-based learning: Towards an experiential gaming model. Internet and Higher Education, 8, 13-24.
30. Kilis, S., & Yildirim, Z. (2018). Online self-regulation questionnaire: Validity and reliability study. Cukurova University Faculty of Education Journal, 47(1), 233-245.
31. Lazonder, A. W., & Harmsen, R. (2016). Meta-analysis of inquiry-based learning: Effects of guidance. Review of Educational Research, 86(3), 681–718.
32. Lister, R. (2011). Programming, syntax and cognitive load (part 2). ACM Inroads, 2(2), 16-17.
33. Liu, Y., Zheng, X.,& Hau, K. T. (2023). Would emphasizing the instrumental value of learning help unmotivated students? Large-scale cross-cultural comparisons. Personality and Individual Differences, 207, 112148. https://doi.org/10.1016/j.paid.2023.112148
34. Liu, Z., & Jeong, A. C. (2022). Connecting learning and playing: the effects of in-game cognitive supports on the development and transfer of computational thinking skills. Educational Technology Research and Development, 70(5), 1867-1891.
35. Mathrani, A., Christian, S., & Ponder-Sutton, A. (2016). Playit: Game based learning approach for teaching programming concepts. Educational Technology and Society, 19(2), 5-17.
36. Mayer, R. E., Mathias, A., & Wetzell, K. (2002). Fostering understanding of multimedia messages through pre-training: Evidence for a two-stage theory of mental model construction. Journal of Experimental Psychology: Applied, 8(3), 147-154. Metwally, A. H. S., Nacke, L. E., Chang, M., Wang, Y., & Yousef, A. M. F. (2021). Revealing the hotspots of educational gamification: An umbrella review. International Journal of Educational Research, 109, 101832. https://doi.org/10.1016/j.ijer.2021.101832
37. Noroozi, O., Dehghanzadeh, H., & Talaee, E. (2020). A systematic review on the impacts of game-based learning on argumentation skills. Entertainment Computing, 35, 100369. https://doi.org/10.1016/j.entcom.2020.100369
38. Paas, F., & Van Merrienboer, J. J. G. (1994). Instructional control of cognitive load in the training of complex cognitive tasks. Educational Psychology Review, 6, 351-372.
39. Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory. Instructional Science, 32, 1-8.
40. Paas, F., Tuovinen, J. E., Tabbers, H. & Van Germen, P. W. M. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38(1), 63-71.
41. Pan, Y., & Ke, F. (2023). Effects of game-based learning supports on students’ math performance and perceived game flow. Education Technology Research & Development, 71, 459–479.
42. Parong, J., & Mayer, R. E. (2018). Learning science in immersive virtual reality. Journal of Educational Psychology, 110(6), 785–797.
43. Pedrosa, D., Cravino, J., Morgado, L., & Barreira, C. (2016). Self-regulated learning in programming. In Proceedings of Int. Conference on Immersive Learning (pp. 87-101). Springer, Cham. https://doi.org/10.1007/978-3-319-41769-1_7 Pol, H. J., Harskamp, E. G., & Suhre, C. J. (2008). The effect of the timing of instructional support in a computer-supported problem-solving program for students in secondary physics education. Computers in Human Behavior, 24(3), 1156-1178.
44. Prensky, M. (2001). Digital Game-Based Learning. McGraw-Hill, NY.
45. Rahimi, S., Shute, V. J., Fulwider, C., Bainbridge, K., Kuba, R., Yang, X., ... & D'Mello, S. K. (2022). Timing of learning supports in educational games can impact students’ outcomes. Computers & Education, 190, 104600. https://doi.org/10.1016/j.compedu.2022.104600
46. Repenning, A., Webb, D., & Ioannidou, A. (2010). Scalable game design and the development of a checklist for getting computational thinking into public schools. In G. Lewandowski, S. Wolfman, T. J. Cortina, & E. L. Walker (Ed.), Proceedings of the 41st ACM Technical Symposium on Computer Science Education (pp. 265-269). NY: ACM. https://doi.org/10.1145/1734263.1734357
47. Ter Vrugte, J., de Jong, T., Vandercruysse, S., Wouters, P., van Oostendorp, H., & Elen, J. (2015). How competition and heterogeneous collaboration interact in prevocational game-based math education. Computers & Education, 89, 42– 52.
48. Tokac, U., Novak, E., & Thompson, C. G. (2019). Effects of game‐based learning on students' mathematics achievement: A meta‐analysis. Journal of Computer Assisted Learning, 35(3), 407-420.
49. Tsai, F. H., Kinzer, C., Hung, K. H., Chen, C. L. A., & Hsu, I. Y. (2013). The importance and use of targeted content knowledge with scaffolding aid in educational simulation games. Interactive Learning Environments, 21(2), 116-128.
50. Van den Hurk, A., Meelissen, M., & van Langen, A. (2019). Interventions in education to prevent STEM pipeline leakage. International Journal of Science Education, 41(2), 150-164.
51. Van Der Meij, H., Leemkuil, H., & Li, J. L. (2013). Does individual or collaborative self-debriefing better enhance learning from games? Computers in human behavior, 29(6), 2471-2479. Van Der Meij, H., Veldkamp, S., & Leemkuil, H. (2020). Effects of scripting on dialogues, motivation and learning outcomes in serious games. British Journal of Educational Technology, 51(2), 459-472.
52. Van Merriënboer, J. J., Clark, R. E., & de Croock, M. B. (2002). Blueprints for complex learning: The 4C/ID-model. Educational Technology Research and Development, 50(2), 39-61.
53. Vandercruysse, S., ter Vrugte, J., de Jong, T., Wouters, P., van Oostendorp, H., Verschaffel, L., ... & Elen, J. (2016). The effectiveness of a math game: The impact of integrating conceptual clarification as support. Computers in Human Behavior, 64, 21-33.
54. Weingärtner, M., & Weingärtner, T. (2023). Quantum Tic-Tac-Toe-learning the concepts of quantum mechanics in a playful way. Computers and Education Open, 4, 100125. https://doi.org/10.1016/j.caeo.2023.100125 Westera, W. (2019). Why and how serious games can become far more effective: Accommodating productive learning experiences, learner motivation and the monitoring of learning gains. Journal of Educational Technology & Society, 22(1), 59-69.
55. Wouters, P., & van Oostendorp, H. (2013). A meta-analytic review of the role of instructional support in game-based learning. Computers & Education, 60(1), 412-425.
56. Wouters, P., Van Oostendorp, H., Boonekamp, R., & Van der Spek, E. (2011). The role of game discourse analysis and curiosity in creating engaging and effective serious games by implementing a back story and foreshadowing. Interacting with Computers, 23(4), 329-336.
57. Xinogalos, S. (2016). Designing and deploying programming courses. Education and Information Technologies, 21(3), 559-588. Yaman, M., Nerdel, C., & Bayrhuber, H. (2008). The effects of instructional support and learner interests when learning using computer simulations. Computers & Education, 51(4), 1784-1794.
58. Yang, X., Rahimi, S., Shute, V., Kuba, R., Smith, G., & Alonso-Fernández, C. (2021). The relationship among prior knowledge, accessing learning supports, learning outcomes, and game performance in educational games. Educational Technology Research and Development, 69(2), 1055–1075.
59. Zhi, R., Lytle, N., & Price, T. W. (2018). Exploring instructional support design in an educational game for K-12 computing education. In Proceedings of the 49th ACM technical symposium on computer science education (pp. 747-752). SIGCSE’18, February 21-24, Baltimore, MD, USA. https://doi.org/10.1145/3159450.3159519
60. Zhonggen, Y. (2019). A meta-analysis of use of serious games in education over a decade. International Journal of Computer Games Technology. https://doi.org/10.1155/2019/4797032
61. Zimmerman, B. J. (2013). From cognitive modeling to self-regulation: A social cognitive career path. Educational Psychology, 48(3), 135–147.
62. Zumbach, J., Rammerstorfer, L., & Deibl, I. (2020). Cognitive and metacognitive support in learning with a serious game about demographic change. Computers in Human Behavior, 103, 120-129.