The Effect of Gamification on Learner Motivation: A Meta-Analysis Study

Abstract

Although plenty of research has been carried out in order to determine the effect of gamification on learner motivation, a specific meta-analysis hasn’t been encountered in the relevant literature. The purpose of this study was to combine the experimental research results examining the effect of gamification on learner motivation and conducted between the years of 2010-2017 through a meta-analysis. The random effects model was used in the study and the mean value for the effect size was estimated to be Hedges’ g = .54. The obtained Hedges’ g value corresponds to the moderate level of effect. Within the scope of the present study, the sub-group analyses for the type of publication, the country where the research was conducted and the number of elements used in the design were also performed and it was determined that the effect size did not differ significantly according to the relevant sub-groups. Some suggestions were made in line with the research results.

Keywords

• Gamification
• meta-analysis
• learner motivation
• game elements

References

1. Note: References with asterisk (*) indicate studies included to meta-analysis. Ayaz, M. F., & Söylemez, M. (2015). The effect of the project-based learning approach on the academic achievements of the students in science classes in Turkey: A Meta-Analysis Study. Education and Science, 178, 255-283.
2. *Barrio, C. M., Muñoz-Organero, M., & Soriano, J. S. (2016). Can gamification improve the benefits of student response systems in learning? An experimental study. IEEE Transactions on Emerging Topics in Computing, 4(3), 429-438.
3. Bell, K. R. (2014). Online 3.0---the rise of the gamer educator the potential role of gamification in online education (Order No. 3635727). Available from ProQuest Dissertations & Theses Global. (1615085697). Retrieved from http://search.proquest.com/docview/1615085697?accountid=15958
4. Bogost, I. (2011). Why gamification is bullshit. In S. P. Walz & S. Deterding (Eds.), The gameful world: Approaches, issues, applications (pp.65-80). London, England: The MIT Press.
5. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Hoboken, NJ: Wiley.
6. *Brühlmann, F. (2015). The Effects of Framing in Gamification: A Study of Failure. Unpublished Master Thesis. University of Basel, Switzerland.
7. Buckley, P., & Doyle, E. (2016) Gamification and student motivation. Interactive Learning Environments, 24(6), 1162-1175. Doi: 10.1080/10494820.2014.964263.
8. Bunchball, Inc (2010). Gamification 101: An introduction to the use of game dynamics to influence behavior. White paper.

9. Chorney, A. I. (2012). Taking the game out of gamification. Dalhousie Journal of Interdisciplinary Management, 8(1), 1-14.
10. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2. edition). Hillsdale, NJ: Lawrence Earlbaum Associates.
11. Cooper, H., Hedges, L. V., & Valentine, J. C. (2009). The handbook of research synthesis and metaanalysis (2nd edition). New York, NY: Russell Sage Publication.
12. de Sousa Borges, S., Durelli, V. H., Reis, H. M., & Isotani, S. (2014). A systematic mapping on gamification applied to education. SAC'14 (pp. 217-222). ACM.
13. Deci, E., & Ryan, R. (1985). Intrinsic motivation and self-determination in human behavior. New York: Plenum Press.
14. Denny, P. (2013). The effect of virtual achievement son student engagement. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Presented at CHI13'. ACM, pp.763–772.
15. Deterding, S. (2011). Situated motivational affordances of game elements: a conceptual model. In:Gamification: Using Game Design Elements in Non-Gaming Contexts, a Workshop at CHI. Presented atCHI2011. ACM, Vancouver, BC, pp. 1–4.
16. Deterding, S., Dixon, D., Khaled, R. & Nacke. L. (2011). From game design elements to gamefulness: defining "gamification". In Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments (MindTrek '11). ACM, New York, NY, USA, 9-15. DOI=10.1145/2181037.2181040.
17. Dicheva, D., Dichev, C., Agre, G., & Angelova, G. (2015). Gamificatio in education: A Systematic mapping study. Educational Technology & Society, 18(3), 75-88.
18. Domínguez, A., Saenz-De-Navarrete, J., De-Marcos, L., Fernández-Sanz, L., PagéS, C., & Martínez-HerráIz, J. J. (2013). Gamifying learning experiences: Practical implications and outcomes. Computers & Education, 63, 380-392.
19. Domínguez, A., Saenz-de-Navarrete,J., de-Marcos,L., Fernández-Sanz,L., Pagés,C., & Martínez-Herráiz, J.-J. (2013). Gamifying learning experiences: practical implications and outcomes. Computers & Education,63, 380–392. http://dx.doi.org/10.1016/j. compedu.2012.12.020.
20. Duval, S., & Tweedie, R. (2000). Trim and fill: a simple funnel‐plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463.
21. *Fitz-Walter, Z. J. (2015). Achievement unlocked: Investigating the design of effective gamification experiences for mobile applications and devices. Doctoral dissertation, Queensland University of Technology.
22. *Fitz-Walter, Z., Johnson, D., Wyeth, P., Tjondronegoro, D., & Scott-Parker, B. (2016). Driven to drive? Investigating the effect of gamification on learner driver behavior, perceived motivation and user experience. Computers in Human Behavior, 71, 586-595.
23. Foster,J.A., Sheridan,P.K., Irish,R., & Frost,G.S. (2012). Gamification as a strategy for promoting deeper investigation in a reverse engineering activity. In:Proceedings of the 2012 American Society for Engineering Education Conference, pp. AC2012–AC 5456.
24. Fragkos, K. C., Tsagris, M., & Frangos, C. C. (2014). Publication bias in meta-analysis: confidence intervals for Rosenthal’s fail-safe number. International scholarly research notices, 2014.
25. Garland, C. M. (2015). Gamification and implications for second language education: A Meta Analysis. Unpublished Master Thesis. Graduate Faculty of St. Cloud State University.
26. Hamari, J., Koivisto, J., & Sarsa, H. (2014). Does gamification work?– A literature review of empirical studies on gamification. Proceedings of the 47th Hawaii International Conference on System Sciences, Hawaii, USA, January 6–9.
27. *Hanus, M. D., & Fox, J. (2015). Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Computers & Education, 80, 152-161.
28. Higgins, J., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta‐analysis. Statistics in medicine, 21(11), 1539-1558.
29. *Hong, G. Y., & Masood, M. (2014). Effects of gamification on lower secondary school students’ motivation and engagement. International Journal of Social, Education, Economics and Management Engineering, 8(12), 3483-3490.
30. *Hsu, K. C. (2016). Social Gamification in Multimedia Instruction: Assessing the Effects of Animation, Reward Strategies, and Social Interactions on Learners Motivation and Academic Performance in Online Settings. Doctoral dissertation, University of Kansas, USA.
31. *Hudiburg, M.L. (2016). Motivation and Learning in an Online Collaborative Project Using Gamification. STEMPS Theses & Dissertations. Paper 5.
32. *Johnson, D. L. (2017). Motivational Effects of Badging from Gamification Theory in Asynchronous Discussion Forums Within Graduate Blended Education. Doctoral dissertation, Lancaster Bible College, USA.
33. Kelly, D.L., Centurino, V.A.S., Martin, M.O. ve Mullis, I.V.S. (2020). TIMSS 2019 encyclopedia: Education policy and curriculum in mathematics and science. Retrieved from https://timssandpirls.bc.edu/timss2019/encyclopedia/
34. *Ketyi, A. (2016). From mobile language learning to gamification: an overlook of research results with business management students over a five-year period. Porta Linguarum. Revista Internacional de Didáctica de las Lenguas Extranjeras, I, 45-60.
35. *Kim, K., & Ahn, S. J. G. (2017). The role of gamification in enhancing intrinsic motivation to use a loyalty program. Journal of Interactive Marketing, 40(C), 41-51.
36. Lee, J. J. ve Hammer, J. (2011). Gamification in Education: What, How, Why Bother? Academic Exchange. Quarterly, 15(2). ss.1-5.
37. *Lieberoth, A. (2015). Shallow gamification: Testing psychological effects of framing an activity as a game. Games and Culture, 10(3), 229-248.
38. Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.
39. Measles, S. ve Abu-Dawood, S. (2015). Gamification: Game–Based Methods and Strategies to Increase Engagement and Motivation within an eLearning Environment. In Society for Information Technology & Teacher Education International Conference (Vol. 2015, No. 1, pp. 8319-8324).
40. Mekler, E. D., Brühlmann, F., Opwis, K., & Tuch, A. N. (2013). Disassembling gamification: the effects of points and meaning on user motivation and performance. Paper presented at the CHI'13 extended abstracts on human factors in computing systems.
41. Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Prisma Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS med, 6(7).
42. Mora, A., Riera, D., González, & Arnedo-Moreno, J. (2015). A literature review of gamification design frameworks. 2015 7th International Conference on Games and Virtual Worlds for Serious Applications (VS-Games) (pp. 1-8). IEEE.
43. Nicholson, S., (2012). A user-centered theoretical framework for meaningful gamification. In:Proceedings of Games Learning Society 8.0.Madison,WI.
44. Robertson, M. (2010). Can't Play, Won't Play. Retrieved March 20, 2017 from http://kotaku.com/5686393/cant-play-wont-play
45. Rosenthal, R. (1979). The “file drawer problem” and tolerance for null results. Psychological Bulletin, 86, 638–641.
46. Rothstein, H. R., Sutton, A. J., & Borenstein, M. (Eds.). (2005). Publication bias in meta-analysis: Prevention, assessment and adjustments. West Sussex: John Wiley ve Sons.
47. *Rouse, K. E. (2013). Gamification in science education: The relationship of educational games to motivation and achievement (Order No. 3569748). Available from ProQuest Dissertations & Theses Global. (1370800410). Retrieved from http://search.proquest.com/docview/1370800410?accountid=15958
48. Ryan, R.M., & Deci, E.L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American psychologist, 55(1), 68-78.
49. *Serpe, R. L. (2017). Gamification Through Algebraic Coding. Master Thesis, College at Brockport, State University of New York, USA.
50. *Stansbury, J. A., & Earnest, D. R. (2017). Meaningful gamification in an industrial/organizational psychology course. Teaching of Psychology, 44(1), 38-45.
51. TEDMEM. (2020). Covid-19 Sürecinde Eğitim: Uzaktan Öğrenme, Sorunlar ve Çözüm Önerileri 2020. Retrieved from https://tedmem.org/
52. Toda, A. M., Do Carmo, R. S., Silva, A. L. ve Brancher, J. D. (2014). Project SIGMA-An Online tool to aid students in Math lessons with gamification concepts. Retrieved December 12, 2015 from http://www.jcc2014.ucm.cl/jornadas/EVENTOS/SCCC%202014/SCCC-9.pdf
53. Todd, A. (2017). Why Gamification is Malarkey. The Morning Watch: Educational and Social Analysis, 44(1-2).
54. Werbach, K. & Hunter, D. (2012). For the Win. How game thinking can revolutionize your business. Philadelphia, PA: Wharton Digital Press.
55. Werbach, K. (2014). Gamification course. Retrieved February 25, 2015 from https://www.coursera.org/course/gamification
56. Wongso, O., Rosmansyah, Y. ve Bandung, Y. (2014). Gamification Framework Model, Based on Social Engagement in E-Learning 2.0. 2nd International Conference on Technology, Informatics, Management, Engineering & Environment. Bandung, Indonesia, August 19-21, 2014.
57. Wood, L. C. ve Reiners T. (2012) Gamification in logistics and supply chain education: Extending active learning. IADIS Internet Technologies and Society, Perth (Australia), November 28-30, 2012, pp. 101-108.
58. *Yapıcı, İ. Ü., & Karakoyun, F. (2017). Gamification in biology teaching: A sample of Kahoot application. Turkish Online Journal of Qualitative Inquiry, 8(4).
59. Yildirim, I. (2016). Development, implementation and evaluation of a gamification based curriculum for the lesson of “Teaching Principles and Methods”. Unpublished Doctoral Dissertation. Department of Educational Sciences, Gaziantep University, Turkey.