Augmented reality applications: A Meta-analysis and thematic analysis

Abstract

The purpose of this study is to determine the effect of augmented reality (AR) applications on academic success through meta-analytic and thematic analysis. 51 items from 45 studies conducted between 2013 and 2019 were examined in different databases on the effect size of AR in relation on academic success, education level, and duration of the application. The effect size was found to exist at a medium level, considered to be significant which means that AR has positive effect on academic success. In terms of thematic analysis, with reference to common codes and themes found in qualitative studies conducted both nationally and internationally, qualitative findings were obtained. An emphasis on the effect of AR on learning, class environment and affective dimensions was made, together with a consideration of some limitations. It was stated that learning environments could be arranged in such a way that students could be motivated through the use of AR, leading to permanent and solid learning. Therefore, careful and planned use of AR can be said to encourage success.

Keywords

• Augmented reality,Academic success,Meta-analysis,Thematic analysis

Artırılmış gerçeklik uygulamaları: Bir meta-analiz ve tematik analiz

Abstract

Bu araştırmanın amacı artırılmış gerçeklik uygulamalarının akademik başarıya etkisini meta-analitik ve meta-tematik yöntemlerle belirlemektir. Bu bağlamda, artırılmış gerçeklik ile ilgili 2013-2019 yılları arasında yapılmış ve çeşitli veri tabanlarından tarama sonucunda ulaşılan 45 çalışmadaki 51 verinin başarı üzerindeki etki büyüklüğü ile öğretim kademesi ve uygulama süresi bağlamında da incelenmiştir. Etki büyüklüğü orta düzeyde bulunarak anlamlı ve etkili bulunmuştur. Bu bulgu artırılmış gerçekliğin akademik başarı üzerindeki etkisinin olumlu yönde etkili olduğunu göstermektedir. Diğer yandan araştırmanın meta-tematik analizi için yine ulusal ve uluslararası alanda, nitel yönlü çalışmaların ortak kod ve temalarından alıntılar yapılarak nitel bulgulara ulaşılmıştır. Artırılmış gerçekliğin öğrenme/sınıf ortamına, duyuşsal boyuta etkisi ile birtakım sınırlılıkların olduğu vurgulanmıştır. Öğrenme ortamlarının artırılmış gerçeklik ile motive eden, kalıcılık sağlayan şekilde düzenlenebileceği, somut öğrenmelerin gerçekleşebileceği ifade edilmiştir. Bu nedenle dikkatli ve planlı artırılmış gerçeklik kullanımının, beraberinde başarıyı getirebileceği belirtilebilir.

Keywords

• Artırılmış gerçeklik,Akademik başarı,Meta-analitik analiz,Tematik analiz

References

1. *(References with an asterisk (*) indicate studies included in the meta‐analysis.)
2. *Abdüsselam, M. S., & Karal, H. (2012). The effect of mixed reality environments on the students’ academic achievement in physics education: 11th grade magnetism topic example. Journal of Research in Education and Teaching, 1(4), 170-181.
3. Akbaş, M. F., & Güngör, C. (2017). A survey on marker-based tracking systems in augmented reality and design of multi-layer marker model. Dokuz Eylul University-Faculty of Engineering Journal of Science and Engineering, 19(56), 599-619.
4. *Akçayır, M. (2016). The effect of augmented reality applications on university students’ laboratory skills, attitudes and task loads. (Unpublished doctoral dissertation), Gazi University, Ankara.
5. Akçayır, G. (2018). The effect of augmented reality in education: A meta-analysis and systematic literature review. (Unpublished doctoral dissertation), Gazi University, Ankara.
6. *Akkuş, İ. (2016). Effects of augmented reality applications on mechanical engineering freshmen’s level of academic achievement and spatial ability in computer aided technical drawing. (Unpublished master's dissertation), Inonu University, Malatya.
7. Alcañiz, M., Contero, M., Perez-Lopez, D. C., & Ortega, M. (2010). Augmented reality technology for education. In New Achievements in Technology, Education and Development. Edited by S. Soomro. New York: IntechOpen, 247-256.
8. Amo, I. F., Erkoyuncu, J. A., Roy, R., Palmarini, R., & Onoufriou, D. (2018). A systematic review of augmented reality content-related techniques for knowledge transfer in maintenance applications. Computers in Industry, 103, 47-71. DOI: 10.1016/j.compind.2018.08.007.

9. Azuma, R. T. (1997). A survey of augmented reality. Presence: Teleoperators and Virtual Environments, 6(4), 355–385.
10. Azuma, R. T. (1999). The challenge of making augmented reality work outdoors. Mixed Reality: Merging Real and Virtual Worlds, Yuichi Ohta and Hideyuki Tamura (ed.) 379–390.
11. *Babur, A. (2016). The effects of using of augmented reality, simulation and real object on learning achievements, motivation and psychomotor performance. (Unpublished doctoral dissertation), Sakarya University, Sakarya.
12. *Bal, E., & Bicen, H. (2016). Computer hardware course application through augmented reality and QR code integration: Achievement levels and views of students. Procedia Computer Science, 102, 267-272.
13. Barsom, E. Z., Graafland, M., & Schijven, M. P. (2016). Systematic review on the effectiveness of augmented reality applications in medical training. Surgical Endoscopy, 30(10), 4174-4183. DOI: 10.1007/s00464-016-4800-6.
14. Batdı, V. (2017). The effect of multiple intelligences on academic achievement: A meta-analytic and thematic study. Educational Sciences: Theory & Practice, 17, 2057–2092. DOI:10.12738/estp.2017.6.0104
15. Batdı, V., & Batdı, H. (2015). Effect of creative drama on academic achievement: A meta-analytic and thematic analysis. Educational Sciences: Theory and Practice, 15(6), 1459-1470. DOI:10.12738/estp.2015.6.0156
16. *Baysan, E. (2015). The impact of augmented reality book (AR-Book) usage on the students’ academic achievements and students’ reviews on it. (Unpublished master’s dissertation), Gazi University, Ankara.
17. Bernard, R. M., Abrami, P. C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., ... & Huang, B. (2004). How does distance education compare with classroom instruction? A meta-analysis of the empirical literature. Review of Educational Research, 74(3), 379-439.
18. Billinghurst, M., & Dunser, A. (2012). Augmented reality in the classroom. Computer, 45(7), 56-63.
19. *Bursali, H., & Yilmaz, R. M. (2019). Effect of augmented reality applications on secondary school students’ reading comprehension and learning permanency. Computers in Human Behavior, 95, 126-135.
20. *Cai, S., Chiang, F. K., & Wang, X. (2013). Using the augmented reality 3D technique for a convex imaging experiment in a physics course. International Journal of Engineering Education, 29(4), 856-865.
21. Cardos¸ R. A. I, David, O. A., & David, D. O. (2017). Virtual reality exposure therapy in flight anxiety: A quantitative meta-analysis. Computers in Human Behavior, 72, 371-380.
22. *Chang, R. C., Chung, L. Y., & Huang, Y. M. (2016). Developing an interactive augmented reality system as a complement to plant education and comparing its effectiveness with video learning. Interactive Learning Environments, 24(6), 1245-1264.
23. *Chang, Y. H. (2013). Applying an AR technique to enhance situated heritage learning in a ubiquitous learning environment. Turkish Online Journal of Educational Technology-TOJET, 12(3), 21-32.
24. *Chen, C. H., Huang, C. Y., & Chou, Y. Y. (2019). Effects of augmented reality-based multidimensional concept maps on students’ learning achievement, motivation and acceptance. Universal Access in the Information Society, 18(2), 257-268.
25. *Chen, C. M., & Tsai, Y. N. (2012). Interactive augmented reality system for enhancing library instruction in elementary schools. Computers & Education, 59(2), 638-652.
26. Chen, C. P., & Wang, C. H. (2015). Employing augmented-reality-embedded instruction to disperse the imparities of individual differences in earth science learning. Journal of Science Education and Technology, 24(6), 835–847.
27. Cheng, K. H., & Tsai, C. C. (2013). Affordances of augmented reality in science learning: Suggestions for future research. Journal of Science Education and Technology, 22(4), 449-462.
28. Cheung, A. C. K., & Slavin, R. E. (2011). The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A metaanalysis. Educational Research Review, 9, 88-113. DOI: 10.1016/j.edurev.2013.01.001.
29. *Chiang, T. H. C., Yang, S. J., & Hwang, G. J. (2014). An augmented reality-based mobile learning system to improve students' learning achievements and motivations in natural science inquiry activities. Educational Technology & Society, 17(4), 352-365.
30. *Chien, Y. C., Su, Y. N., Wu, T. T., & Huang, Y. M. (2019). Enhancing students’ botanical learning by using augmented reality. Universal Access in the Information Society, 18(2), 231-241.
31. *Civelek, T., Ucar, E., Ustunel, H., & Aydın, M. K. (2014). Effects of a haptic augmented simulation on K-12 students' achievement and their attitudes towards Physics. Eurasia Journal of Mathematics, Science & Technology Education, 10(6), 565-574.
32. *Coşkun, M. (2018). The impact of the teaching supported with mobile application and augmented reality on motivation and anxiety about Science lesson, attitude towards Astronomy, and academic success of students. (Unpublished master’s dissertation), Mustafa Kemal University, Hatay.
33. *Cubillo, J., Martin, S., Castro, M., & Boticki, I. (2015). Preparing augmented reality learning content should be easy: UNED ARLE—an authoring tool for augmented reality learning environments. Computer Applications in Engineering Education, 23(5), 778-789.
34. *Çömen, H. (2018). Investigation of a hybrid design book based on the writing to learn activities supported learning environment in the seventh-grade electrical energy unit. (Unpublished master’s dissertation), Uşak University, Uşak.
35. Delello, J. A. (2014). Insights from pre-service teachers using science-based augmented reality. Journal of Computers in Education, 1(4), 295–311.
36. *Demirel, T. (2017). The effect of augmented reality activities supported by argumentation approach on academic achievement, critical thinking skills, motivation towards science and technology course and argumentation skills. (Unpublished doctoral dissertation), Çukurova University, Adana.
37. *Doğan, Ö. (2016). The effectiveness of augmented reality supported materials on vocabulary learning and retention. (Unpublished master’s dissertation), Abant Izzet Baysal University, Bolu.
38. Dunleavy, M., & Dede, C. (2014). Augmented reality teaching and learning. In J. M. Spector, M. D. Merrill, J. Elen and M. J. Bishop (Eds.), The handbook of research for educational communications and technology, (4th ed.). New York: Springer.
39. Dunleavy, M., Dede, C., & Mitchell, R. (2009). Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning. Journal of Science Education and Technology, 18(1), 7-22, DOI: 10.1007/s10956-008-9119-1.
40. *Erbas C, Demirer V. (2019). The effects of augmented reality on students' academic achievement and motivation in a Biology course. Journal of Computer Assisted Learning, 35, 450–458. DOI: 10.1111/jcal.12350.
41. *Ersoy H., Duman E., & Öncü S. (2016). Motivation and success with augmented reality: An experimental study. Journal of Instructional Technologies & Teacher Education, 5(1), 39-44.
42. Erwin, E.J., Brotherson, M.J., & Summers, J.A. (2011). Understanding qualitative meta synthesis: issues and opportunities in early childhood intervention research. Journal of Early Intervention, 33(3), 186-200.
43. *Fidan, M., (2018). The impact of problem-based science teaching assisted with augmented reality applications on academic achievement, retention, attitude and belief of self-efficacy. (Unpublished doctoral dissertation), Abant Izzet Baysal University, Bolu.
44. Fleck, S., Simon, G., & Christian Bastien, J. M. (September, 2014). AIBLE: An inquiry-based augmented reality environment for teaching astronomical phenomena. 2014 IEEE International Symposium on Mixed and Augmented Reality - Media, Art, Social Science, Humanities and Design, Münich, Germany.
45. *Gopalan, V. (2016). Conventional approach vs augmented reality textbook on learning performance: A study in science learning among secondary school students. Revista de la Facultad de Ingeniería, 31(5), 19-26.
46. Graham M., Zook M., & Boulton A. (2013). Augmented reality in urban places: Contested content and the duplicity of code. Transactions of the Institute of British Geographers. 38(3), 464–479. DOI: 10.1111/j.1475-5661.2012.00539.x.
47. Hedges, L. V. & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
48. *Ibáñez, M. B., Di-Serio, A., Villaran-Molina, D., & Delgado-Kloos, C. (2014). Experimenting with electromagnetism using augmented reality: Impact on flow student experience and educational effectiveness. Computers & Education, 71, 1-13.
49. Huang, T. C., Chen, C. C., & Chou, Y. W. (2016). Animating eco-education: To see, feel, and discover in an augmented reality-based experiential learning environment. Computers & Education, 96, 72-82.
50. *Ibáñez, M. B., Di-Serio, A., Villaran-Molina, D., & Delgado-Kloos, C. (2016). Support for augmented reality simulation systems: the effects of scaffolding on learning outcomes and behavior patterns. IEEE Transactions on Learning Technologies, 9(1), 46-56.
51. Ivanova, M., & Ivanov, G. (2011). Enhancement of learning and teaching in computer graphics through marker augmented reality technology. International Journal on New Computer Architectures and their Applications, 1(1), 176-184.
52. İbili, E., & Şahin, S. (2013). Software design and development of an interactive 3D Geometry book using augmented reality: ARGE3D. Afyon Kocatepe University Journal of Science and Engineering, 13(1), 1-8. DOI: 10.5578/fmbd.6213.
53. *Jee, H. K., Lim, S., Youn, J., & Lee, J. (2014). An augmented reality-based authoring tool for e-learning applications. Multimedia Tools and Applications, 68(2), 225-235.
54. Kamphuis, C., Barsom, E., Schijven, M., & Christoph, N. (2014). Augmented reality in medical education? Perspectives on Medical Education, 4(1), 300-311.
55. *Karagozlu, D. (2018). Determination of the impact of augmented reality application on the success and problem-solving skills of students. Quality & Quantity, 52(5), 2393-2402.
56. Kerawalla, L., Luckin, R., Seljeflot, S., & Woolard, A. (2006). Making it real: Exploring the potential of augmented reality for teaching primary school science. Virtual Reality, 10(3-4), 163-174.
57. *Kose, U., Koc, D., & Yucesoy, S. A. (2013). An augmented reality based mobile software to support learning experiences in computer science courses. Procedia Computer Science, 25, 370-374.
58. *Küçük, S. (2015). Effects of learning anatomy via mobile augmented reality on medical students’ academic achievement, cognitive load, and views toward implementation. (Unpublished doctoral dissertation), Atatürk University, Erzurum.
59. Lee, K. (2012). Augmented reality in education and training. TechTrends, 56(2), 13-21.
60. Lehner, O. M., & Kaniskas, J. (2012). Opportunity recognition in social entrepreneurship: A thematic meta-analysis. The Journal of Entrepreneurship, 21(1), 25–58.
61. *Lin, H. C. K., Chen, M. C., & Chang, C. K. (2015). Assessing the effectiveness of learning solid geometry by using an augmented reality-assisted learning system. Interactive Learning Environments, 23(6), 799-810.
62. Lin, H. C. K., Hsieh, M. C., Wang, C. H., Sie, Z. Y., & Chang, S. H. (2011). Establishment and usability evaluation of an interactive AR learning system on conservation of fish. Turkish Online Journal of Educational Technology-TOJET, 10(4), 181-187.
63. *Lin, T. J., Duh, H. B. L., Li, N., Wang, H. Y., & Tsai, C. C. (2013). An investigation of learners' collaborative knowledge construction performances and behavior patterns in an augmented reality simulation system. Computers & Education, 68, 314-321.
64. Liu, C., Cao, S., Tse, W., & Xu, X. (2017). Augmented reality–assisted intelligent window for cyber-physical machine tools. International Journal of Industrial and Manufacturing Systems Engineering, 44, 280-286. DOI: 10.1016/j.jmsy.2017.04.008.
65. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. Thousand Oaks, California: Sage.
66. Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Prisma Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Medicine, 6(7), e1000097.
67. Özarslan, Y. (2013). The effect of augmented reality enhanced learning materials on learners' achievement and learners' satisfaction. (Unpublished doctoral dissertation), Anadolu University, Eskişehir.
68. *Özbek, F. (2018). Effect of the augmented reality in the elementary school 4th grade Turkish course on the achievement and motivation of the students. (Unpublished master’s dissertation), Adnan Menderes University, Aydın.
69. Pérez-Sanagustín, M., Hernández-Leo, D., Santos, P., Kloos, C. D., & Blat, J. (2014). Augmenting reality and formality of informal and non-formal settings to enhance blended learning. IEEE Transactions on Learning Technologies, 7(2), 118-131.
70. Phillips, W., Lee, H., Ghobadian, A., O’Regan, N., & James, P. (2015). Social ınnovation and social entrepreneurship: a systematic review. Group & Organization Management, 40(3), 428–461 DOI: 10.1177/1059601114560063.
71. Rabbi, I., & Ullah, S. (2013). A survey on augmented reality challenges and tracking. Acta Graphica, 24(1-2), 29-46.
72. Radu, I. (2014). Augmented reality in education: A meta-review and cross-media analysis. Personal and Ubiquitous Computing, 18, 1533-1543.
73. Rauh, S., Zsebedits, D., Tamplon, E., Bolch, S., & Meixner, G. (2015, September). Using Google Glass for mobile maintenance and calibration tasks in the AUDI A8 production line. In 2015 IEEE 20th Conference on Emerging Technologies & Factory Automation (ETFA) (pp. 1-4). IEEE, Luxembourg.
74. *Renner, J. C. (2014). Does augmented reality affect high school students’ learning outcomes in Chemistry? (Unpublished doctoral dissertation), Grand Canyon University, Phoenix, Arizona.
75. Rosenberg, M., Adams, D., & Gurevitch, J. (2000). MetaWin statistical software for meta-analysis version 2.0, Massachusetts, MA: Sinauer Associates Inc.
76. Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638 641.
77. Serino, M., Cordrey, K., McLaughlin, L., & Milanaik, R. L. (2016). Pokémon Go and augmented virtual reality games: a cautionary commentary for parents and pediatricians. Current Opinion in Pediatrics, 28(5), 673-677.
78. Schrier, K. (2006, July). Using augmented reality games to teach 21st century skills. In ACM SIGGRAPH 2006 Educators program (pp. 15-23). Boston: ACM Press. DOI: 10.1145/1179295.1179311.
79. *Sırakaya, M., (2015). Effects of augmented reality applications on students' achievement, misconceptions and course engagement. (Unpublished doctoral dissertation), Gazi University, Ankara.
80. *Solak, E., & Cakir, R. (2016). Investigating the role of augmented reality technology in the language classroom. Online Submission, 18(4), 1067-1085.
81. *Sommerauer, P., & Müller, O. (2014). Augmented reality in informal learning environments: A field experiment in a mathematics exhibition. Computers & Education, 79, 59-68.
82. Somyürek, S. (2014). Gaining the attention of generation Z in learning process: Augmented reality. Educational Technology Theory and Practice, 4(1), 63-80.
83. *Şahin, D. (2017). Effect of science teaching with the augmented reality technology on secondary school students' achievement and their attitude towards the course. (Unpublished master’s dissertation), Atatürk University, Erzurum.
84. *Şentürk, M. (2018). The investigation with Solomon four-group design on the effect of using mobile augmented reality(AR) applications in the unit titled solar system and beyond in the seventh-grade on the students' academic success, motivation, science and technology attitude. (Unpublished master’s dissertation), Kocaeli University, Kocaeli.
85. *Tarng, W., Ou, K. L., Yu, C. S., Liou, F. L., & Liou, H. H. (2015). Development of a virtual butterfly ecological system based on augmented reality and mobile learning technologies. Virtual Reality, 19(3-4), 253-266.
86. Taşkıran, A., Koral, E., & Bozkurt, A. (2015). Artırılmış gerçeklik uygulamasının yabancı dil eğitiminde kullanılması. In Akademik Bilisim, Eskisehir, Turkey.
87. Tekerdere, H. & Göker, H. (2016). Examining the effectiveness of augmented reality applications in education: A meta-analysis. International Journal of Environmental & Science Education, 11(16), 9469-9481.
88. Thalheimer, W., & Cook, S. (2002). How to calculate effect sizes from published research: A simplified methodology. Work-Learning Research, 1. http://education.gsu.edu/coshima/EPRS8530/Effect_Sizes_pdf4.pdf.
89. Van Krevelen, D. W. F., & Poelman, R. (2010). Augmented reality: Technologies, applications and limitations. International Journal of Virtual Reality, 9(2), 1-20.
90. Weber, R. (1990). Basis content analysis (2nd ed.). Thousand Oaks, CA: Sage Publications.
91. Viera, A. J. & Garrett, J. M. (2005). Understanding interobserver agreement: The kappa statistic. Family Medicine, 37(5), 360-363.
92. Walczak, K., Wojciechowski, R., & Cellary, W. (2006, November). Dynamic interactive VR network services for education. In Proceedings of the ACM symposium on virtual reality software and technology (pp. 277-286). (VRST’06). New York: ACM. DOI: 10.1145/1180495.1180552.
93. Wojciechowski, R., & Cellary, W. (2013). Evaluation of learners’ attitude toward learning in ARIES augmented reality environments. Computers & Education, 68, 570–585.
94. Wu, H. K., Lee, S. W. Y., Chang, H. Y., & Liang, J. C. (2013). Current status, opportunities and challenges of augmented reality in education. Computers &Education, 62, 41-49.
95. *Wu, P. H., Hwang, G. J., Yang, M. L., & Chen, C. H. (2018). Impacts of integrating the repertory grid into an augmented reality-based learning design on students’ learning achievements, cognitive load and degree of satisfaction. Interactive Learning Environments, 26(2), 221-234.
96. *Yang, M. T., & Liao, W. C. (2014). Computer-assisted culture learning in an online augmented reality environment based on free-hand gesture interaction. IEEE Transactions on Learning Technologies, 7(2), 107-117.
97. *Yao, S. (2018). Teaching structural analysis with a mobile augmented reality application. (Unpublished master’s dissertation), Iowa State University, Ames, Iowa.
98. Yen J. C., Tsai C. H., & Wang J. Y. (2012). The effects of augmented reality on students’ moon phases concept learning and their conceptual changes of misconception. International Conference on Business and Information, Sapporo, Japan.
99. *Yıldırım, P. (2018). The impact of science teaching made with mobile augmented reality technology on science and technology attitudes and academic achievement of secondary school students. (Unpublished master’s dissertation), Fırat University, Elazığ.
100. Yıldırım, A., & Şimşek, H. (2013). Sosyal bilimlerde nitel araştırma yöntemleri, Seckin Publishing, Ankara, Turkey.
101. Yilmaz, Z. A., & Batdi, V. (2016). A meta-analytic and thematic comparative analysis of the integration of augmented reality applications into education. Education and Science, 41(188), 273-289.
102. Yuen, S., Yaoyuneyong, G., & Johnson, E. (2011). Augmented reality: An overview and five directions for AR in education. Journal of Educational Technology Development and Exchange, 4(1), 119–140