Developing a Mobile Augmented Reality Application for Destroyed Historical Sites

Abstract

Many cultural heritage sites have been destroyed due to earthquakes and other reasons over the years. Tourists want to visit these areas but find nothing but ruins and small information signs. Some works were taken to museums for preservation. New technologies offer a solution to tourists who want to see these ruins in a real environment. Tourists have the opportunity to see ruins and non-existent historical places using their mobile devices through mobile applications. This research aims to develop a model of the Mausoleum of Halicarnassus, one of the seven wonders of the world, with Augmented Reality (AR) technology. The growing popularity of AR, which encompasses applications in education, entertainment and tourism, is a catalyst for the development of a mobile application designed to provide an immersive and educational experience for visitors to the historic site. Using computer vision and machine learning algorithms, the application detects and tracks the user's location and movements, placing virtual 3D models of the works in the real-world environment captured by the device's camera. This study creates Augmented Reality models for other destroyed historical sites or artifacts and presents the 3D models and the model developed with Vuforia API via Unity. AR offers a new approach to presenting historical information and recreating missing artifacts by seamlessly blending digital content with the physical world. The Halicarnassus mausoleum is one of the most difficult digital objects to develop. The shape of the mausoleum was created according to the descriptions of the person who saw it last. As a result of the research, a completely destroyed historical artifact can be revived and viewed at any point using AR. It is recommended to work on much more advanced versions of the model developed in this first stage in the future.

Keywords

• Augmented Reality
• Heritage
• Artefacts
• Mobile Application

Kayıp Tarihi Eserler için Mobil Arttırılmış Gerçeklik Uygulaması Geliştirilmesi

Öz

Birçok kültürel miras alanı yıllar içinde gerçekleşen depremler ve diğer sebeplerden dolayı tahrip olmuştur. Turistler bu alanları ziyaret etmek istemekte fakat yıkıntılardan ve küçük bilgi levhalarından başka bir şey bulamamaktadır. Bazı eserler ise korunma amacıyla müzelere götürülmüştür. Yeni teknolojiler, bu kalıntıları gerçek ortamda görmek isteyen turistlere bir çözüm sunmaktadır. Turistler, mobil uygulamalar aracılığıyla mobil cihazlarını kullanarak kalıntıların ve var olmayan tarihi yerleri görebilme imkanına sahip olmaktadır. Bu araştırma, Dünya’nın yedi harikasından biri olan Halikarnas Mozolesinin Artırılmış Gerçeklik (AG) teknolojisi ile model geliştirmeyi amaçlamaktadır. Eğitim, eğlence ve turizm alanlarındaki uygulamaları kapsayan AG'in artan popülaritesi, tarihi bölgeyi ziyaret edenlere sürükleyici ve eğitici bir deneyim sunmak üzere tasarlanmış bir mobil uygulamanın geliştirilmesinde katalizör görevi görmektedir. Bilgisayar görüşü ve makine öğrenimi algoritmalarından yararlanan uygulama, kullanıcının konumunu ve hareketlerini algılayıp takip ederek, eserlerin sanal 3 boyutlu modellerini cihazın kamerası tarafından yakalanan gerçek dünya ortamına yerleştirilmesi ile gerçekleşmektedir. Bu çalışma, tahrip edilen diğer tarihi alanlar veya eserler için Artırılmış Gerçeklik modelleri oluşturmakta 3D modeller ile Unity üzerinden Vuforia API ile geliştirilen modeli sunmaktadır. AR, dijital içeriği fiziksel dünyayla kusursuz bir şekilde harmanlayarak tarihsel bilgilerin sunulmasına ve eksik eserlerin yeniden yaratılmasına yeni bir yaklaşım sunmaktadır. Halikarnas mozolesi geliştirilmesi en zor dijital objelerden biri olmaktadır. Mozolonin şekli en son gören kişinin tasvirleri doğrultusunda oluşturulmuştur. Araştırma sonucunda tamamen yok olmuş bir tarihi eserin yeniden canlandırılması ve AR kullanarak istenilen noktada görülebilmesi sağlanmaktadır. Bu ilk aşamada geliştirilen modelin ilerleyen zamanda çok daha gelişmiş versiyonlarının üzerinde çalışılması önerilmektedir.

Anahtar Kelimeler

• Artırılmış Gerçeklik
• Miras
• Eserler
• Mobil Uygulama

References

1. Andrade, J. G., and Dias, P. (2020). A phygital approach to cultural heritage: augmented reality at Regaleira. Virtual Archaeology Review, 11(22): 15-25.
2. Amygdalau, K. (2017). Anatolian Travels, 19th Century: A Review. Meltem İzmir Akdeniz Akademisi Dergisi, (1): 79-83.
3. Barrado-Timón, D. A., and Hidalgo-Giralt, C. (2019). The historic city, its transmission and perception via augmented reality and virtual reality and the use of the past as a resource for the present: a new era for urban cultural heritage and tourism?. Sustainability, 11(10): 2835.
4. Beneyto, J., and Alcaniz, M. (2022). The impact of AR technology on museums' visitor experience: An empirical study. Journal of Business Research, 144: 31-42.
5. Berger, M., Tagliasacchi, A., Seversky, L., Alliez, P., Levine, J., Sharf, A., and Silva, C. (2014). State of the art in surface reconstruction from point clouds. In Eurographics 2014-State of the Art Reports,1 (1): 161-185.
6. Billinghurst, M., and Duenser, A. (2012). AR in the Museum. In Advances in Computer Entertainment Technology (pp. 1-14). Springer Berlin Heidelberg.
7. Boboc, R. G., Duguleană, M., Voinea, G. D., Postelnicu, C. C., Popovici, D. M., and Carrozzino, M. (2019). Mobile augmented reality for cultural heritage: Following the footsteps of Ovid among different locations in Europe. Sustainability, 11(4): 1167.
8. Bottani, E., and Vignali, G. (2019). AR technology in the manufacturing industry: A review of the last decade. Lise Transactions, 51(3): 284-310.

9. Brittanica 2020: Mausoleum-of-Halicarnassus at Brittanica, viewed 12 August 2020, https://www.britannica.com/topic/Mausoleum-of-Halicarnassus Buhalis, D., and Amaranggana, A. (2015). Smart tourism destinations enhancing tourism experience through personalisation of services. In Information and communication technologies in tourism (pp. 377-389). Springer, Cham.
10. Čejka, J., Zsíros, A., and Liarokapis, F. (2020). A hybrid augmented reality guide for underwater cultural heritage sites. Personal and Ubiquitous Computing, 24: 815-828.
11. Chen, H., Huang, H., Lin, C., and Chen, C. (2020). The application of AR in tourism. Journal of Hospitality, Leisure, Sport and Tourism Education, 26: 100246.
12. Corso, A. (2019). The Masters of the Mausoleum of Halicarnassus. Journal of Intercultural and Interdisciplinary Archaeology, 03: 109-121.
13. Dudgeon, J.E. and Gopalakrishnan, R. (1996). Fractal-based modeling of 3D terrain surfaces. Proceedings of SOUTHEASTCON '96, Tampa, FL, USA, 246-252.
14. Favro, D. (2006). In the Eyes of the Beholder: Virtual Reality Re-creations and Academia. In: L. Haselberger and J. Humphrey (eds.) Imaging Ancient Rome: Documentation, Visualization, Imagination. Journal of Roman Archaeology Suppl. Series, 61. Portsmouth: Journal of Roman Archaeology, 321–334.
15. Fiala, M. (2005). ARTag, a fiducial marker system using digital techniques. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) IEEE, 1: 590-596).
16. Florescu, S., and Nistor, D. E. (2020, May). Assessing the emotional impact of cognitive affordance in the built environment through AR. In Proceedings of the 11th Annual Symposium on Simulation for Architecture and Urban Design: 1-8.
17. Gherghina, A., Olteanu, A. C., and Tapus, N. (2013, January). A marker-based AR system for mobile devices. In 2013 11th RoEduNet International Conference (pp. 1-6). IEEE.
18. Grisolia, J. M., Diego-Mas, J. A., Marcos-Jorquera, D., and Sánchez-Franco, M. J. (2020). The Effect of AR on Visitors' Engagement, Emotions, Learning Outcomes, and Satisfaction in Cultural Heritage Sites. Journal of Travel Research, 59(2): 279-293.
19. Grzegorczyk, T., Sliwinski, R., and Kaczmarek, J. (2019). Attractiveness of AR to consumers. Technology Analysis and Strategic Management, 31(11): 1257-1269.
20. Graziano, T., and Privitera, D. (2020). Cultural heritage, tourist attractiveness and augmented reality: insights from Italy. Journal of Heritage Tourism, 15(6): 666-679.
21. Gonzalez Vargas, J. C., Fabregat, R., Carrillo-Ramos, A., and Jové, T. (2020). Survey: Using augmented reality to improve learning motivation in cultural heritage studies. Applied Sciences, 10(3): 897.
22. Gong, Z., Wang, R., and Xia, G. (2022a). AR as a tool for engaging museum experience: a case study on Chinese art pieces. Digital, 2(1): 33-45.
23. Gong, X., Shi, Y., Liu, X., Liu, Z., and Lu, C. (2022b). Effects of immersive AR on tourists’ flow experiences and behavioral intentions: A cognitive appraisal perspective. Journal of Travel Research, 61(1): 3-18
24. Göğebakan, Y. (2017). Relating “Quadriga” As A Group of Four Horses To “Four Horsemen of The Apocalypse” And Artistic Reflections. Anadolu Üniversitesi Sanat and Tasarim Dergisi, 105.
25. Günay, V. (2017). Osmanlıların İzmir Diyeti: Bodrum. İzmir Araştırmaları Dergisi, 5: 149-174.
26. He, Z., Wu, L., and Li, X. R. (2018). When art meets tech: The role of AR in enhancing museum experiences and purchase intentions. Tourism Management, 68: 127-139.
27. Hidayat, I. K., Rini, D. R., Novica, D. R., and Daud, K. A. M. (2019). Implementing AR and gamification in tourism. International Journal of Technology Management and Information System, 1(1): 39-45.
28. Hincapié, M., Díaz, C., Zapata-Cárdenas, M. I., Rios, H. D. J. T., Valencia, D., and Güemes-Castorena, D. (2021). Augmented reality mobile apps for cultural heritage reactivation. Computers and Electrical Engineering, 93: 107281.
29. Hunsucker, A. J., Baumgartner, E., and McClinton, K. (2018). Evaluating an AR-based museum experience. Interactions, 25(4): 66-68.
30. Hsieh, Y.-C., Chen, Y.-C., and Yen, D. C. (2020). The impact of AR on theme park experiences: A case study of Disneyland. Journal of Hospitality and Tourism Technology, 11(2): 256-276.
31. Hsu, Y. C., and Huang, Y. M. (2013). Factors affecting museum visitors’ behavioral intentions to use AR technologies. Educational Technology and Society, 16(2): 184-197.
32. Jenkins, I. (2010). The Mausolea of Halicarnassus. Bulletin of the Institute of Classical Studies. Supplement, 121-135.
33. Jung, H., Lee, J. H., and Chung, K. (2019). Factors influencing consumers' adoption of mobile AR games: An empirical study. Telematics and Informatics, 37: 63-74.
34. Kaghat, F. Z., Azough, A., Fakhour, M., and Meknassi, M. (2020). A new audio AR interaction and adaptation model for museum visits. Computers and Electrical Engineering, 84: 106606.
35. Khan, M. A., Israr, S., S Almogren, A., Din, I. U., Almogren, A., and Rodrigues, J. J. (2021). Using AR and deep learning to enhance Taxila Museum experience. Journal of Real-Time Image Processing, 18: 321-332.
36. Kim, J., and Lee, J. (2018). The impact of AR menu on consumers’ behavioral intentions in a restaurant. Journal of Travel and Tourism Marketing, 35(4): 443-454.
37. Knutov, E., Baalsrud Hauge, J., and Ljungstrand, P. (2019). AR in museums: A review of recent advancements and future directions. Journal of Cultural Heritage, 39: 172-181.
38. Koca, F., and Akça, B. (2017). Antik Karya’nın Başkenti Halikarnassos’ Da Kent Planlama Anlayışı. Sosyal ve Beşeri Bilimler Araştirmalari Dergisi, 19(40): 42-54.
39. Kounavis, C. D., Michalopoulou, E., and Tsiamitros, D. E. (2019). AR in cultural heritage tourism: A field trial at Knossos. Journal of Destination Marketing and Management, 12: 67-75.
40. Kyriakou, P., and Hermon, S. (2019a). Can I touch this? Using natural interaction in a museum AR system. Digital Applications in Archaeology and Cultural Heritage, 12: e00088.
41. Kyriakou, K., and Hermon, S. (2019b). AR in heritage tourism: A review. Journal of Heritage Tourism, 14(5): 478-493.
42. Lee, J. Y., Kim, J., and Lee, C.-K. (2018). The effects of AR on hotel booking. Journal of Hospitality and Tourism Technology, 9(2): 173-183.
43. Lee, K. J., and Kim, H. W. (2018). The impact of AR on consumers' experiences: The case of art museum. Journal of Business Research, 86: 104-112.
44. Li, S., Li, Y., and Ma, Y. (2021). Exploring the influence of AR on museum visitors' perceived value, satisfaction, and loyalty: A case of the Palace Museum in Beijing. Journal of Travel Research, 60(8), 1769-1784.
45. Liarokapis, F., Mourkoussis, N., and White, M. (2016). Evaluating the effectiveness of mobile AR applications in museums. Journal of Cultural Heritage, 17: 423-431.
46. Litvak, E., and Kuflik, T. (2020). Enhancing cultural heritage outdoor experience with augmented-reality smart glasses. Personal and ubiquitous computing, 24: 873-886.
47. Marchand, E., Uchiyama, H. and Spindler, F. (2016). Pose Estimation for AR: A Hands-On Survey. in IEEE Transactions on Visualization and Computer Graphics, 22 (12): 2633-2651.
48. Marino, E., Barbieri, L., Colacino, B., Fleri, A. K., and Bruno, F. (2021). An AR inspection tool to support workers in Industry 4.0 environments. Computers in Industry, 127: 103412.
49. Marzouk, A., Maher, A., and Mahrous, T. (2019). The influence of AR and virtual reality combinations on tourist experience. Journal of the Faculty of Tourism and Hotels-University of Sadat City, 3(2): 1-19.
50. Montero, A., Zarraonandia, T., Diaz, P., and Aedo, I. (2019). Designing and implementing interactive and realistic AR experiences. Universal Access in the Information Society, 18: 49-61.
51. Ozkaramanli, D., and Karakus, T. (2021). The Impact of AR Applications on Tourists’ Satisfaction and Behavioral Intentions: An Empirical Study. Journal of Travel Research, 60(7): 1508-1521.
52. Öncü, Ö. E. (2018). MÖ 4. Yüzyıl Karia İmar Programında Mimarlar ve Mimari Atölyeler. Sosyal Bilimler Arastırmaları Dergisi, 8(3): 663-676.
53. Parekh, P., Patel, S., Patel, N., and Shah, M. (2020). Systematic review and meta-analysis of AR in medicine, retail, and games. Visual Computing for Industry, Biomedicine, and Art, 3: 1-20.
54. Papaioannou, G. (2019). AR in Museums and Cultural Heritage Settings. AR in Educational Settings, 369-382.
55. Payntar, N. D., Hsiao, W. L., Covey, R. A., and Grauman, K. (2021). Learning patterns of tourist movement and photography from geotagged photos at archaeological heritage sites in Cuzco, Peru. Tourism management, 82 : 104165.
56. Petrelli, D., Dulake, N., and Giannachi, G. (2021). Visitors' Engagement with Mixed Reality Exhibitions in Heritage Sites: A Case Study in Bolsover Castle. Journal of Computer-Mediated Communication, 26(2): 176-191.
57. Petrovych, O., Zavalniuk, I., Bohatko, V., Poliarush, N., and Petrovych, S. (2023). Motivational Readiness of Future Teachers-Philologists to Use the Gamification with Elements of AR in Education. International Journal of Emerging Technologies in Learning (Online), 18(3): 4.
58. Reddy, C. O. (2022). Data visualization using marker-based AR techniques. Journal of Positive School Psychology, 3850-3856.
59. Remondino, Fabio and El-Hakim, Sabry. (2006). Image‐based 3D Modelling: A Review. The Photogrammetric Record, 21: 269 - 291.
60. Romano, B., Sands, S., and Pallant, J. I. (2021). AR and the customer journey: an exploratory study. Australasian Marketing Journal, 29(4): 354-363.
61. Rowland, I. D., and Howe, T. N. (Eds.). (2001). Vitruvius:'Ten books on architecture. Cambridge University Press.
62. Rovithis, E., Floros, A., Moustakas, N., Vogklis, K., and Kotsira, L. (2019). Bridging Audio and AR towards a New Generation of Serious Audio-Only Games. Electronic Journal of e-Learning, 17(2): 144-156.
63. Sadek, S. S. (2020). Novel features of interactive AR advertisements and its effect on stimulating user engagement: study of Egyptian tourism advertising. Journal of Arqhitecture, Arts, and Humanistic Sciences, 5(1): 1049-1073.
64. Schall, G., Hajek, N., Hlavacs, H., and Stitz, H. (2018). A Survey of Marker-Based AR for Outdoor Applications. In Proceedings of the 16th ACM International Conference on Advances in Mobile Computing and Multimedia (pp. 37-48). ACM.
65. Serravalle, F., Ferraris, A., Vrontis, D., Thrassou, A., and Christofi, M. (2019a). AR in the tourism industry: A multi-stakeholder analysis of museums. Tourism Management Perspectives, 32: 100549.
66. Serravalle, L., Camba, J. D., and Marques, L. (2019b). The impact of AR on museum visitor experience: A review of the literature. Journal of Hospitality, Leisure, Sport and Tourism Education, 25: 100212.
67. Tay, C. A., Dominic, B. B., Ho, H. I., Annuar, N., and Saferinor, N. E. M. (2023). Gamified AR Mobile Application for Tourism in Kuching. In Proceedings of the 9th International Conference on Computational Science and Technology: ICCST 2022, 27–28 August, Johor Bahru, Malaysia (pp. 355-366). Singapore: Springer Nature Singapore.
68. The British Museum (2021). Mausoleum At Halicarnassus. Erişim tarihi: 21 Ocak 2021. https://www.bmimages.com/results.asp?txtkeys1=mausoleum%20at%20halicarnassus
69. Truong, Q. L., Thai, V. Q., Nguyen, D. T., Nguyen, D. D., and Le, T. L. (2020). Enhancing visitors' experience and understanding of cultural heritage sites using AR: The case of Huế imperial city, Vietnam. Sustainability, 12(14): 5791.
70. Tosun, S., and Ustundag, A. (2021). The role of AR in enhancing museum visitors' emotional and cognitive experiences. Journal of Destination Marketing and Management, 20: 100555.
71. Trunfio, M., Lucia, M. D., Campana, S., and Magnelli, A. (2022). Innovating the cultural heritage museum service model through virtual reality and AR: The effects on the overall visitor experience and satisfaction. Journal of Heritage Tourism, 17(1): 1-19.
72. Voitik, N. V., and Maslov, M. D. (2019). AR technologies in tourism. Научный результат. Технологии бизнеса и сервиса, 5(3): 3-11.
73. Vuforia 2020: Vuforia Developer Library, viewed 12 August 2020, https://library.vuforia.com/getting-started/overview.html
74. Wang, C., and Zhu, Y. (2022). A Survey of Museum Applied Research Based on Mobile AR. Computational Intelligence and Neuroscience, 2022: 1-22.
75. Wang, D., and Dunston, P. S. (2017). AR: A review of its application in education. In Proceedings of the 2017 IEEE Global Engineering Education Conference (EDUCON) (pp. 1330-1337). IEEE.
76. Wypustek, A. (2013). The Deceased as the Chosen Ones and the Lovers of Deities. In Images of Eternal Beauty in Funerary Verse Inscriptions of the Hellenistic and Greco-Roman Periods (pp. 125-155). Brill. Unity 2020: Unity Technologies, viewed 12 August 2020, https://unity.com/
77. Uygur, A., Göral, R., and Bozkurt, İ. (2015). Muğla İli Milas İlçesinin Kültür Turizmi Açısından İncelenmesi. Selçuk Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, (34): 95-105.