Creation of a Virtual Tour .Exe Utilizing Very High-Resolution RGB UAV Data

Abstract

In the last decades, developments in game engine technology led to a raised attraction to the virtual reality (VR) and augmented reality (AR) concepts which offer users an interactive synthetic environment. Also, with the travel limitations of the current COVID-19 pandemic, VR tour applications that visualize the geospatial data gained popularity more than ever. In this study, a three-dimensional (3D) VR tour application was created for Gebze Technical University (GTU) Campus by integrating unmanned aerial vehicle (UAV) data into an artificial environment by using cross-platform game development engine Unity. For creating high-quality 3D models of the Campus, different imaging geometries and flight altitudes were applied. The aerial photos were achieved with a ground sampling distance (GSD) of ≤2.2 cm with a 20 megapixel (MP) Sony Exmor RGB camera. Point cloud processing and the generation of high-quality 3D products were carried out by structure from motion (SfM) based photogrammetric software Agisoft Metashape. Using 86 well-distributed ground control points (GCPs), geometric correction accuracy of ±2 cm (~0.9 pixels) was reached as root mean square error (RMSE). Generated 3D models were imported into the Unity environment and the negative influence of high polygon data on the application performance was reduced by applying occlusion culling and space subdivision rendering optimization algorithms. The visual potential of the VR was improved by adding 3D individual object models such as trees, benches and arbors. For enhancing the information content of the VR tour, interactive information panels including the building metadata such as building name, block name and total floor area were placed. Finally, a first-person player was implemented for a realistic VR experience.

Keywords

• UAV
• SFM
• 3D Textured Mesh Model
• Virtual Reality
• Unity

References

1. Achille, C., Adami, A., Chiarini, S., Cremonesi, S., Fassi, F., Fregonese, L., Taffurelli, L. (2015). UAV-based photogrammetry and integrated technologies for architectural applications—methodological strategies for the after-quake survey of vertical structures in Mantua (Italy). Sensors, 15(7), 15520-15539.
2. Annis, A., Nardi, F., Petroselli, A., Apollonio, C., Arcangeletti, E., Tauro, F., ... Grimaldi, S. (2020). UAV-DEMs for small-scale flood hazard mapping. Water, 12(6), 1717.
3. Bekele, M. K., Pierdicca, R., Frontoni, E., Malinverni, E. S., Gain, J. (2018). A survey of augmented, virtual, and mixed reality for cultural heritage. Journal on Computing and Cultural Heritage, 11(2), 1-36.
4. Bendig, J., Willkomm, M., Tilly, N., Gnyp, M. L., Bennertz, S., Qiang, C., ... Bareth, G. (2013). Very high-resolution crop surface models (CSMs) from UAV-based stereo images for rice growth monitoring in Northeast China. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci., 40, 45-50.
5. Brooks, F. P. (1999). What's real about virtual reality?. IEEE Computer Graphics and Applications, 19(6), 16-27.
6. Bruno, F., Barbieri, L., Lagudi, A., Cozza, M., Cozza, A., Peluso, R., Muzzupappa, M. (2018). Virtual dives into the underwater archaeological treasures of South Italy. Virtual Reality, 22(2), 91-102.
7. Bruno, F., Bruno, S., De Sensi, G., Luchi, M. L., Mancuso, S., Muzzupappa, M. (2010). From 3D reconstruction to virtual reality: A complete methodology for digital archaeological exhibition. Journal of Cultural Heritage, 11(1), 42-49.
8. Campana, S. (2017). Drones in archaeology. State‐of‐the‐art and future perspectives. Archaeological Prospection, 24(4), 275-296.

9. Carl, E., Stein, A. T., Levihn-Coon, A., Pogue, J. R., Rothbaum, B., Emmelkamp, P., ... Powers, M. B. (2019). Virtual reality exposure therapy for anxiety and related disorders: A meta-analysis of randomized controlled trials. Journal of Anxiety Disorders, 61, 27-36.
10. Carrozzino, M., Bergamasco, M. (2010). Beyond virtual museums: Experiencing immersive virtual reality in real museums. Journal of Cultural Heritage, 11(4), 452-458.
11. Coorg, S., Teller, S. (1997). Real-time occlusion culling for models with large occluders. In Proceedings of the Symposium on Interactive 3D Graphics, Providence RI, USA, 83-90.
12. Duan, H., Li, J., Fan, S., Lin, Z., Wu, X., Cai, W. (2021). Metaverse for social good: A university campus prototype. In Proceedings of the 29th ACM International Conference on Multimedia, 153-161.
13. Eisenbeiss, H., Zhang, L. (2006). Comparison of DSMs generated from mini UAV imagery and terrestrial laser scanner in a cultural heritage application. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 36(5), 90-96.
14. Gaitatzes, A., Christopoulos, D., Roussou, M. (2001). Reviving the past: cultural heritage meets virtual reality. In Proceedings of the 2001 Conference on Virtual Reality, Archeology, and Cultural Heritage, Athens, Greece, 103-110.
15. Guttentag, D. A. (2010). Virtual reality: Applications and implications for tourism. Tourism Management, 31(5), 637-651.
16. Hirokawa, R., Kubo, D., Suzuki, S., Meguro, J. I., Suzuki, T. (2007). A small UAV for immediate hazard map generation. AIAA Infotech@ Aerospace 2007 Conference and Exhibit, California, USA, 2725.
17. Hoffman, H. G., Patterson, D. R., Carrougher, G. J. (2000). Use of virtual reality for adjunctive treatment of adult burn pain during physical therapy: a controlled study. The Clinical Journal of Pain, 16(3), 244-250.
18. Huang, Y. C., Backman, K. F., Backman, S. J., Chang, L. L. (2016). Exploring the implications of virtual reality technology in tourism marketing: An integrated research framework. International Journal of Tourism Research, 18(2), 116-128.
19. Jafri, R., Campos, R. L., Ali, S. A., Arabnia, H. R. (2017). Visual and infrared sensor data-based obstacle detection for the visually impaired using the Google project tango tablet development kit and the unity engine. IEEE Access, 6, 443-454.
20. Kato, A., Ohno, N. (2009). Construction of three-dimensional tooth model by micro-computed tomography and application for data sharing. Clinical Oral Investigations, 13(1), 43-46.
21. Lin, A. Y. M., Novo, A., Har-Noy, S., Ricklin, N. D., Stamatiou, K. (2011). Combining GeoEye-1 satellite remote sensing, UAV aerial imaging, and geophysical surveys in anomaly detection applied to archaeology. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 4(4), 870-876.
22. Lo Brutto, M., Garraffa, A., Meli, P. (2014). UAV platforms for cultural heritage survey: first results. ISPRS Annals of Photogrammetry, Remote Sensing & Spatial Information Sciences, 2(5), 227-234.
23. Ma, M., Zheng, H., Lallie, H. (2010). Virtual reality and 3D animation in forensic visualization. Journal of Forensic Sciences, 55(5), 1227-1231.
24. Martínez-Carricondo, P., Carvajal-Ramírez, F., Yero-Paneque, L., Agüera-Vega, F. (2020). Combination of nadiral and oblique UAV photogrammetry and HBIM for the virtual reconstruction of cultural heritage. Case study of Cortijo del Fraile in Níjar, Almería (Spain). Building Research & Information, 48(2), 140-159.
25. Ozdogan, N., Sefercik, U. G., Kilinc, Y., Caliskan, E., Atalay, C. (2021). Determination of water quality by analysing unmanned aerial vehicle data and physico-chemical parameters: The case of Aydınlar (Gülüç) Stream. Journal of European Science and Technology, 23, 572-582.
26. Pepe, M., Costantino, D. (2021). UAV photogrammetry and 3D modelling of complex architecture for maintenance purposes: The case study of the masonry bridge on the Sele River, Italy. Periodica Polytechnica Civil Engineering, 65(1), 191-203.
27. Powers, M. B., Emmelkamp, P. M. (2008). Virtual reality exposure therapy for anxiety disorders: A meta-analysis. Journal of Anxiety Disorders, 22(3), 561-569.
28. Rua, H., Alvito, P. (2011). Living the past: 3D models, virtual reality and game engines as tools for supporting archaeology and the reconstruction of cultural heritage–the case-study of the Roman villa of Casal de Freiria. Journal of Archaeological Science, 38(12), 3296-3308.
29. Schmitt, Y. S., Hoffman, H. G., Blough, D. K., Patterson, D. R., Jensen, M. P., Soltani, M., ... Sharar, S. R. (2011). A randomized, controlled trial of immersive virtual reality analgesia, during physical therapy for pediatric burns. Burns, 37(1), 61-68.
30. Sefercik, U. G., Kavzoglu, T., Nazar, M., Atalay, C., Madak, M. (2021). UAV-Based 3D virtual tour creation. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, XLVI-4/W5-2021, 493-499.
31. Sefercik, U. G., Tanrikulu, F., Atalay, C. (2019). Photogrammetric 3D modelling potential comparison of SFM-based new generation image matching software. In the 40th Asian Conference on Remote Sensing, 1-8.
32. Sieberth, T., Dobay, A., Affolter, R., Ebert, L. C. (2019). Applying virtual reality in forensics–a virtual scene walkthrough. Forensic Science, Medicine and Pathology, 15(1), 41-47.
33. Silva, L. A., Sanchez San Blas, H., Peral García, D., Sales Mendes, A., Villarubia González, G. (2020). An architectural multi-agent system for a pavement monitoring system with pothole recognition in UAV images. Sensors, 20(21), 6205.
34. Smith, N. G., Passone, L., Al-Said, S., Al-Farhan, M., Levy, T. E. (2014). Drones in archaeology: integrated data capture, processing, and dissemination in the al-Ula Valley, Saudi Arabia. Near Eastern Archaeology, 77(3), 176-181.
35. Sutherland, I. E. (1965). The ultimate display. Proceedings of the International Federation of Information Processing Congress, New York, USA, 2, 506-508.
36. Tan, Y., Li, Y. (2019). UAV photogrammetry-based 3D road distress detection. ISPRS International Journal of Geo-Information, 8(9), 409.
37. Tsouros, D. C., Bibi, S., Sarigiannidis, P. G. (2019). A review on UAV-based applications for precision agriculture. Information, 10(11), 349.
38. Westoby, M. J., Brasington, J., Glasser, N. F., Hambrey, M. J., Reynolds, J. M. (2012). ‘Structure-from-Motion’photogrammetry: A low-cost, effective tool for geoscience applications. Geomorphology, 179, 300-314.
39. Yung, R., Khoo-Lattimore, C. (2019). New realities: a systematic literature review on virtual reality and augmented reality in tourism research. Current Issues in Tourism, 22(17), 2056-2081.