Accuracy analysis of 3 dimensional point clouds obtained from camera and LiDAR sensors of smart phones and tablets

Öz

The importance of portable and wearable smart mobile devices (phone, tablet, watch, glasses, etc.) is increasing day by day in the digitalizing spatial information industry. Smartphones come to the fore in this industry with their usage rate and economic market share. Cameras with different resolutions are used in these devices, which are relatively low cost compared to professional equipment and have many sensor features. With the laser scanning (LiDAR) sensor feature added to some smartphones and tablet models that have been recently introduced to the market, this development was taken one step further and the infrastructure for the effective use of camera + LiDAR sensors in engineering measurement applications has been developed. These features offer cheaper alternatives for 3D modeling and augmented reality (Augmented Reality, AR). In this study, images and point clouds obtained with camera+LiDAR sensors of objects defined in different sizes and geometric shapes in indoor and outdoor spaces by using smart devices (phone + tablet) for high accuracy spatial information production with 3D measurement and modeling were analyzed and then they were compared using C2C and M3C2 deviation analysis methods. Considering the results, the accuracy of the 3D models obtained with smart mobile devices embedded with innovative technological sensors has revealed that these devices are a dominant alternative for the use of these devices in many different sectors in terms of the spatial information industry.

Anahtar Kelimeler

• Structure-from-motion
• Smart phones and tablets
• LiDAR
• 3D point cloud generation
• Accuracy assessment

Proje Numarası

FBA-2021-4295

Akıllı telefon ve tabletlerin kamera ve LiDAR sensörlerinden elde edilen 3 boyutlu nokta bulutlarının doğruluk analizi

Öz

Taşınabilir ve giyilebilir akıllı mobil cihazların (telefon, tablet, kol saati, gözlük vb.) önemi dijitalleşen mekânsal bilgi endüstrisinde her geçen gün artmaktadır. Akıllı telefonlar gerek kullanım oranı gerekse ekonomik pazar payıyla bu endüstride ön plana çıkmaktadır. Profesyonel donanımlara kıyasla görece düşük maliyetli olan ve birçok sensör özelliğine sahip bu cihazlarda, farklı çözünürlükte kameralar kullanılmaktadır. Son olarak piyasaya sunulan bazı akıllı telefon ve tablet modellerine eklenen lazer tarama (LiDAR) sensör özelliğiyle bu gelişim bir adım daha ileri taşınarak, kamera+LiDAR sensörlerinin mühendislik ölçme uygulamalarında efektif kullanımının altyapısı geliştirilmiştir. 3 boyutlu (3B) modelleme ve artırılmış gerçeklik (Augmented Reality, AR) için bu özellikler maliyet bakımından daha ucuz alternatifler sunmaktadır. Bu çalışmada 3B ölçme ve modelleme ile yüksek doğrulukta mekânsal bilgi üretimi için akıllı cihazlar (telefon+tablet) kullanılarak, iç ve dış mekânlarda farklı boyut ve geometrik şekillerde tanımlanan nesnelerin kamera+LiDAR sensörleriyle elde edilen görüntüleri ve nokta bulutları analiz edilmiş, C2C ve M3C2 sapma analizi yöntemleri kullanılarak karşılaştırılmıştır. Elde edilen bulgular dikkate alındığında, yenilikçi teknolojik sensörlere sahip akıllı mobil cihazlarla elde edilen 3B model uygulama sonuçlarının doğruluğu, bu cihazların mekânsal bilgi endüstrisi açısından birçok farklı sektörde kullanımı için baskın bir alternatif olduğunu ortaya koymuştur.

Anahtar Kelimeler

• Hareketten nesne oluşturma
• Akıllı telefon ve tabletler
• LiDAR
• 3B nokta bulutu üretimi
• Doğruluk analizi

Destekleyen Kurum

Yıldız Teknik Üniversitesi Bilimsel Araştırma Projeleri Koordinasyon Birimi

Proje Numarası

FBA-2021-4295

Teşekkür

Bu proje, Yıldız Teknik Üniversitesi Bilimsel Araştırma Projeleri Koordinasyon Birimi tarafından (Proje No: FBA-2021-4295) desteklenmiştir.

Kaynakça

1. [1] Roering J.J., Mackey B.H., Marshall J.A., Sweeney K.E., Deligne N.I., Booth A.M., Handwerger A.L., Cerovski Darriau C., ‘You are HERE’: Connecting the dots with airborne lidar for geomorphic fieldwork, Geomorphology, 200, 172-183, DOI: 10.1016/j.geomorph.2013.04.009, 2013.
2. [2] Jazayeri I., Rajabifard A., Kalantari M., A geometric and semantic evaluation of 3D data sourcing methods for land and property information, Land Use Policy, 36, 219-230, DOI: 10.1016/j.landusepol.2013.08.004, 2014.
3. [3] McKean J., Roering J., Objective landslide detection and surface morphology mapping using high-resolution airborne laser altimetry, Geomorpholog, 57, 331-351, DOI: 10.1016/S0169-555X (03)00164-8, 2004.
4. [4] Fuller T.K., Perg L.A., Willenbring J.K., Lepper K., Field evidence for climate-driven changes in sediment supply leading to strath terrace formation, Geology, 37, 467-470, DOI:10.1130/G25487A.1, 2009.
5. [5] Van Den Eeckhaut M., Poesen J., Gullentops F., Vandekerckhove L., Hervás J., Regional mapping and characterisation of old landslides in hilly regions using LiDAR-based imagery in Southern Flanders, Quaternary Research, 75, 721-733, DOI:10.1016/j.yqres.2011.02.006, 2011.
6. [6] Ventura G., Vilardo G., Terranova C., Sessa E.B., Tracking and evolution of complex active landslides by multitemporal airborne LiDAR data: The Montaguto landslide (Southern Italy), Remote Sensing of Environment, 115, 3237-3248, DOI:10.1016/j.res.2011.07.007, 2011.
7. [7] Jerolmack D.J., Ewing R.C., Falcini F., Martin R.L., Masteller C., Phillips C., Reitz M., Buynevich I., Internal boundary layer model for the evolution of desert dune fields, Nature Geoscience, 5, 206–209, DOI: 10.1038/ngeo1381, 2012.
8. [8] Brunier G., Fleury J., Anthony, E.J., Gardel A., Dussouillez P., Close-range airborne Structure-from-Motion Photogrammetry for high-resolution beach morphometric surveys: Examples from an embayed rotating beach, Geomorphology, 261, 76-88, DOI: 10.1016/j.geomorph.2016.02.025, 2016.

9. [9] Dietrich J.T., River scape mapping with helicopter-based Structure-from-Motion photogrammetry, Geomorphology, 252, 144-157, DOI: 10.1016/j.geomorph.2015.05.008, 2016.
10. [10] Barazzetti L., Binda L., Scaioni M., Taranto P., Photogrammetric survey of complex geometries with low-cost software: Application to the ‘G1’ temple in Myson, Vietnam, Journal of Cultural Heritage, 12, 253-262, DOI: 10.1016/j.culher.2010.12.004, 2011.
11. [11] Dorshow W.B., Modeling agricultural potential in Chaco Canyon during the Bonito phase: a predictive geospatial approach, Journal of Archaeological Science, 39, 2098-2115, DOI:10.1016/j.jas.2012.02.004, 2012.
12. [12] Clapuyt F., Vanacker V., Van Oost K., Reproducibility of UAV based earth topography reconstructions based on Structure-from-Motion algorithms, Geomorphology, 260, 4-15, DOI:10.1016/j.geomorph.2015.05.011, 2016.
13. [13] Baier W., Rando C., Developing the use of Structure-from-Motion in mass grave documentation, Forensic science international, 261, 19-25, DOI: 10.1016/j.forsciint.2015.12.008, 2016.
14. [14] Hesse R., Combining Structure-from-Motion with high and intermediate resolution satellite images to document threats to archaeological heritage in arid environments, Journal of Cultural Heritage, 2, 192–201, DOI: 10.1016/j.culher.2014.04.003, 2016.
15. [15] Zhang P., Arre T.J., Ide-Ektessabi A., A line scan camera based structure from motion for high-resolution 3D reconstruction, Journal of Cultural Heritage, 5, 656–663, DOI: 10.1016/j.culher.2015.01.003, 2016.
16. [16] Tsui O.W., Coops N.C., Wulder M.A., Marshall P.L., Integrating airborne LiDAR and space-borne radar via multivariate kriging to estimate above-ground biomass, Remote Sensing of Environment, 139, 340-352, DOI: 10.1016/j.res.2013.08.012, 2013.
17. [17] Huang C., Peng Y., Lang M., Yeo I.Y., McCarty G., Wetland inundation mapping and change monitoring using Landsat and airborne LiDAR data, Remote Sensing of Environment, 141, 231-242, DOI: 10.1016/j.res.2013.10.020, 2014.
18. [18] Reese H., Nyström M., Nordkvist K., Olsson H., Combining airborne laser scanning data and optical satellite data for classification of alpine vegetation, International Journal of Applied Earth Observation and Geoinformation, 27, 81-90, DOI:10.1016/j.jag.2013.05.003, 2014.
19. [19] Vousdoukas M.I., Kirupakaramoorthy T., Oumeraci H., de la Torre M., Wübbold F., Wagner B., Schimmels S., The role of combined laser scanning and video techniques in monitoring wave-by-wave swash zone processes, Coastal Engineering, 83, 150-165, DOI: 10.1016/j.costaleng.2013.10.013, 2014.
20. [20] Leon J.X., Roelfsema Ch.M., Saunders M.I., Phinn S.R., Measuring coral reef terrain roughness using ‘Structure-from-Motion’ close-range photogrammetry, Geomorphology, 242, 21-28, DOI: 10.1016/j.geomorph.2015.01.030, 2015.
21. [21] Jay S., Rabatel G., Hadoux X., Moura D., Gorretta N., In-field crop row phenotyping from 3D modeling performed using Structure from Motion, Computers and Electronics in Agriculture, 110, 70-77, DOI: 10.1016/j.compag.2014.09.021, 2015.
22. [22] Armesto J., Roca-Pardińas J., Lorenzo H., Arias P., Modelling masonry arches shape using terrestrial laser scanning data and nonparametric methods, Engineering Structures, 32, 607-615, DOI: 10.1016/j.engstruct.2009.11.007, 2010.
23. [23] Bhatla A., Choe S.Y., Fierro O., Leite F., Evaluation of accuracy of as-built 3D modeling from photos taken by handheld digital camera, Automation in Construction, 28, 116–127, DOI:10.1016/j.autcon.2012.06.003, 2012.
24. [24] González-Jorge H., Riveiro B., Arias P., Armesto J., Photogrammetry and laser scanner technology applied to length measurements in car testing laboratories, Measurement, 45, 354-363, DOI: 10.1016/j.measurement.2011.11.010, 2012.
25. [25] Srinivasan S., Popescu S., Eriksson M., Sheridan R., Ku N.W., Terrestrial laser scanning as an effective tool to retrieve tree level height, crown width, and stem diameter, Remote Sens, 7, 1877–1896, 2015.
26. [26] Liang X., Kankare V., Hyyppä J., Wang Y., Kukko A., Haggrén H., Holopainen M., Terrestrial laser scanning in forest inventories, ISPRS J. Photogram. Remote Sens, 115, 63–77, 2016.
27. [27] Liang X., Hyyppä J., Kaartinen H., Lehtomäki M., Pyörälä J., Pfeifer N., Huang H., International benchmarking of terrestrial laser scanning approaches for forest inventories, ISPRS J. Photogram. Remote Sens, 144, 137–179, 2018a.
28. [28] Liang X., Kukko A., Hyyppä J., Lehtomäki M., Pyörälä J., Yu X., Wang Y., Insitu measurements from mobile platforms: an emerging approach to address the old challenges associated with forest inventories, ISPRS J. Photogram. Remote Sens, 143, 97–107, 2018b.
29. [29] Brasington J., Vericat D., Rychkov I., Modeling riverbed morphology, roughness, and surface sedimentology using high-resolution terrestrial laser scanning, Water Resour. Res., 48(11), 2012.
30. [30] Mali V.K., Kuiry S.N., Assessing the accuracy of high-resolution topographic data generated using freely available packages based on SfM-MVS approach, Measurement, 124 (2018) 338–350, 2018.
31. [31] Chandler J., Ashmore P., Paola C., Gooch M., Varkaris F., Monitoring river-channel change using terrestrial oblique digital imagery and automated digital photogrammetry, Ann. Assoc. Am. Geogr., 92, 631–644, http://dx.doi.org/10.1111/1467-8306.00308, 2002.
32. [32] Carbonneau P.E., Lane S.N., Bergeron N.E., Cost-effective non-metric close-range digital photogrammetry and its application to a study of coarse gravel river beds, Int. J. Remote Sens., 24, 2837–2854, http://dx.doi.org/10.1080/01431160110108364, 2003.
33. [33] Lane S.N., Widdison P.E., Thomas R.E., Ashworth P.J., Best J.L., Lunt I.A., Sambrook Smith G.H., Simpson C.J. Quantification of braided river channel change using archival digital image analysis, Earth Surf. Process. Landforms, 35, 2010.
34. [34] Helle R.H., Lemu H.G., A case study on use of 3D scanning for reverse engineering and quality control, Materials Today: Proceedings, 45, 5255-5262, 2021.
35. [35] Gümüşboğa İ., Design of an automated stock-taking system based on unmanned aerial vehicles, Journal of the Faculty of Engineering and Architecture of Gazi University, 37 (4), 1767-1782, 2022.
36. [36] Şener Z., Uzar, M., New trend in object oriented image analysis - ontology, Journal of the Faculty of Engineering and Architecture of Gazi University, 35 (1), 479-494, 2020.
37. [37] Uzar M., Tunalioglu N., Arican D., Arda T., Investigation of the filtering methods on 3D models using terrestrial laser scanning data, Journal of the Faculty of Engineering and Architecture of Gazi University, 34(4), 2019.
38. [38] Chghaf M., Rodriguez S., Ouardi A.E., Camera, LiDAR and multi-modal SLAM systems for autonomous ground vehicles: a survey, Journal of Intelligent & Robotic Systems, 105(1), 1-35, 2022.
39. [39]URL1:https://appft1.uspto.gov/netacgi/nphParser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220200309955%22.PGNR.&OS=DN/20200309955&RS=DN/20200309955
40. [40]URL2:https://patft.uspto.gov/netacgi/nphParser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1=10,955,234.PN.&OS=PN/10,955,234&RS=PN/10,955,234
41. [41] Jeong H., Ahn H., Shin D., Ahn Y., Choi C., A Comparative Assessment of the Photogrammetric Accuracy of Mapping Using UAVs with Smart Devices, Photogrammetric Engineering & Remote Sensing, 85 (12), 889–897. 0099-1112/19/889–897, 2019.
42. [42] Wróżyński R., Pyszny K., Sojka M., Przybyła C., Murat-Błażejewska S., Ground volume assessment using Structure from Motion photogrammetry with a smartphone and a compact camera, Open Geosci., 9, 281–294, 2017.
43. [43] Stitt J.M., Svancara L.K., Vierling L.A., Vierling K.T., Smartphone LIDAR Can Measure Tree Cavity Dimensions for Wildlife Studies, Wildlife Society Bulletin, 43(1), 159–166, DOI: 10.1002/wsb.949, 2019.
44. [44] Fan G., Dong Y., Chen D., Chen F., New Method for Forest Resource Data Collection Based on Smartphone Fusion with Multiple Sensors, Mobile Information Systems, 2020.
45. [45] Luetzenburg G., Kroon A., Bjørk A.A., Evaluation of the Apple iPhone 12 Pro LiDAR for an Application in Geosciences. Scientific reports, 11(1), 1-9, 2021.
46. [46] Tavani S., Billi A., Corradetti A., Mercuri M., Bosman A., Cuffaro M., ... , Carminati E., Smartphone assisted fieldwork: Towards the digital transition of geoscience fieldwork using LiDAR-equipped iPhones, Earth-Science Reviews, 103969, 2022.
47. [47] King F., Kelly R., Fletcher C.G., Evaluation of LiDAR-derived Snow Depth Estimates from the iPhone 12 Pro, IEEE Geoscience and Remote Sensing Letters, 2022.
48. [48] Westoby M.J., Brasington J., Glasser N.F., Hambrey M.J., Reynolds J.M., Structure-from-Motion photogrammetry: A low cost, effective tool for geoscience applications, Geomorphology, 179, 300-314, DOI: 10.1016/j.geomorph.2012.08.021, 2012.
49. [49] Sarıtürk B., Şeker D.Z., Sfm Tekniği ile 3B Obje Modellenmesinde Kullanılan Ticari ve Açık-Kaynak Kodlu Yazılımların Karşılaştırılması, AKÜ FEMÜBİD, 17, 126-131, 2017.
50. [50] Mahami H., Nasirzadeh F., Hosseininaveh Ahmadabadian A., Nahavandi S., Automated progress controlling and monitoring using daily site images and building information modelling. Buildings, 9 (3), 70, 2019.