Assessing the contribution of RGB VIs in improving building extraction from RGB-UAV images

Abstract

Buildings are a fundamental component of the built environment, and accurate information regarding their size, location, and distribution is vital for various purposes. The ever-increasing capabilities of unmanned aerial vehicles (UAVs) have sparked an interest in exploring various techniques to delineate buildings from the very high-resolution images obtained from UAV photogrammetry. However, the limited spectral information in UAV images, particularly the number of bands, can hinder the differentiation between various materials and objects. This setback can affect the ability to distinguish between different materials and objects. To address this limitation, vegetative ındices (VIs) have been employed to enhance the spectral strength of UAV orthophotos, thereby improving building classification. The objective of this study is to evaluate the contribution of four specific VIs: the green leaf index (GLI), red-green-blue vegetation index (RGBVI), visual atmospherically resistant index (VARI), and triangular greenness index (TGI). The significance of this contribution lies in assessing the potential of each VI to enhance building classification. The approach utilized the geographic object-based image analysis (GeoBIA) approach and a random forest classifier. To achieve this aim, five datasets were created, with each dataset comprising the RGB-UAV image and a corresponding RGB VI. The experimental results on the test dataset and a post-classification assessment indicated a general improvement in the classification when the VIs were added to the RGB orthophoto.

Keywords

• Building extraction
• UAV
• GeoBIA
• RGB vegetative indices
• Random forest

Ethical Statement

The authors declare that the submitted manuscript is original. The authors also acknowledge that the current research has been conducted ethically, and all authors have agreed to the final shape of the research. The authors declare that this manuscript does not involve researching humans or animals.

References

1. Schlosser, A. D., Szabó, G., Bertalan, L., Varga, Z., Enyedi, P., & Szabó, S. (2020). Building extraction using orthophotos and dense point cloud derived from visual band aerial imagery based on machine learning and segmentation. Remote Sensing, 12(15), 2397. https://doi.org/10.3390/rs12152397 Hu, Q., Zhen, L., Mao, Y., Zhou, X., & Zhou, G. (2021). Automated building extraction using satellite remote sensing imagery. Automation in Construction, 123, 103509. https://doi.org/10.1016/j.autcon.2020.103509
2. Li, J., Huang, X., Tu, L., Zhang, T., & Wang, L. (2022). A review of building detection from very high resolution optical remote sensing images. GIScience & Remote Sensing, 59(1), 1199-1225. https://doi.org/10.1080/15481603.2022.2101727
3. Dai, Y., Gong, J., Li, Y., & Feng, Q. (2017). Building segmentation and outline extraction from UAV image-derived point clouds by a line growing algorithm. International Journal of Digital Earth, 10(11), 1077-1097. https://doi.org/10.1080/17538947.2016.1269841
4. Temenos, A., Temenos, N., Doulamis, A., & Doulamis, N. (2022). On the exploration of automatic building extraction from RGB satellite images using deep learning architectures based on U-Net. Technologies, 10(1), 19. https://doi.org/10.3390/technologies10010019
5. Daranagama, S., & Witayangkurn, A. (2021). Automatic building detection with polygonizing and attribute extraction from high-resolution images. ISPRS International Journal of Geo-Information, 10(9), 606. https://doi.org/10.3390/ijgi10090606
6. Lin, Huertas, & Nevatia. (1994). Detection of buildings using perceptual grouping and shadows. In 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 62-69. https://doi.org/10.1109/CVPR.1994.323811
7. Jaynes, C. O., Stolle, F., & Collins, R. T. (1994, December). Task driven perceptual organization for extraction of rooftop polygons. In Proceedings of 1994 IEEE Workshop on Applications of Computer Vision, 152-159. https://doi.org/10.1109/ACV.1994.341303
8. Chen, R., Li, X., & Li, J. (2018). Object-based features for house detection from RGB high-resolution images. Remote Sensing, 10(3), 451. https://doi.org/10.3390/rs10030451

9. Lu, T., Ming, D., Lin, X., Hong, Z., Bai, X., & Fang, J. (2018). Detecting building edges from high spatial resolution remote sensing imagery using richer convolution features network. Remote Sensing, 10(9), 1496. https://doi.org/10.3390/rs10091496
10. Cheng, G., & Han, J. (2016). A survey on object detection in optical remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, 117, 11-28. https://doi.org/10.1016/j.isprsjprs.2016.03.014
11. Lefèvre, S., Weber, J., & Sheeren, D. (2007). Automatic building extraction in VHR images using advanced morphological operators. In 2007 Urban Remote Sensing Joint Event, 1-5. https://doi.org/10.1109/URS.2007.371825
12. Ahmadi, S., Zoej, M. V., Ebadi, H., Moghaddam, H. A., & Mohammadzadeh, A. (2010). Automatic urban building boundary extraction from high resolution aerial images using an innovative model of active contours. International Journal of Applied Earth Observation and Geoinformation, 12(3), 150-157. https://doi.org/10.1016/j.jag.2010.02.001
13. Yari, D., Mokhtarzade, M., Ebadi, H., & Ahmadi, S. (2014). Automatic reconstruction of regular buildings using a shape‐based balloon snake model. The Photogrammetric Record, 29(146), 187-205. https://doi.org/10.1111/phor.12060
14. Huertas, A., & Nevatia, R. (1988). Detecting buildings in aerial images. Computer Vision, Graphics, and Image Processing, 41(2), 131-152. https://doi.org/10.1016/0734-189X(88)90016-3
15. Peng, J., & Liu, Y. C. (2005). Model and context‐driven building extraction in dense urban aerial images. International Journal of Remote Sensing, 26(7), 1289-1307. https://doi.org/10.1080/01431160512331326675
16. Sirmacek, B., & Unsalan, C. (2008). Building detection from aerial images using invariant color features and shadow information. In 2008 23rd International Symposium on Computer and Information Sciences, 1-5. https://doi.org/10.1109/ISCIS.2008.4717854
17. Liow, Y. T., & Pavlidis, T. (1990). Use of shadows for extracting buildings in aerial images. Computer Vision, Graphics, and Image Processing, 49(2), 242-277. https://doi.org/10.1016/0734-189X(90)90139-M
18. Irvin, R. B., & McKeown, D. M. (1989). Methods for exploiting the relationship between buildings and their shadows in aerial imagery. IEEE Transactions on Systems, Man, and Cybernetics, 19(6), 1564-1575. https://doi.org/10.1109/21.44071
19. Wu, G., Shao, X., Guo, Z., Chen, Q., Yuan, W., Shi, X., ... & Shibasaki, R. (2018). Automatic building segmentation of aerial imagery using multi-constraint fully convolutional networks. Remote Sensing, 10(3), 407. https://doi.org/10.3390/rs10030407
20. Kokeza, Z., Vujasinović, M., Govedarica, M., Milojević, B., & Jakovljević, G. (2020). Automatic building footprint extraction from UAV images using neural networks. Geodetski Vestnik, 64(04), 545-561. https://doi.org/10.15292/geodetski-vestnik.2020.04.545-561
21. Norman, M., Shahar, H. M., Mohamad, Z., Rahim, A., Mohd, F. A., & Shafri, H. Z. M. (2021). Urban building detection using object-based image analysis (OBIA) and machine learning (ML) algorithms. In IOP Conference Series: Earth and Environmental Science, 620(1), 012010. https://doi.org/10.1088/1755-1315/620/1/012010
22. Comert, R., & Kaplan, O. (2018). Object based building extraction and building period estimation from unmanned aerial vehicle data. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 4(3), 71-76. https://doi.org/10.5194/isprs-annals-IV-3-71-2018
23. Aminipouri, M. (2009). Object-oriented analysis of very high resolution orthophotos for estimating the population of slum areas, case of Dar-Es-Salaam, Tanzania [Master's thesis, University of Twente].
24. Guo, Z., & Du, S. (2017). Mining parameter information for building extraction and change detection with very high-resolution imagery and GIS data. GIScience & Remote Sensing, 54(1), 38-63. https://doi.org/10.1080/15481603.2016.1250328
25. Benarchid, O., Raissouni, N., El Adib, S., Abbous, A., Azyat, A., Achhab, N. B., ... & Chahboun, A. (2013). Building extraction using object-based classification and shadow information in very high resolution multispectral images, a case study: Tetuan, Morocco. Canadian Journal on Image Processing and Computer Vision, 4(1), 1-8.
26. Frishila, A. A., & Kamal, M. (2019). The effectiveness of spectral features for building extraction using geographic object-based image analysis (GEOBIA). The 40th Asian Conference on Remote Sensing (ACRS 2019), 1-10.
27. Hossain, M. D., & Chen, D. (2022). A hybrid image segmentation method for building extraction from high-resolution RGB images. ISPRS Journal of Photogrammetry and Remote Sensing, 192, 299-314. https://doi.org/10.1016/j.isprsjprs.2022.08.024
28. Dornaika, F., Moujahid, A., El Merabet, Y., & Ruichek, Y. (2016). Building detection from orthophotos using a machine learning approach: An empirical study on image segmentation and descriptors. Expert Systems with Applications, 58, 130-142. https://doi.org/10.1016/j.eswa.2016.03.024
29. Argyridis, A., & Argialas, D. P. (2016). Building change detection through multi-scale GEOBIA approach by integrating deep belief networks with fuzzy ontologies. International Journal of Image and Data Fusion, 7(2), 148-171. https://doi.org/10.1080/19479832.2016.1158211
30. Davydova, K., Cui, S., & Reinartz, P. (2016). Building footprint extraction from digital surface models using neural networks. In Image and Signal Processing for Remote Sensing XXII, 10004, 187-196. https://doi.org/10.1117/12.2240727
31. Li, Y., Zhu, L., Shimamura, H., & Tachibanab, K. (2010). An integrated system on large scale building extraction from DSM. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 38, 35-39.
32. Singh, D., Maurya, R., Shukla, A. S., Sharma, M. K., & Gupta, P. R. (2012). Building extraction from very high resolution multispectral images using NDVI based segmentation and morphological operators. In 2012 Students Conference on Engineering and Systems, 1-5. https://doi.org/10.1109/SCES.2012.6199034
33. Öztürk, M. Y., & Çölkesen, İ. (2021). The impacts of vegetation indices from UAV-based RGB imagery on land cover classification using ensemble learning. Mersin Photogrammetry Journal, 3(2), 41-47. https://doi.org/10.53093/mephoj.943347
34. Kucharczyk, M., Hay, G. J., Ghaffarian, S., & Hugenholtz, C. H. (2020). Geographic object-based image analysis: a primer and future directions. Remote Sensing, 12(12), 2012. https://doi.org/10.3390/rs12122012
35. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., & Süsstrunk, S. (2012). SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11), 2274-2282. https://doi.org/10.1109/TPAMI.2012.120
36. Liao, N., Liu, H., Li, C., Ren, X., & Guo, B. (2022). Simple linear ıterative clustering with efficiency. Advances in Intelligent Information Hiding and Multimedia Signal Processing, 1, 109-117. https://doi.org/10.1007/978-981-19-1057-9_11
37. Zhang, H., & Zhu, Y. (2019). Kslic: K-mediods clustering based simple linear iterative clustering. In Chinese Conference on Pattern Recognition and Computer Vision (PRCV), 519-529. https://doi.org/10.1007/978-3-030-31723-2_44
38. Sibaruddin, H. I., Zulhaidi, H., Shafri, M., Pradhan, B., & Haron, N. A. (2018). UAV-based approach to extract topographic and as-built information by utilising the OBIA technique. Journal of Geosciences and Geomatics, 6(3), 103-123. https://doi.org/10.12691/jgg-6-3-2
39. Norman, M., Shahar, H. M., Mohamad, Z., Rahim, A., Mohd, F. A., & Shafri, H. Z. M. (2021). Urban building detection using object-based image analysis (OBIA) and machine learning (ML) algorithms. IOP Conference Series: Earth and Environmental Science, 620(1), 1-11. https://doi.org/10.1088/1755-1315/620/1/012010
40. Breiman, L. (2001). Random forests. Machine Learning, 45, 5-32.
41. Kumar, A., & Sinha, N. (2020). Classification of forest cover type using random forests algorithm. In Advances in Data and Information Sciences: Proceedings of ICDIS 2019, 94, 395-402. https://doi.org/10.1007/978-981-15-0694-9_37
42. Xiao, Y., Huang, W., & Wang, J. (2020). A random forest classification algorithm based on dichotomy rule fusion. In 2020 IEEE 10th International Conference on Electronics Information and Emergency Communication (ICEIEC), 182-185. https://doi.org/10.1109/ICEIEC49280.2020.9152236
43. Jiang, J., Cai, W., Zheng, H., Cheng, T., Tian, Y., Zhu, Y., ... & Yao, X. (2019). Using digital cameras on an unmanned aerial vehicle to derive optimum color vegetation indices for leaf nitrogen concentration monitoring in winter wheat. Remote Sensing, 11(22), 2667. https://doi.org/10.3390/rs11222667
44. Hunt Jr, E. R., Doraiswamy, P. C., McMurtrey, J. E., Daughtry, C. S., Perry, E. M., & Akhmedov, B. (2013). A visible band index for remote sensing leaf chlorophyll content at the canopy scale. International Journal of Applied Earth Observation and Geoinformation, 21, 103-112. https://doi.org/10.1016/j.jag.2012.07.020
45. Bendig, J., Yu, K., Aasen, H., Bolten, A., Bennertz, S., Broscheit, J., ... & Bareth, G. (2015). Combining UAV-based plant height from crop surface models, visible, and near infrared vegetation indices for biomass monitoring in barley. International Journal of Applied Earth Observation and Geoinformation, 39, 79-87. https://doi.org/10.1016/j.jag.2015.02.012
46. Gitelson, A. A., Kaufman, Y. J., Stark, R., & Rundquist, D. (2002). Novel algorithms for remote estimation of vegetation fraction. Remote Sensing of Environment, 80(1), 76-87. https://doi.org/10.1016/S0034-4257(01)00289-9
47. Louhaichi, M., Borman, M. M., & Johnson, D. E. (2001). Spatially located platform and aerial photography for documentation of grazing impacts on wheat. Geocarto International, 16(1), 65-70. https://doi.org/10.1080/10106040108542184
48. Kaur, R., & Pandey, P. (2022). A review on spectral indices for built-up area extraction using remote sensing technology. Arabian Journal of Geosciences, 15(5), 391. https://doi.org/10.1007/s12517-022-09688-x
49. Tsai, Y. H., Stow, D., & Weeks, J. (2011). Comparison of object-based image analysis approaches to mapping new buildings in Accra, Ghana using multi-temporal QuickBird satellite imagery. Remote Sensing, 3(12), 2707-2726. https://doi.org/10.3390/rs3122707