Object-Based Integration Using Deep Learning and Multi-Resolution Segmentation in Building Extraction from Very High Resolution Satellite Imagery

Abstract

In recent years, there has been an increase in studies on the analysis of urban areas and the detection of changes in a fast and reliable way. In this respect, the classification of buildings is one of the prominent current issues of computer vision. As in many areas, the use of deep learning architectures is among the trending applications. Semantic segmentation applications have become widespread by using convolutional neural networks (CNN) to determine the building footprint. However, at the beginning of the problems encountered in the prediction images obtained after segmentation processes with deep learning, the noise formed by the effect of salt and pepper comes. In this study, the integration of the use of U-Net and SegNet algorithms, which are among the state-of-the-art CNN architectures, with the Object-Based Image Analysis (OBIA) and Multi-Resolution Segmentation (MRS) algorithm is used. Experiments were performed on the open shared Wuhan University Building Inference Dataset (WHUBED) consisting of very high-resolution satellite images (Gaofen-2, Worldview-2 and Ikonos). The model in the study, provides improvements in overall accuracy, F1 score, Dice score and Intersection over Union (IoU) metrics over the prediction results obtained using CNN alone. Building footprint maps obtained by building extraction are presented in the last section as comparative images.

Keywords

• Deep learning
• Object-Based Image Analysis
• Multi-Resoluion Segmentation
• İntegration

Çok Yüksek Çözünürlüklü Uydu Görüntülerinden Bina Çıkarımında Derin Öğrenme ve Çoklu-Çözünürlüklü Bölütleme Kullanılarak Nesne-Tabanlı Entegrasyon

Öz

Son yıllarda, kentsel alanlarda yapılan analizler ve değişimlerin tespitinin hızlı ve güvenilir şekilde gerçekleştirilmesi konusundaki çalışmalarda artış olmuştur. Bu doğrultuda, binaların sınıflandırılması bilgisayarlı görünün ön plana çıkan güncel konularından biridir. Birçok alanda olduğu gibi bu konuda da derin öğrenme mimarilerinin kullanımı trend uygulamalar arasındadır. Bina ayak izinin belirlenmesi amacıyla evrişimsel sinir ağları (ESA) kullanılarak semantik segmentasyon uygulamaları yaygınlaşmıştır. Ancak derin öğrenme ile segmentasyon işlemleri sonrası elde edilen tahmin görüntülerinde karşılaşılan problemlerin başında tuz-biber etkisiyle oluşmuş gürültüler gelmektedir. Bu çalışmada güncel ESA mimarilerinden olan U-Net ve SegNet algoritmalarının kullanımının, Nesne-Tabanlı Görüntü Analizinin (NTGA), Çoklu-Çözünürlüklü Bölütleme (ÇÇB) algoritmasıyla entegrasyonu kullanılmıştır. Deneyler çok yüksek çözünürlüklü uydu görüntülerinden (Gaofen-2, Worldview-2 ve Ikonos) oluşan açık paylaşımlı Wuhan Üniversitesi Bina Çıkarımı Veri seti (WHUBED) üzerinde gerçekleştirilmiştir. ESA+ÇÇB modeli genel doğruluk, F1 skor, Dice skoru ve Intersection over Union (IoU) metriklerinde, sadece ESA kullanımıyla elde edilen tahmin sonuçlarına göre iyileştirmeler sağlamıştır. Bina sınıflandırılması ile elde edilen haritalar karşılaştırılmalı görseller olarak son kısımda sunulmuştur.

Anahtar Kelimeler

• derin öğrenme
• çoklu-çözünürlüklü bölütleme
• nesne-tabanlı görüntü analizi
• entegrasyon

References

1. Abdollahi, A., Pradhan, B., & Alamri, A. M. (2022). An ensemble architecture of deep convolutional Segnet and Unet networks for building semantic segmentation from high-resolution aerial images. Geocarto International, 37(12), 3355-3370.
2. Ali, K., & Johnson, B. A. (2022). Land-Use and Land-Cover Classification in Semi-Arid Areas from Medium-Resolution Remote-Sensing Imagery: A Deep Learning Approach. Sensors, 22(22), 8750.
3. Arıkan, D. & Yıldız, F. (2023). Göktürk-1 Uydu Görüntülerinden U-Net Modeli Kullanılarak Binaların Segmentasyonu . Türkiye Uzaktan Algılama Dergisi , 5 (1) , 50-58 . DOI: 10.51489/tuzal.1300939
4. Atik, S. O., & Ipbuker, C. (2021). Integrating convolutional neural network and multiresolution segmentation for land cover and land use mapping using satellite imagery. Applied Sciences, 11(12), 5551.
5. Atik, S. O., Atik, M. E., & Ipbuker, C. (2022). Comparative research on different backbone architectures of DeepLabV3+ for building segmentation. Journal of Applied Remote Sensing, 16(2), 024510-024510.
6. Atik, S. O., & Ipbuker, C. (2022). Building Extraction in VHR Remote Sensing Imagery Through Deep Learning. Fresenius Environ. Bull, 31, 8468-8473.
7. Attri, P., Chaudhry, S., & Sharma, S. (2015). Remote sensing & GIS based approaches for LULC change detection—A review. Int. J. Curr. Eng. Technol, 5, 3126-3137.4e
8. Baatz, M. (2000). Multi resolution segmentation: an optimum approach for high quality multi scale image segmentation. In Beutrage zum AGIT-Symposium. Salzburg, Heidelberg, 2000 (pp. 12-23).

9. Badrinarayanan V., Kendall A. and Cipolla R., (2017) "SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 12, pp. 2481-2495, 1 Dec., doi: 10.1109/TPAMI.2016.2644615.
10. Balarabe, A. T., & Jordanov, I. (2021). LULC image classification with convolutional neural network. In 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS (pp. 5985-5988). IEEE.
11. Baştuğ Koç, A., Akgün, D. (2021). U-net Mimarileri ile Glioma Tümör Segmentasyonu Üzerine Bir Literatür Çalışması. Avrupa Bilim ve Teknoloji Dergisi, (26), 407-414.
12. Bi, Q., Qin, K., Zhang, H., Zhang, Y., Li, Z., & Xu, K. (2019). A multi-scale filtering building index for building extraction in very high-resolution satellite imagery. Remote Sensing, 11(5), 482.
13. de Pinho, C. M. D., Fonseca, L. M. G., Korting, T. S., De Almeida, C. M., & Kux, H. J. H. (2012). Land-cover classification of an intra-urban environment using high-resolution images and object-based image analysis. International Journal of Remote Sensing, 33(19), 5973-5995.
14. Dewali, S. K., Jain, K., Varshney, D., Dhamija, S., & Pundir, E. (2023). Combining OBIA, CNN, and UAV Photogrammetry for Automated Avalanche Deposit Detection and Characterization. Advances in Space Research.
15. Ghorbanzadeh, O., Gholamnia, K., & Ghamisi, P. (2022). The application of ResU-net and OBIA for landslide detection from multi-temporal sentinel-2 images. Big Earth Data, 1-26.
16. Ghorbanzadeh, O., Tiede, D., Wendt, L., Sudmanns, M., & Lang, S. (2021). Transferable instance segmentation of dwellings in a refugee camp-integrating CNN and OBIA. European Journal of Remote Sensing, 54(sup1), 127-140.
17. Hossain, M. D., & Chen, D. (2019). Segmentation for Object-Based Image Analysis (OBIA): A review of algorithms and challenges from remote sensing perspective. ISPRS Journal of Photogrammetry and Remote Sensing, 150, 115-134.
18. Ji, S., Wei, S., & Lu, M. (2019). A scale robust convolutional neural network for automatic building extraction from aerial and satellite imagery. International journal of remote sensing, 40(9), 3308-3322.
19. Kaiser, P., Wegner, J. D., Lucchi, A., Jaggi, M., Hofmann, T., & Schindler, K. (2017). Learning aerial image segmentation from online maps. IEEE Transactions on Geoscience and Remote Sensing, 55(11), 6054-6068.
20. Kawamura, K., Asai, H., Yasuda, T., Soisouvanh, P., & Phongchanmixay, S. (2021). Discriminating crops/weeds in an upland rice field from UAV images with the SLIC-RF algorithm. Plant Production Science, 24(2), 198-215.
21. Kurnaz, E., & Ceylan, R. (2020, October). Pancreas segmentation in abdominal CT images with U-Net model. In 2020 28th Signal Processing and Communications Applications Conference (SIU) (pp. 1-4). IEEE.
22. Li, H., Tian, Y., Zhang, C., Zhang, S., & Atkinson, P. M. (2022). Temporal sequence Object-based CNN (TS-OCNN) for crop classification from fine resolution remote sensing image time-series. The Crop Journal, 10(5), 1507-1516.
23. Liu, T., Yang, L., & Lunga, D. (2021). Change detection using deep learning approach with object-based image analysis. Remote Sensing of Environment, 256, 112308.
24. Manaf, S. A., Mustapha, N., Sulaiman, M. N., Husin, N. A., Shafri, H. Z. M., & Razali, M. N. (2018). Hybridization of SLIC and Extra Tree for Object Based Image Analysis in Extracting Shoreline from Medium Resolution Satellite Images. International Journal of Intelligent Engineering & Systems, 11(1).
25. Marcu, A., & Leordeanu, M. (2016). Dual local-global contextual pathways for recognition in aerial imagery. arXiv preprint arXiv:1605.05462.
26. Merchant, M. A. (2021). Classification of open water features using OBIA and deep learning. In 2021 IEEE international geoscience and remote sensing symposium IGARSS (pp. 104-107). IEEE.
27. Pan, Z., Xu, J., Guo, Y., Hu, Y., & Wang, G. (2020). Deep learning segmentation and classification for urban village using a worldview satellite image based on U-Net. Remote Sensing, 12(10), 1574.
28. Qin, Y., Wu, Y., Li, B., Gao, S., Liu, M., & Zhan, Y. (2019). Semantic segmentation of building roof in dense urban environment with deep convolutional neural network: A case study using GF2 VHR imagery in China. Sensors, 19(5), 1164.
29. Ronneberger, O., Fischer, P., Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab, N., Hornegger, J., Wells, W., Frangi, A. (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. MICCAI 2015. Lecture Notes in Computer Science, vol 9351. Springer, Cham. https://doi.org/10.1007/978-3-319-24574-4_28
30. Lee, S. H., & Baik, J. J. (2011). Evaluation of the vegetated urban canopy model (VUCM) and its impacts on urban boundary layer simulation. Asia-Pacific Journal of Atmospheric Sciences, 47, 151-165.
31. Tang, Z., Li, M., & Wang, X. (2020). Mapping tea plantations from VHR images using OBIA and convolutional neural networks. Remote Sensing, 12(18), 2935.
32. Thenkabail, P. S., Schull, M., & Turral, H. (2005). Ganges and Indus river basin land use/land cover (LULC) and irrigated area mapping using continuous streams of MODIS data. Remote Sensing of Environment, 95(3), 317-341.
33. Tian, J., & Chen, D. M. (2007). Optimization in multi‐scale segmentation of high‐resolution satellite images for artificial feature recognition. International Journal of Remote Sensing, 28(20), 4625-4644.
34. Torunlar, H. , Tuğaç, M. G. & Duyan, K. (2021). Nesne Tabanlı Sınıflandırma Yönteminde Sentinel-2A Uydu Görüntüleri Kullanılarak Tarımsal Ürün Desenlerinin Belirlenmesi; Konya - Karapınar Örneği . Türkiye Uzaktan Algılama Dergisi , 3 (2) , 36-46 . DOI: 10.51489/tuzal.932912
35. Wang, M. (2008). A multiresolution remotely sensed image segmentation method combining rainfalling watershed algorithm and fast region merging. Int. Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 37, 1213-1217.
36. Xu, Z., Su, C., & Zhang, X. (2021). A semantic segmentation method with category boundary for Land Use and Land Cover (LULC) mapping of Very-High Resolution (VHR) remote sensing image. International Journal of Remote Sensing, 42(8), 3146-3165.
37. Yi, Y., Zhang, Z., Zhang, W., Zhang, C., Li, W., & Zhao, T. (2019). Semantic segmentation of urban buildings from VHR remote sensing imagery using a deep convolutional neural network. Remote sensing, 11(15), 1774.