Boundary Extraction Based on Dual Stream Deep Learning Model in High Resolution Remote Sensing Images

Abstract

Boundary extraction in remote sensing has an important task in studies such as environmental observa-tion, risk management and monitoring urban growth. Although significant progress has been made in the different calculation methods proposed, there are issues that need improvement, especially in terms of accuracy, efficiency and speed. In this study, dual stream network architecture of three different models that can obtain boundary extraction by using normalized Digital Surface Model (nDSM), Normalized Difference Vegetation Index (NDVI) and Near-Infrared (IR) band as the second stream, was explained. Model I is designed as the original HED, whereas the second stream of Model II, III, and IV use nDSM, nDSM + NDVI and nDSM + NDVI + IR, respectively. Thus, by comparing the models trained based on different data combinations, the contribution of different input data to the success of boundary extraction was revealed. For the training of the models, boundary maps produced from The International Society for Photogrammetry and Remote Sensing (ISPRS) Potsdam data set and input datasets augmented by rotation, mirroring and rotation were used. When the test results obtained from two-stream and multi-data-based models are evaluated, 11% better accuracy has achieved with Model IV compared to the original HED. The outcomes clearly revealed the importance of using multispectral band, height data and vegetation information as input data in boundary extraction beside commonly used RGB images.

Keywords

• Remote sensing
• photogrammetry
• orthophotos
• deep learning
• boundary extraction

References

1. Altınoluk E., Akçay Ö., Kınacı A. C., Avşar Ö., Polat A. B. & Aydar U. (2020). Effects of Orthophoto Band Combinations on Semantic Segmentation. Intercontinental Geoinformation Days (IGD), (pp. 9-12).
2. Chen, L. C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2014). Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv preprint arXiv:1412.7062.
3. Chen, L. C., Barron, J. T., Papandreou, G., Murphy, K., & Yuille, A. L. (2016). Semantic image segmentation with task-specific edge detection using cnns and a discriminatively trained domain transform. Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4545-4554).
4. Cheng, G., & Han, J. (2016). A survey on object detection in optical remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, 117, 11-28.
5. Dai, J., He, K., & Sun, J. (2016). Instance-aware semantic segmentation via multi-task network cascades. Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3150-3158).
6. Guo, Y., Liu, Y., Georgiou, T., & Lew, M. S. (2018). A review of semantic segmentation using deep neural networks. International journal of multimedia information retrieval, 7(2), 87-93.
7. Han, X., Zhong, Y., & Zhang, L. (2017). An efficient and robust integrated geospatial object detection framework for high spatial resolution remote sensing imagery. Remote Sensing, 9(7), 666.
8. Hariharan, B., Arbelaez, P., Bourdev, L., Maji, S., & Malik, J. (2011). Semantic Contours from Inverse Detectors, International conference on computer vision (pp. 991-998). IEEE.

9. Kinzie, J., & Kuh, G. D. (2004). Going DEEP: Learning from campuses that share responsibility for student success. About Campus, 9(5), 2-8.
10. Kokkinos, I. (2015). Pushing the boundaries of boundary detection using deep learning. arXiv preprint arXiv:1511.07386.
11. Kokkinos, I. (2017). Ubernet: Training a universal convolutional neural network for low-, mid-, and high-level vision using diverse datasets and limited memory. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 6129-6138).
12. Lee, C. Y., Xie, S., Gallagher, P., Zhang, Z., & Tu, Z. (2015, February). Deeply-supervised nets. In Artificial intelligence and statistics (pp. 562-570). PMLR.
13. Maninis, K. K., Pont-Tuset, J., Arbeláez, P., & Van Gool, L. (2017). Convolutional oriented boundaries: From image segmentation to high-level tasks. IEEE transactions on pattern analysis and machine intelligence, 40(4), 819-833.
14. Marmanis, D., Schindler, K., Wegner, J. D., Galliani, S., Datcu, M., & Stilla, U. (2018). Classification with an edge: Improving semantic image segmentation with boundary detection. ISPRS Journal of Photogrammetry and Remote Sensing, 135, 158-172.
15. Rottensteiner, F., Sohn, G., Jung, J., Gerke, M., Baillard, C., Benitez, S., & Breitkopf, U. (2012). The ISPRS benchmark on urban object classification and 3D building reconstruction. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences I-3 (2012), Nr. 1, 1(1), 293-298.
16. Shen, W., Wang, X., Wang, Y., Bai, X., & Zhang, Z. (2015). Deepcontour: A deep convolutional feature learned by positive-sharing loss for contour detection. Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3982-3991).
17. Shorten, C., & Khoshgoftaar, T. M. (2019). A survey on image data augmentation for deep learning. Journal of Big Data, 6(1), 1-48.
18. Xie, S., & Tu, Z. (2015). Holistically-nested edge detection. Proceedings of the IEEE international conference on computer vision (pp. 1395-1403).
19. Yu, Z., Feng, C., Liu, M. Y., & Ramalingam, S. (2017). Casenet: Deep category-aware semantic edge detection. Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5964-5973).