Effect of dilation rate on Nested U-Net model performance in remote sensing

Abstract

High spatial resolution remote sensing images contain substantial detailed multi-scale objects. Convolutional neural networks (CNNs) are not efficient enough for detecting these objects of varying sizes. Among the multitude of CNN approaches, the Nested U-Net (UNet++) model shows great potential to capture more complex details by progressively enriching highresolution feature maps. However, there is more room for improving the Nested U-Net architecture by increasing its ability to detect multi-scale objects. The nested blocks used in this architecture rely on standard convolutional layers, which are of limited efficacy in capturing pixel information. Thus, larger receptive fields are required to extract multi-scale feature information. Although many approaches are available for increasing the receptive fields in the Nested U-Net model, these methods usually make the computational efforts very heavy. Therefore, this study uses dilated convolutions in the Nested UNet architecture to broaden the receptive field without augmenting computational demand. To this extent, the paper performs experiments with different dilation rates in the convolution blocks to understand the benefits of employing dilated convolutions in Nested U-Net architecture. Experiments using two remote sensing image sets show that the Nested U-Net model with dilated convolutions performs well for images containing both visible and multispectral wavelengths. While being able to provide performance improvement, experimental results also demonstrate that only the optimal dilation rate scheme in the proposed approach is beneficial.

Keywords

• Semantic segmentation
• remote sensing
• dilated convolution
• multispectral images
• Nested U-Net model

References

1. Piao, S., Liu, J., Accuracy improvement of UNet based on dilated convolution, J. Phys. Conf. Ser., 1345 (5) (2019), 052066, https://doi.org/10.1088/1742-6596/1345/5/052066.
2. Ma, B., Chang, C. Y., Semantic segmentation of high-resolution remote sensing images using multiscale skip connection network, IEEE Sens. J., 22 (4) (2021), 3745-3755, https://doi.org/10.1109/JSEN.2021.3139629.
3. Ding, L., Zhang, J., Bruzzone, L., Semantic segmentation of large-size VHR remote sensing images using a two-stage multiscale training architecture, IEEE Trans. Geosci. Remote Sens., 58 (8) (2020), 5367-5376, https://doi.org/10.1109/TGRS.2020.2964675.
4. Li, X., Lei, L., Kuang, G., Multilevel adaptive-scale context aggregating network for semantic segmentation in high-resolution remote sensing images, IEEE Geosci. Remote Sens. Lett., 19 (2021), 1-5, https://doi.org/10.1109/LGRS.2021.3091284.
5. Du, S., Du, S., Liu, B., Zhang, X., Mapping large-scale and fine-grained urban functional zones from VHR images using a multi-scale semantic segmentation network and object based approach, Remote Sens. Environ., 261 (2021), 112480, https://doi.org/10.1016/j.rse.2021.112480.
6. Zhou, Z., Rahman Siddiquee, M. M., Tajbakhsh, N., Liang, J., Unet++: A nested U-Net architecture for medical image segmentation, Proc. Int. Workshop Deep Learn. Med. Image Anal., (2018), 3-11, https://doi.org/10.1007/978-3-030-00889-5−1.
7. Zhong, Y., Shi, Z., Zhang, Y., Zhang, Y., Li, H., CSAN-UNet: Channel spatial attention nested UNet for infrared small target detection, Remote Sens., 16 (11) (2024), 1894, https://doi.org/10.3390/rs16111894.
8. Jiang, J., Liu, L., Cui, Y., Zhao, Y., A Nested UNet based on multi-scale feature extraction for mixed Gaussian-impulse removal, Appl. Sci., 13 (17) (2023), 9520, https://doi.org/10.3390/app13179520.

9. Kushnure, D. T., Tyagi, S., Talbar, S. N., LiM-Net: Lightweight multi-level multiscale network with deep residual learning for automatic liver segmentation in CT images, Biomed. Signal Process. Control, 80 (2023), 104305, https://doi.org/10.1016/j.bspc.2022.104305.
10. Yang, B., Liu, Z., Duan, G., Tan, J., Residual shape adaptive dense-nested Unet: Redesign the long lateral skip connections for metal surface tiny defect inspection, Pattern Recognit., 147 (2024), 110073, https://doi.org/10.1016/j.patcog.2023.110073.
11. Wu, Z., Tang, Y., Hong, B., Liang, B., Liu, Y., Enhanced precision in dam crack width measurement: Leveraging advanced lightweight network identification for pixel-level accuracy, Int. J. Intell. Syst., 2023 (2023), 9940881, https://doi.org/10.1155/2023/9940881.
12. Liu, Y., Liu, J., Ning, X., Li, J., MS-CNN: multiscale recognition of building rooftops from high spatial resolution remote sensing imagery, Int. J. Remote Sens., 43 (1) (2022), 270-298, https://doi.org/10.1080/01431161.2021.2018146.
13. Zhang, T., Yang, Z., Xu, Z., Li, J., Wheat yellow rust severity detection by efficient DF-UNet and UAV multispectral imagery, IEEE Sens. J., 22 (9) (2022), 9057-9068, https://doi.org/10.1109/JSEN.2022.3156097.
14. Zhou, G., Yu, J., Zhou, S., LSCB: a lightweight feature extraction block for SAR automatic target recognition and detection, Int. J. Remote Sens., 44 (8) (2023), 2548-2572, https://doi.org/10.1080/01431161.2023.2203342.
15. Ren, K., Chen, X., Wang, Z., Liang, X., Chen, Z., Miao, X., HAM-transformer: A hybrid adaptive multi-scaled transformer net for remote sensing in complex scenes, Remote Sens., 15 (19) (2023), 4817, https://doi.org/10.3390/rs15194817.
16. Chen, L. C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A. L., Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs, IEEE Trans. Pattern Anal. Mach. Intell., 40(4) (2017), 834-848, https://doi.org/10.1109/TPAMI.2017.2699184.
17. He, H., Zhang, C., Chen, J., Geng, R., Chen, L., Liang, Y., Xu, Y., A hybrid-attention nested UNet for nuclear segmentation in histopathological images, Front. Mol. Biosci., 8 (2021), 614174, https://doi.org/10.3389/fmolb.2021.614174.
18. Agnes, S. A., Anitha, J., Solomon, A. A., Two-stage lung nodule detection framework using enhanced UNet and convolutional LSTM networks in CT images, Comput. Biol. Med., 149 (2022), 106059, https://doi.org/10.1016/j.compbiomed.2022.106059.
19. Yang, K., Yi, J., Chen, A., Liu, J., Chen, W., ConDinet++: Full-scale fusion network based on conditional dilated convolution to extract roads from remote sensing images, IEEE Geosci. Remote Sens. Lett., 19 (2021), 1-5, https://doi.org/10.1109/LGRS.2021.3093101.
20. Safarov, S., Whangbo, T. K., A-DenseUNet: Adaptive densely connected UNet for polyp segmentation in colonoscopy images with atrous convolution, Sensors, 21 (4) (2021), 1441, https://doi.org/10.3390/s21041441.
21. Zhao, H., Zhang, H., Zheng, X., A multiscale attention-guided UNet++ with edge constraint for building extraction from high spatial resolution imagery, Appl. Sci., 12 (12) (2022), 5960, https://doi.org/10.3390/app12125960.
22. Ulku, I., ResLMFFNet: a real-time semantic segmentation network for precision agriculture, J. Real-Time Image Process., 21 (4) (2024), 101, https://doi.org/10.1007/s11554-024-01474-0.
23. Ronneberger, O., Fischer, P., Brox, T., U-net: Convolutional networks for biomedical image segmentation, Proc. Int. Conf. Med. Image Comput. Comput.-Assist. Interv., (2015), 234-241, https://doi.org/10.1007/978-3-319-24574-4−28.
24. Alom, M. Z., Yakopcic, C., Hasan, M., Taha, T. M., Asari, V. K., Recurrent residual U-Net for medical image segmentation, J. Med. Imaging, 6 (1) (2019), 014006, https://doi.org/10.1117/1.JMI.6.1.014006.
25. Delibasoglu, I., Cetin, M., Improved U-Nets with inception blocks for building detection, J. Appl. Remote Sens., 14 (4) (2020), 044512, https://doi.org/10.1117/1.JRS.14.044512.
26. Khanh, T. L. B., Dao, D. P., Ho, N. H., Yang, H. J., Baek, E. T., Lee, G., Yoo, S. B., Enhancing U-Net with spatial-channel attention gate for abnormal tissue segmentation in medical imaging, Appl. Sci., 10 (17) (2020), 5729, https://doi.org/10.3390/app10175729.