Investigating the performance of super-resolved remote sensing images on coastline segmentation with deep learning based methods

Abstract

The use of satellite imagery in critical areas, such as environmental monitoring and natural disaster management, is becoming increasingly important. Applications like monitoring coastal areas, detecting coastal erosion, and tracking land use changes demand high accuracy and detailed analysis. Traditional methods for coastline segmentation are often limited by the low resolution (LR) and high complexity of satellite imagery. To address this challenge, Super Resolution (SR) algorithms are employed to enhance the resolution of satellite images, which is particularly beneficial when examining areas with intricate structures, such as coastlines. In this context, the integration of SR and segmentation techniques presents an innovative approach to achieving greater accuracy and efficiency in satellite image analysis. In this study, the resolution of satellite images was enhanced using the Super Resolution Generative Adversarial Networks (SRGAN) model. Thanks to the flexible architecture of the SRGAN model, it was successfully adapted to work with satellite images, yielding satisfactory results. Coastal segmentation was performed using low-resolution, super-resolved, and high-resolution Gokturk-1 (GT-1) satellite images, employing U-net, LinkNet, and DeepLabV3+ segmentation models for comparison. The results indicated that increment in image resolution significantly affects segmentation success. Additionally, better performance in coastline segmentation was achieved with U-net and LinkNet models. Although the DeepLabV3+ model is effective for segmentation, it tends to capture less detail compared to the other two models. Overall, the combination of SRGAN and the LinkNet segmentation model produced results that were closest to reality

Keywords

• Remote Sensing
• Super Resolution
• Image Processing
• Coastline Segmentation
• Deep Learning

Supporting Institution

TUBITAK

Project Number

121Y366

References

1. Saleem, A., Mahmood, S. (2023). Spatio-temporal assessment of urban growth using multi-stage satellite imageries in Faisalabad, Pakistan. Advanced Remote Sensing, 3(1), 10–18.
2. Zadbagher, E., Marangoz, A.M, Becek, K. (2023). Characterizing and estimating forest structure using active remote sensing: An overview. Advanced Remote Sensing, 3(1), 38–46.
3. Efe, E., Algancı, U. (2023). Çok zamanlı Sentinel 2 uydu görüntüleri ve makine öğrenmesi tabanlı algoritmalar ile arazi örtüsü değişiminin belirlenmesi. Geomatik, 8(1), 27-34.
4. Yiğit, A.Y., Şenol, H.İ., Kaya, Y. (2022). Çok zamanlı multispektral uydu verilerinin Marmara Gölü kıyı değişimi analizinde kullanılması. Geomatik. 2022, 7(3), 253-260.
5. Bakırman, T., Sertel, E. (2023). A benchmark dataset for deep learning-based airplane detection: HRPlanes. International Journal of Engineering and Geosciences, 8(3), 212-223.
6. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., … & Bengio, Y. (2014). Generative Adversarial Networks (Version 1). arXiv, https://doi.org/10.48550/ARXIV.1406.2661
7. Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., … & Shi, W. (2016). Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network (Version 5). arXiv, https://doi.org/10.48550/ARXIV.1609.04802
8. Salgueiro Romero, L., Marcello, J., Vilaplana, V. (2020). Super-Resolution of Sentinel-2 Imagery Using Generative Adversarial Networks. Remote Sensing, 12(15), 2424.

9. Xiong, Y., Guo, S., Chen, J., Deng, X., Sun, L., Zheng, X., Xu, W. (2020). Improved SRGAN for Remote Sensing Image Super-Resolution Across Locations and Sensors. Remote Sensing, 12(8), 1263.
10. 10.Ronneberger, O., Fischer, P., Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, 234-241, https://doi.org/10.1007/978-3-319-24574-4_28
11. 11.Chaurasia, A., Culurciello, E. (2017). LinkNet: Exploiting encoder representations for efficient semantic segmentation. IEEE Visual Communications and Image Processing (VCIP), St. Petersburg, FL, USA, 1- 4. https://doi.org/10.1109/vcip.2017.8305148
12. Chen, L-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A. L. (2014). Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs (Version 4). arXiv, https://doi.org/10.48550/ARXIV.1412.7062
13. Chen, L-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H. (2018). Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation (Version 3). arXiv, https://doi.org/10.48550/ARXIV.1802.02611
14. Alsabhan, W., Alotaiby, T. (2022). Automatic Building Extraction on Satellite Images Using Unet and ResNet50. Computational Intelligence and Neuroscience, (1), 5008854.
15. 15.Sariturk, B., Seker, D.Z., Ozturk, O., Bayram, B. (2022). Performance evaluation of shallow and deep CNN architectures on building segmentation from high-resolution images. Earth Science Informatics, 15, 1801–1823.
16. Zhang, Z., Liu, Q., Wang, Y. (2018). Road Extraction by Deep Residual U-Net. IEEE Geoscience and Remote Sensing Letters, 15(5), 749-753.
17. Han, J., Wang, Z., Wang, Y., Hou, W. (2022). Building extraction algorithm from remote sensing images based on improved DeepLabv3+ network. Journal of Physics: Conference Series, 2303, 012010.
18. Kaya, Y., Şenol, H. İ., Yiğit, A. Y., & Yakar, M. (2023). Car detection from very high-resolution UAV images using deep learning algorithms. Photogrammetric Engineering & Remote Sensing, 89(2), 117-123.
19. Şenol, H. İ., Kaya, Y., Yiğit, A. Y., & Yakar, M. (2024). Extraction and geospatial analysis of the Hersek Lagoon shoreline with Sentinel-2 satellite data. Survey Review, 56(397), 367-382.
20. Yang, T., Jiang, S., Hong, Z., Zhang, Y., Han, Y., Zhou, R., … & Kuc, T. (2020). Sea-Land Segmentation Using Deep Learning Techniques for Landsat-8 OLI Imagery. Marine Geodesy, 43(2), 105-133.
21. Panuntun, I. A., Chen, Y-N., Jamaluddin, I., Tran, T. L. C. (2024). Evaluation of Deep Learning Semantic Segmentation for Land Cover Mapping on Multispectral, Hyperspectral and High Spatial Aerial Imagery. arXiv ,https://doi.org/10.48550/ARXIV.2406.14220 Ünel, F. B., Kuşak, L., Çelik, M., Alptekin, A., & Yakar, M. (2020). Kıyı çizgisinin belirlenerek mülkiyet durumunun incelenmesi. Türkiye Arazi Yönetimi Dergisi, 2(1), 33-40.
22. Badrinarayanan, V., Kendall, A., Cipolla, R. (2017). SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), 2481 - 2495.
23. Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J. (2016). Pyramid Scene Parsing Network (Version 2). arXiv, https://doi.org/10.48550/ARXIV.1612.01105
24. Jégou, S., Drozdzal, M., Vazquez, D., Romero, A., Bengio, Y. (2016). The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation (Version 3), arXiv, https://doi.org/10.48550/ARXIV.1611.09326
25. Lin, G., Milan, A., Shen, C., Reid, I. (2016). RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation (Version 3). arXiv, https://doi.org/10.48550/ARXIV.1611.06612
26. Lin, T-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S. (2016). Feature Pyramid Networks for Object Detection (Version 2). arXiv, https://doi.org/10.48550/ARXIV.1612.03144
27. He, K., Zhang, X., Ren, S., Sun, J. (2016). Deep Residual Learning for Image Recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 770-778
28. He, K., Zhang, X., Ren, S., Sun, J. (2015). Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification (Version 1). arXiv, https://doi.org/10.48550/ARXIV.1502.01852
29. Simonyan, K., Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition (Version 6). arXiv, https://doi.org/10.48550/ARXIV.1409.1556
30. Chen, L-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A. L. (2018). DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(10), 834 - 848.
31. Akar, Ö., Saralıoğlu, E., Güngör, O., Bayata, H. F. (2024). Semantic segmentation of very-high spatial resolution satellite images: A comparative analysis of 3D-CNN and traditional machine learning algorithms for automatic vineyard detection. International Journal of Engineering and Geosciences, 9(1), 12-24.
32. Mogaraju, J. K. (2024). Machine learning assisted prediction of land surface temperature (LST) based on major air pollutants over the Annamayya District of India. International Journal of Engineering and Geosciences, 9(2), 233-246.
33. Çetin, Ş. B. (2023). Real-ESRGAN: A deep learning approach for general image restoration and its application to aerial images. Advanced Remote Sensing, 3(2), 90–99.