Evaluating the performance of deep learning-based segmentation algorithms for land use land cover mapping in a heterogenous vegetative environment

Abstract

Land Use and Land Cover (LULC) maps are important geospatial information sources for different applications such as city planning, vegetation analysis, natural resource management, natural disaster analysis, and land change determination. In recent decades, the demand for more frequent creation and updating of LULC maps has grown significantly, driven by the rapid and continuous changes occurring on the Earth surface. Moreover, the increased availability of satellite images and processing power led to improvements in LULC mapping. However, traditional classification approaches are prone to several errors emerging from high human interaction and algorithm limitations. In addition, they generally suffer from processing time performance due to software limitations and generally singular hardware configurations, especially when very high resolution (VHR) images are of concern. In this study, we aim to produce LULC maps of the Aksu region of Bursa city Türkiye, using Worldview-3 VHR images and deep learning (DL) methods. We applied two widely used DL architectures, Unet++ and DeepLabv3+, and evaluated results using overall accuracy, average accuracy, error matrix, weighted accuracy, recall, precision, F-1 score, IoU score, and kappa metrics. Among several experimental setups, we achieved the best accuracy with the Unet++ architecture, using the ResNeXt-50 backbone and Adam optimizer, resulting in an approximately 84% IoU score and 91% F-1 score. This study demonstrates that utilizing appropriate datasets and CNN-based segmentation models for LULC mapping ensures efficient, accurate, and high-performance results, significantly contributing to long-term monitoring and sustainable development goals. .

Keywords

• Land use - Land cover
• Unet++
• Deeplabv3
• Worldview 3

References

1. Treitz, P., & Rogan, J. (2004). Remote sensing for mapping and monitoring land-cover and land-use change-an introduction. Progress in Planning, 61, 269-279.https://doi.org/10.1016/S0305-9006(03)00064-3
2. Mora, B., Tsendbazar, N., Herold, M., & Arino, O. (2014). Global Land Cover Mapping: Current Status and Future Trends. In: Manakos, I., Braun, M. (eds) Land Use and Land Cover Mapping in Europe. Remote Sensing and Digital Image Processing, vol 18. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-7969-3_2
3. Rogan, J., & Chen, D. (2004). Remote sensing technology for mapping and monitoring land-cover and land-use change. Progress in Planning, 61, 301-325. https://doi.org/10.1016/S0305-9006(03)00066-7.
4. Saleem, A., & Mahmood, S. (2023). Spatio-temporal assessment of urban growth using multi-stage satellite imageries in Faisalabad, Pakistan. Advanced Remote Sensing, 3(1), 10–18.
5. Zadbagher, E., Marangoz, A. M., & Becek, K. (2023). Characterizing and estimating forest structure using active remote sensing: An overview. Advanced Remote Sensing, 3(1), 38–46.
6. Efe, E., & Alganci, U. (2023). Çok zamanlı Sentinel 2 uydu görüntüleri ve makine öğrenmesi tabanlı algoritmalar ile arazi örtüsü değişiminin belirlenmesi. Geomatik, 8(1), 27-34. https://doi.org/10.29128/geomatik.1092838
7. Pala, İ., & Alganci, U. (2025). Investigating the performance of super-resolved remote sensing images on coastline segmentation with deep learning based methods. International Journal of Engineering and Geosciences, 10(1), 93-106. https://doi.org/10.26833/ijeg.1522143
8. Carter, S., & Herold, M. (2019). Specifications of land cover datasets for SDG indicator monitoring. Retrieved from: https://unstats.un.org/sdgs/iaeg-sdgs/tier-classification-for-globalindicators/10th-meeting---september-2019/10.2_--UNEP-WCMC_Sarah_Carter.pdf.

9. Nguyen, H. Q., Doan, T. D., Tomppo, E., & McRoberts, R. E. (2020). Land Use/Land Cover Mapping Using Multitemporal Sentinel-2 Imagery and Four Classification Methods—A Case Study from Dak Nong, Vietnam. Remote Sensing, 12(9), 1367. https://doi.org/10.3390/rs12091367.
10. Helber, P., Bischke, B., Dengel, A., & Borth, D. (2019). EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2217. https://doi.org/10.1109/JSTARS.2019.2918242.
11. Shi, Y., Qi, Z., Liu, X., Niu, N., Zhang, H. (2019). Urban land use and land cover classification using multisource remote sensing images and social media data. Remote Sensing, 11, 2719. https://doi.org/10.3390/rs11222719.
12. Uba, N. K. (2019). Land Use and Land Cover Classification Using Deep Learning Techniques. arXiv:1905.00510.https://doi.org/10.48550/arXiv.1905.00510.
13. Sertel, E., Ekim, B., Osgouei, P. E., & Kabadayi, M. E. (2022). Land Use and Land Cover Mapping Using Deep Learning Based Segmentation Approaches and VHR Worldview-3 Images. Remote Sensing, 14(18), 4558. https://doi.org/10.3390/rs14184558.
14. Henry, C. D., Storie, C. D., Palaniappan, M., Alhassan, V., Swamy, M., Aleshinloye, D. K., Curtis, A., & Kim, D. (2019). Automated LULC map production using deep neural networks. International Journal of Remote Sensing, 40(11), 4416–4440. https://doi.org/10.1080/01431161.2018.1563840
15. Long, J., Shelhamer, E., Darrell, T., (2015). Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition. 07-12-June-2015, 3431–3440. https://doi.org/10.1109/CVPR.2015.7298965.
16. Bakırman, T., & Sertel, E. (2023). A benchmark dataset for deep learning-based airplane detection: HRPlanes. International Journal of Engineering and Geosciences, 8(3), 212-223. https://doi.org/10.26833/ijeg.1107890.
17. Akar, Ö., Saralıoğlu, E., Güngör, O., Bayata, H. F. (2024). Semantic segmentation of very-high spatial resolution satellite images: A comparative analysis of 3D-CNN and traditional machine learning algorithms for automatic vineyard detection. International Journal of Engineering and Geosciences, 9(1), 12-24. https://doi.org/10.26833/ijeg.1252298.
18. Nasiri, V., Deljouei, A., Moradi, F., Sadeghi, S. M. M., & Borz, S. A. (2022). Land Use and Land Cover Mapping Using Sentinel-2, Landsat-8 Satellite Images, and Google Earth Engine: A Comparison of Two Composition Methods. Remote Sensing, 14(9), 1977. https://doi.org/10.3390/rs14091977.
19. Sertel, E., Kabadayı, M.E., Sengul, G.S., & Tumer, I.N. (2024). HexaLCSeg: A historical benchmark dataset from Hexagon satellite images for land cover segmentation [Software and Data Sets]. IEEE Geoscience and Remote Sensing Magazine, 12, 197-206. https://doi.org/10.1109/MGRS.2024.3394248.
20. Zhang, Y. (2002). A new automatic approach for effectively fusing Landsat 7 as well as IKONOS images. In IEEE International Geoscience and Remote Sensing Symposium, 4, 2429-2431. http://dx.doi.org/10.1109/IGARSS.2002.1026567.
21. Wang, P., & Sertel, E. (2021). Channel–spatial attention-based pan-sharpening of very high-resolution satellite images. Knowledge-Based Systems, 229, 107324. https://doi.org/10.1016/j.knosys.2021.107324.
22. Sklearn Package. (2023) Sklearn. Last Access: 30 December 2023, https://scikit-learn.org/stable/about.html#citing-scikit-learn.
23. Ekim, B., Sertel, E., Kabadayı, M.E. (2021). Automatic Road extraction from historical maps using deep learning techniques: A regional case study of Turkey in a German World War II Map. ISPRS International Journal of Geo-Information, 10(8), 492. https://doi.org/10.3390/ijgi10080492.
24. Stone, M. (1974). Cross-Validatory Choice and Assessment of Statistical Predictions. Journal of the Royal Statistical Society, Series B (Methodological), 36(2), 111–147.
25. Hastie, T., Tibshirani, R., & Friedman, J. H. (2009). The elements of statistical learning. In Springer series in statistics.https://doi.org/10.1007/978-0-387-84858-7.
26. Joseph, V. R., & Vakayil, A. (2021). SPLIT: an optimal method for data splitting. Technometrics, 64(2), 166–176. https://doi.org/10.1080/00401706.2021.1921037.
27. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J. T., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Liu, F., & Chintala, S. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. arXiv:1912.01703 https://doi.org/10.48550/arxiv.1912.01703.
28. Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., & Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, 248-255. https://doi.org/10.1109/CVPR.2009.5206848.
29. Mandt, S., Hoffman, M. D., & Blei, D. M. (2017). Stochastic gradient descent as approximate bayesian inference. Journal of Machine Learning Research, 18(134), 1-35. https://doi.org/10.48550/arXiv.1704.04289.
30. Smith, S. L., Duckworth, D., Rezchikov, S., Le, Q. V., & Sohl-Dickstein, J. (2018). Stochastic natural gradient descent draws posterior samples in function space. arXiv:1806.09597 https://doi.org/10.48550/arXiv.1806.09597.
31. Kaggle. (2023). Kaggle: your machine learning and data science community. https://www.kaggle.com/.
32. Castelluccio, M., Poggi, G., Sansone, C., & Verdoliva, L. (2015). Land use classification in remote sensing images by convolutional neural networks. arXiv.1508.00092. https://doi.org/10.48550/arXiv.1508.00092.
33. Luus, F. P., Salmon, B. P., van den Bergh, F., & Maharaj, B. (2015). Multiview deep learning for land use classification. IEEE Geoscience and Remote Sensing Letters, 12(12), 2448-2452. https://doi.org/10.1109/LGRS.2015.2483680.
34. Marmanis, D., Datcu, M., Esch, T., & Stilla, U. (2016). Deep learning earth observation classification using ImageNet pretrained networks. IEEE Geoscience and Remote Sensing Letters, 13(1), 105-109. https://doi.org/10.1109/LGRS.2015.2499239.
35. Ünel, F. B., Kuşak, L., Yakar, M., & Doğan, H. (2023). Coğrafi bilgi sistemleri ve analitik hiyerarşi prosesi kullanarak Mersin ilinde otomatik meteoroloji gözlem istasyonu yer seçimi. Geomatik, 8(2), 107-123.
36. Alhassan, V., Henry, C. D., Ramanna, S., & Storie, C. D. (2019). A deep learning framework for land-use/land-cover mapping and analysis using multispectral satellite imagery. Neural Computing and Applications, 32(12), 8529–8544. https://doi.org/10.1007/s00521-019-04349-9.
37. Zhang, P., Ke, Y., Zhang, Z., Wang, M., Li, P., & Zhang, S. (2018). Urban Land Use and Land Cover Classification Using Novel Deep Learning Models Based on High Spatial Resolution Satellite Imagery. Sensors, 18(11), 3717. https://doi.org/10.3390/s18113717.
38. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2017). DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4), 834-848. https://doi.org/10.1109/TPAMI.2017.2699184.
39. Chen, L. C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), 801-818. https://doi.org/10.48550/arXiv.1802.02611.
40. Zhou, Z., Siddiquee, M. M. R., Tajbakhsh, N., & Liang, J. (2019). Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE transactions on medical imaging, 39(6), 1856-1867. https://doi.org/10.1109/TMI.2019.2959609.
41. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770-778. https://doi.org/10.1109/CVPR.2016.90.
42. He, K., & Sun, J. (2015). Convolutional neural networks at constrained time cost. In Proceedings of the IEEE conference on computer vision and pattern recognition, 5353-5360. https://doi.org/10.48550/arXiv.1412.1710.
43. Srivastava, R. K., Greff, K., & Schmidhuber, J. (2015). Highway networks. https://doi.org/10.48550/arXiv.1505.00387.
44. Stoyanov, D., Taylor, Z., Carneiro, G., Syeda-Mahmood, T., Martel, A., Maier-Hein, L., ... & Madabhushi, A. (Eds.). (2018). Deep learning in medical image analysis and multimodal learning for clinical decision support. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: : 4th International Workshop DLMIA 2018 and 8th International Workshop ML-CDS 2018 Held in Conjunction With MICCAI 2018 Granada Spain September 20 2018 Proceedings, Cham, Switzerland:Springer, vol. 11045, 2018.
45. Lin, T. Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017). Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, 2980-2988. https://doi.org/10.1109/ICCV.2017.324.
46. Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., Jorge Cardoso, M. (2017). Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations. In: Cardoso, M., et al. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. DLMIA ML-CDS 2017 2017. Lecture Notes in Computer Science (), vol 10553. Springer, Cham. http://doi.org/10.1007/978-3-319-67558-9_28.
47. Mulyanto, M., Faisal, M., Prakosa, S. W., & Leu, J. S. (2021). Effectiveness of focal loss for minority classification in network intrusion detection systems. Symmetry, 13(1), 4. http://doi.org/10.3390/sym13010004.
48. Duchi, J., Hazan, E., & Singer, Y. (2011). Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12(7), 2121-2159.
49. Zeiler, M. D. (2012). Adadelta: an adaptive learning rate method. arXiv:1212.5701. https://doi.org/10.48550/arXiv.1212.5701
50. Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv:1412.6980 https://doi.org/10.48550/arXiv.1412.6980.
51. Teferi, E., Bewket, W., Uhlenbrook, S., & Wenninger, J. (2013). Understanding recent land use and land cover dynamics in the source region of the Upper Blue Nile, Ethiopia: Spatially explicit statistical modeling of systematic transitions. Agriculture, Ecosystems and Environment, 165, 98-117. https://doi.org/10.1016/j.agee.2012.11.007.
52. Powers, D.M. (2020). Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. arXiv:2010.16061. https://doi.org/10.48550/arXiv.2010.16061.
53. Congalton, R. G., & Green, K. (2008). Assessing the Accuracy of Remotely Sensed Data. In CRC Press eBooks. https://doi.org/10.1201/9781420055139
54. Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., & Savarese, S. (2019). Generalized intersection over union: A metric and a loss for bounding box regression. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 658-666. https://doi.org/10.48550/arXiv.1902.09630.
55. Längkvist, M., Kiselev, A., Alirezaie, M., Loutfi, A. (2016). Classification and segmentation of satellite orthoimagery using convolutional neural networks. Remote Sensing, 8, 329. http://doi.org/10.3390/rs8040329
56. Cheng, G., Xie, X., Han, J., Guo, L., Xia, G.-S. (2020). Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and Opportunities. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13, 3735–3756. http://doi.org/10.1109/JSTARS.2020.3005403
57. Yuan, X., Shi, J., Gu, L. (2021). A Review of Deep Learning Methods for Semantic Segmentation of Remote Sensing Imagery. Expert Systems with Applications, 169, 114417. http://doi.org/10.1016/j.eswa.2020.114417
58. Boonpook, W., Tan, Y., Nardkulpat, A., Torsri, K., Torteeka, P., Kamsing, P., ... & Jainaen, M. (2023). Deep learning semantic segmentation for land use and land cover types using Landsat 8 imagery. ISPRS International Journal of Geo-Information, 12(1), 14. https://doi.org/10.3390/ijgi12010014
59. Du, S., Du, S., Liu, B., Zhang, X. (2020). Incorporating DeepLabv3+ and object-based image analysis for semantic segmentation of very high resolution remote sensing images. International Journal of Digital Earth, 14, 1–22. http://doi.org/10.1080/17538947.2020.1831087
60. Martins, V. S., Kaleita, A. L., Gelder, B. K., da Silveira, H. L., & Abe, C. A. (2020). Exploring multiscale object-based convolutional neural network (multi-OCNN) for remote sensing image classification at high spatial resolution. ISPRS Journal of Photogrammetry and Remote Sensing, 168, 56-73.
61. Bengana, N., Heikkila, J. (2021). Improving land cover segmentation across satellites using domain adaptation. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14, 1399–1410. http://doi.org/10.1109/JSTARS.2020.3042887
62. Kemker, R., Salvaggio, C., Kanan, C. (2018). Algorithms for semantic segmentation of multispectral remote sensing imagery using deep learning. ISPRS Journal of Photogrammetry and Remote Sensing, 145, 60–77. http://doi.org/10.1016/j.isprsjprs.2018.04.014