Semantic segmentation of very-high spatial resolution satellite images: A comparative analysis of 3D-CNN and traditional machine learning algorithms for automatic vineyard detection

Year 2024, Volume: 9 Issue: 1, 12 - 24, 15.02.2024

Özlem Akar , Ekrem Saralıoğlu , Oğuz Güngör , Halim Ferit Bayata

https://doi.org/10.26833/ijeg.1252298

Cited By: 2

Abstract

The Erzincan (Cimin) grape, which is an endemic product, plays a significant role in the economy of both the region it is cultivated in and the overall country. Therefore, it is crucial to closely monitor and promote this product. The objective of this study was to analyze the spatial distribution of vineyards by utilizing advanced machine learning and deep learning algorithms to classify high-resolution satellite images. A deep learning model based on a 3D Convolutional Neural Network (CNN) was developed for vineyard classification. The proposed model was compared with traditional machine learning algorithms, specifically Support Vector Machine (SVM), Random Forest (RF), and Rotation Forest (ROTF). The accuracy of the classifications was assessed through error matrices, kappa analysis, and McNemar tests. The best overall classification accuracies and kappa values were achieved by the 3D CNN and RF methods, with scores of 86.47% (0.8308) and 70.53% (0.6279) respectively. Notably, when Gabor texture features were incorporated, the accuracy of the RF method increased to 75.94% (0.6364). Nevertheless, the 3D CNN classifier outperformed all others, yielding the highest classification accuracy with an 11% advantage (86.47%). The statistical analysis using McNemar's test confirmed that the χ2 values for all classification outcomes exceeded 3.84 at the 95% confidence interval, indicating a significant enhancement in classification accuracy provided by the 3D CNN classifier. Additionally, the 3D CNN method demonstrated successful classification performance, as evidenced by the minimum-maximum F1-score (0.79-0.97), specificity (0.95-0.99), and accuracy (0.91-0.99) values.

Keywords

Machine Learning, Deep Learning, Random Forest, Gabor, Image Classification

Supporting Institution

Erzincan Binali Yıldırım University Scientific Research Project

Project Number

636

Thanks

This work was supported by Erzincan Binali Yıldırım University Scientific Research Project [Grant Number: 636].

References

• Weaver, R. J. (1976). Grape growing. John Wiley & Sons.
• Akpınar, E., & Çelikoğlu, Ş. (2016). Karaerik (Cimin) üzümünün Erzincan ekonomisine ve tanıtımına katkıları. Uluslararası Erzincan Sempozyumu, 2, 15-23.
• Bulut, İ. (2006). Genel tarım bilgileri ve tarımın coğrafi esasları (Ziraat Coğrafyası). Gündüz Eğitim ve Yayıncılık, Ankara, 255.
• Republic of Turkey Ministry of Agriculture and Forestry. (2021). 2021-January Agricultural Products Markets Report: GRAPE, https://arastirma.tarimorman.gov.tr/tepge/Menu/27/Tarim-Urunleri-Piyasalari
• Erzincan Directorate of Provincial Agriculture and Forestry (2022). https://erzincan.tarimorman.gov.tr/Menu/66/Tarimsal-Veriler
• Christian, B., & Krishnayya, N. S. R. (2009). Classification of tropical trees growing in a sanctuary using Hyperion (EO-1) and SAM algorithm. Current Science, 96(12), 1601-1607.
• Prins, A. J., & Van Niekerk, A. (2020). Regional Mapping of Vineyards Using Machine Learning and LiDAR Data. International Journal of Applied Geospatial Research (IJAGR), 11(4), 1-22. https://doi.org/10.4018/IJAGR.2020100101
• Darra, N., Psomiadis, E., Kasimati, A., Anastasiou, A., Anastasiou, E., & Fountas, S. (2021). Remote and proximal sensing-derived spectral indices and biophysical variables for spatial variation determination in vineyards. Agronomy, 11(4), 741. https://doi.org/10.3390/agronomy11040741
• Vélez, S., Ariza-Sentís, M., & Valente, J. (2023). Mapping the spatial variability of Botrytis bunch rot risk in vineyards using UAV multispectral imagery. European Journal of Agronomy, 142, 126691. https://doi.org/10.1016/j.eja.2022.126691
• Gungor, O., Boz, Y., Gokalp, E., Comert, C., & Akar, A. (2010). Fusion of low and high resolution satellite images to monitor changes on costal zones. Scientific Research and Essays, 5(7), 654-662.
• Chi, M. V., Thi, L. P., & Si, S. T. (2009, October). Monitoring urban space expansion using Remote sensing data in Ha Long city, Quang Ninh province in Vietnam. In 7th FIG Regional Conference Spatial Data Serving People: Land Governance and the Environment–Building the Capacity Hanoi, Vietnam, 19-22.
• Kaya, Y., & Polat, N. (2023). A linear approach for wheat yield prediction by using different spectral vegetation indices. International Journal of Engineering and Geosciences, 8(1), 52-62. https://doi.org/10.26833/ijeg.1035037
• Akar, A., & Gökalp, E. (2018). Designing a sustainable rangeland information system for Turkey. International Journal of Engineering and Geosciences, 3(3), 87-97. https://doi.org/10.26833/ijeg.412222
• Zhang, W., Xue, X., Sun, Z., Guo, Y. F., Chi, M., & Lu, H. (2007). Efficient feature extraction for image classification. IEEE 11th International Conference on Computer Vision, 1-8. https://doi.org/10.1109/ICCV.2007.4409058
• Huang, Y., Fipps, G., Lacey, R. E., & Thomson, S. J. (2011). Landsat satellite multi-spectral image classification of land cover and land use changes for GIS-based urbanization analysis in irrigation districts of Lower Rio Grande Valley of Texas. Journal of Applied Remote Sensing, 2(1), 27-36.
• Akar, Ö., & Tunç Görmüş, E. (2019). Göktürk-2 ve Hyperion EO-1 uydu görüntülerinden rastgele orman sınıflandırıcısı ve destek vektör makineleri ile arazi kullanım haritalarının üretilmesi. Geomatik, 4(1), 68-81. https://doi.org/10.29128/geomatik.476668
• Ahady, A. B., & Kaplan, G. (2022). Classification comparison of Landsat-8 and Sentinel-2 data in Google Earth Engine, study case of the city of Kabul. International Journal of Engineering and Geosciences, 7(1), 24-31. https://doi.org/10.26833/ijeg.860077
• Sefercik, U. G., Kavzoğlu, T., Çölkesen, I., Nazar, M., Öztürk, M. Y., Adali, S., & Dinç, S. (2023). 3D positioning accuracy and land cover classification performance of multispectral RTK UAVs. International Journal of Engineering and Geosciences, 8(2), 119-128. https://doi.org/10.26833/ijeg.1074791
• Cengiz, A. V. C. I., Budak, M., Yağmur, N., & Balçik, F. (2023). Comparison between random forest and support vector machine algorithms for LULC classification. International Journal of Engineering and Geosciences, 8(1), 1-10. https://doi.org/10.26833/ijeg.987605
• Tirmanoğlu, B., Ismailoğlu, I., Kokal, A. T., & Musaoğlu, N. (2023). Yeni nesil multispektral ve hiperspektral uydu görüntülerinin arazi örtüsü/arazi kullanımı sınıflandırma performanslarının karşılaştırılması: Sentinel-2 ve PRISMA Uydusu. Geomatik, 8(1), 79-90. https://doi.org/10.29128/geomatik.1126685
• Çömert, R., Matci, D. K., & Avdan, U. (2019). Object based burned area mapping with random forest algorithm. International Journal of Engineering and Geosciences, 4(2), 78-87. https://doi.org/10.26833/ijeg.455595
• Sun, Z., Di, L., Fang, H., & Burgess, A. (2020). Deep learning classification for crop types in north dakota. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13, 2200-2213. https://doi.org/10.1109/JSTARS.2020.2990104
• Gao, J. (2009). Digital analysis of remotely sensed imagery. McGraw-Hill Education, New York. ISBN: 9780071604659
• Jay, S., Lawrence, R., Repasky, K., & Keith, C. (2009). Invasive species mapping using low-cost hyperspectral imagery. In ASPRS Annual Conference.
• Ok, A. O., Akar, O., & Gungor, O. (2012). Evaluation of random forest method for agricultural crop classification. European Journal of Remote Sensing, 45(1), 421-432. https://doi.org/10.5721/EuJRS20124535
• Akar, Ö., & Güngör, O. (2015). Integrating multiple texture methods and NDVI to the Random Forest classification algorithm to detect tea and hazelnut plantation areas in northeast Turkey. International Journal of Remote Sensing, 36(2), 442-464. https://doi.org/10.1080/01431161.2014.995276
• Ntouros, K. D., Gitas, I. Z., & Silleos, G. N. (2009, August). Mapping agricultural crops with EO-1 Hyperion data. In 2009 First Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing, 1-4. https://doi.org/10.1109/WHISPERS.2009.5289057
• Kpienbaareh, D., Sun, X., Wang, J., Luginaah, I., Bezner Kerr, R., Lupafya, E., & Dakishoni, L. (2021). Crop type and land cover mapping in northern Malawi using the integration of sentinel-1, sentinel-2, and planetscope satellite data. Remote Sensing, 13(4), 700. https://doi.org/10.3390/rs13040700
• Wang, S., Azzari, G., & Lobell, D. B. (2019). Crop type mapping without field-level labels: Random forest transfer and unsupervised clustering techniques. Remote sensing of environment, 222, 303-317. https://doi.org/10.1016/j.rse.2018.12.026
• Akar, Ö., Saralıoğlu, E., Güngör, O., & Bayata, H. F. (2021). Determination of vineyards with support vector machine and deep learning-based Image classification. Intercontinental Geoinformation Days, 3, 26-29.
• Grinblat, G. L., Uzal, L. C., Larese, M. G., & Granitto, P. M. (2016). Deep learning for plant identification using vein morphological patterns. Computers and Electronics in Agriculture, 127, 418-424. https://doi.org/10.1016/j.compag.2016.07.003
• Ferentinos, K. P. (2018). Deep learning models for plant disease detection and diagnosis. Computers and electronics in agriculture, 145, 311-318. https://doi.org/10.1016/j.compag.2018.01.009
• Chlingaryan, A., Sukkarieh, S., & Whelan, B. (2018). Machine learning approaches for crop yield prediction and nitrogen status estimation in precision agriculture: A review. Computers and Electronics in Agriculture, 151, 61-69. https://doi.org/10.1016/j.compag.2018.05.012
• Abdullahi, H. S., Sheriff, R., & Mahieddine, F. (2017). Convolution neural network in precision agriculture for plant image recognition and classification. Seventh International Conference on Innovative Computing Technology (INTECH), 10, 256-272.
• Zhao, H., Duan, S., Liu, J., Sun, L., & Reymondin, L. (2021). Evaluation of five deep learning models for crop type mapping using sentinel-2 time series images with missing information. Remote Sensing, 13(14), 2790. https://doi.org/10.3390/rs13142790
• Zhong, L., Hu, L., & Zhou, H. (2019). Deep learning based multi-temporal crop classification. Remote Sensing of Environment, 221, 430-443. https://doi.org/10.1016/j.rse.2018.11.032
• TR Erzincan Governorate. (2021). http://www.erzincan.gov.tr/erzincan-uzumu
• Padwick, C., Deskevich, M., Pacifici, F., & Smallwood, S. (2010). WorldView-2 pan-sharpening. In Proceedings of the ASPRS 2010 Annual Conference, San Diego, CA, USA, 2630, 1-14.
• Akar, Ö. (2019). Göktürk-2 ve Worldview-2 Uydu Görüntüleri için Görüntü Keskinleştirme Yöntemlerinin Değerlendirilmesi. Erzincan University Journal of Science and Technology, 12(2), 874-885.
• Li, H., Jing, L., & Tang, Y. (2017). Assessment of pansharpening methods applied to WorldView-2 imagery fusion. Sensors, 17(1), 89. https://doi.org/10.3390/s17010089
• Anshu, S. K., Pande, H., Tiwari, P. S., & Shukla, S. (2017). Evaluation of Fusion Techniques for High Resolution Data-A Worldview-2 Imagery. International Journal of Applied Remote Sensing and GIS, 4, 10-22.
• Fu, L., Ma, J., Chen, Y., Larsson, R., & Zhao, J. (2019). Automatic detection of lung nodules using 3D deep convolutional neural networks. Journal of Shanghai Jiaotong University (Science), 24, 517-523. https://doi.org/10.1007/s12204-019-2084-4
• Zhu, X. X., Tuia, D., Mou, L., Xia, G. S., Zhang, L., Xu, F., & Fraundorfer, F. (2017). Deep learning in remote sensing: A comprehensive review and list of resources. Geoscience and Remote Sensing Magazine, 5(4), 8-36. https://doi.org/10.1109/MGRS.2017.2762307
• Ji, S., Xu, W., Yang, M., & Yu, K. (2012). 3D convolutional neural networks for human action recognition. Transactions on Pattern Analysis and Machine Intelligence, 35(1), 221-231. https://doi.org/10.1109/TPAMI.2012.59
• Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015). Learning spatiotemporal features with 3d convolutional networks. In Proceedings of the IEEE international conference on computer vision, 4489-4497.
• Xu, Z., Guan, K., Casler, N., Peng, B., & Wang, S. (2018). A 3D convolutional neural network method for land cover classification using LiDAR and multi-temporal Landsat imagery. ISPRS journal of photogrammetry and remote sensing, 144, 423-434. https://doi.org/10.1016/j.isprsjprs.2018.08.005
• Ji, S., Zhang, C., Xu, A., Shi, Y., & Duan, Y. (2018). 3D convolutional neural networks for crop classification with multi-temporal remote sensing images. Remote Sensing, 10(1), 75. https://doi.org/10.3390/rs10010075
• Mei, S., Yuan, X., Ji, J., Zhang, Y., Wan, S., & Du, Q. (2017). Hyperspectral image spatial super-resolution via 3D full convolutional neural network. Remote Sensing, 9(11), 1139. https://doi.org/10.3390/rs9111139
• Saralioglu, E., & Gungor, O. (2022). Semantic segmentation of land cover from high resolution multispectral satellite images by spectral-spatial convolutional neural network. Geocarto International, 37(2), 657-677. https://doi.org/10.1080/10106049.2020.1734871
• Li, Y., Zhang, H., & Shen, Q. (2017). Spectral–spatial classification of hyperspectral imagery with 3D convolutional neural network. Remote Sensing, 9(1), 67. https://doi.org/10.3390/rs9010067
• Pérez, F., & Granger, B. E. (2007). IPython: a system for interactive scientific computing. Computing in Science & Engineering, 9(3), 21-29. https://doi.org/10.1109/MCSE.2007.53
• Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.
• Watts, J. D., & Lawrence, R. L. (2008). Merging random forest classification with an object-oriented approach for analysis of agricultural lands. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 37(B7), 579-582
• Waske, B., Heinzel, V., Braun, M., & Menz, G. (2007). Random forests for classifying multi-temporal sar data. Envisat Symposium, 2007, 23-27.
• Gislason, P. O., Benediktsson, J. A., & Sveinsson, J. R. (2004). Random forest classification of multisource remote sensing and geographic data. In IGARSS 2004. International Geoscience and Remote Sensing Symposium, 2, 1049-1052. https://doi.org/10.1109/IGARSS.2004.1368591
• Pal, M. (2003, July). Random forests for land cover classification. In IGARSS 2003. 2003 IEEE International Geoscience and Remote Sensing Symposium. Proceedings (IEEE Cat. No. 03CH37477) 6, 3510-3512. https://doi.org/10.1109/IGARSS.2003.1294837
• Rodriguez, J. J., Kuncheva, L. I., & Alonso, C. J. (2006). Rotation forest: A new classifier ensemble method. IEEE transactions on pattern analysis and machine intelligence, 28(10), 1619-1630. https://doi.org/10.1109/TPAMI.2006.211
• Xia, J., Du, P., He, X., & Chanussot, J. (2013). Hyperspectral remote sensing image classification based on rotation forest. IEEE Geoscience and Remote Sensing Letters, 11(1), 239-243. https://doi.org/10.1109/LGRS.2013.2254108
• Liu, K. H., & Huang, D. S. (2008). Cancer classification using rotation forest. Computers in biology and medicine, 38(5), 601-610. https://doi.org/10.1016/j.compbiomed.2008.02.007
• Vapnik, V. (1999). The nature of statistical learning theory. Springer science & business media.
• Özkan, Y. (2008). Veri Madenciliği Yöntemleri, Papatya Yayıncılık, İstanbul.
• Mather, P., & Tso, B. (2016). Classification methods for remotely sensed data. CRC press.
• Stephens, D., & Diesing, M. (2014). A comparison of supervised classification methods for the prediction of substrate type using multibeam acoustic and legacy grain-size data. PloS one, 9(4), e93950. https://doi.org/10.1371/journal.pone.0093950
• Çölkesen, İ., & Yomralıoğlu, T. (2014). Arazi örtüsü ve kullanımının haritalanmasında WorldView-2 uydu görüntüsü ve yardımcı verilerin kullanımı. Harita Dergisi, 152(2), 12-24.
• Thanh Noi, P., & Kappas, M. (2018). Comparison of random forest, k-nearest neighbor, and support vector machine classifiers for land cover classification using Sentinel-2 imagery. Sensors, 18(1), 18. https://doi.org/10.3390/s18010018
• Kavzoglu, T., & Colkesen, I. (2009). A kernel functions analysis for support vector machines for land cover classification. International Journal of Applied Earth Observation and Geoinformation, 11(5), 352-359. https://doi.org/10.1016/j.jag.2009.06.002
• Congalton, R. G., & Green, K. (2019). Assessing the accuracy of remotely sensed data: principles and practices. CRC Press.
• Congalton, R. G., & Green, K. (2019). Assessing the accuracy of remotely sensed data: principles and practices. CRC Press.
• Pontius Jr, R. G., & Millones, M. (2011). Death to Kappa: birth of quantity disagreement and allocation disagreement for accuracy assessment. International Journal of Remote Sensing, 32(15), 4407-4429. https://doi.org/10.1080/01431161.2011.552923
• Akar, A. (2022). Improving the accuracy of random forest‐based land‐use classification using fused images and digital surface models produced via different interpolation methods. Concurrency and Computation: Practice and Experience, 34(6), e6787. https://doi.org/10.1002/cpe.6787
• Foody, G. M. (2004). Thematic map comparison. Photogrammetric Engineering & Remote Sensing, 70(5), 627-633. https://doi.org/10.14358/PERS.70.5.627
• Amini, S., Saber, M., Rabiei-Dastjerdi, H., & Homayouni, S. (2022). Urban land use and land cover change analysis using random forest classification of landsat time series. Remote Sensing, 14(11), 2654. https://doi.org/10.3390/rs14112654