Classification of Hyperspectral Images Using 3D Convolutional Neural Network

Abstract

Hyperspectral image classification is commonly used for the analysis of remotely sensed images.A hyperspectral image contains rich spectral and spatial information of ground objects that has great potential in applications.The use of spectral spatial information significantly improves the performance of hyperspectral image classification.Hyperspectral images are shown as 3D cubes.Therefore, 3D spatial filtering offers an inherently simple and effective method to simultaneously extract spectral spatial features in such images.In this study, a 3D convolutional neural network (3D CNN) method is proposed for hyperspectral image classification.The proposed method effectively extracts deep spectral spatially combined features.At the same time, the hyperspectral image cube displays data in aggregate without relying on any pre-processing or post-processing. The hyperspectral image cube is first divided into small overlapping 3D patches.Then these patches are processed to create 3D feature maps using a 3D kernel function on multiple adjacent bands that also preserve spectral information.The proposed method was tested with Indian pines, Pavia university and Botswana datasets.As a result of the experimental studies, the overall accuracy results were obtained 99.35% for Indian pines, 99.90% for the University Pavia, and 99.59% for Botswana.The results were compared with 4 different deep learning-based methods.From the experimental results, it is seen that our proposed 3D CNN method performs better.

Keywords

• Hyperspectral image classification
• Deep learning
• 2D CNN
• 3D CNN

3 Boyutlu Evrişimsel Sinir Ağı Kullanılarak Hiperspektral Görüntülerin Sınıflandırılması

Öz

Hiperspektral görüntü sınıflandırma, uzaktan algılanan görüntülerin analizi için yaygın olarak kullanılmaktadır. Bir hiperspektral görüntü, uygulamalarda büyük potansiyele sahip olan yer nesnelerinin zengin spektral bilgilerini ve uzamsal bilgilerini içermektedir. Spektral uzamsal bilgi kullanımı hiperspektral görüntü sınıflandırmasının performansını önemli ölçüde arttırmaktadır. Hiperspektral görüntüler, 3B küpler biçiminde gösterilmektedir. Bu nedenle, 3B uzamsal filtreleme, bu tür görüntülerdeki spektral uzamsal özellikleri eşzamanlı olarak çıkarmak için doğal olarak basit ve etkili bir yöntem sunmaktadır. Bu çalışmada, hiperspektral görüntü sınıflandırması için bir 3B evrişimli sinir ağı (3B ESA) yöntemi önerilmiştir. Önerilen yöntem, derin spektral uzamsal birleştirilmiş özellikleri etkin bir şekilde çıkarmaktadır. Aynı zamanda herhangi bir ön işleme veya son işleme dayanmadan hiperspektral görüntü küpü verileri toplu olarak görüntülemektedir. Hiperspektral görüntü küpü önce küçük üst üste binen 3B parçalara bölünmektedir. Daha sonra bu parçalar, spektral bilgileri de koruyan birden çok bitişik bant üzerinde bir 3B çekirdek işlevi kullanarak 3B özellik haritaları oluşturmak için işlenmektedir. Önerilen yöntem indian pines, pavia üniversitesi ve botswana veri setleri ile test edilmiştir. Deneysel çalışmalar sonucunda, indian pines için %99,35, pavia üniversitesi için %99,90 ve botswana için ise %99,59 genel doğruluk sonuçları elde edilmiştir. Sonuçlar, 4 farklı derin öğrenme tabanlı yöntemle karşılaştırılmıştır. Deneysel sonuçlardan, önerilen 3B ESA yöntemimizin daha iyi performans gösterdiği görülmektedir.

Anahtar Kelimeler

• Hiperspektral görüntü sınıflandırma
• Derin öğrenme
• 2BESA
• 3B ESA

Kaynakça

1. [1] Li Y, Zhang H, Shen Q. Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network. Remote Sens. 2017;9(1). https://doi: 10.3390/rs9010067.
2. [2] Sun H, Ren J, Zhao H, Yan Y, Zabalza J, Marshall S. Superpixel based feature specific sparse representation for spectral-spatial classification of hyperspectral images. Remote Sens. 2019;11(5). https://doi: 10.3390/rs11050536.
3. [3] Dou P, Zeng C. Hyperspectral image classification using feature relations map learning. Remote Sens. 2020;12(18). https://doi: 10.3390/RS12182956.
4. [4] Ahmad M. Spatial prior fuzziness pool-based interactive classification of hyperspectral images. Remote Sens. 2019;11(9):1–19. https://doi: 10.3390/rs11091136.
5. [5] Ahmad M, Khan MA, Mazzara M, Distefano S, Ali M, Sarfraz MS. A Fast and Compact 3-D CNN for Hyperspectral Image Classification. IEEE Geosci. Remote Sens. Lett. 2020:1–5. https://doi: 10.1109/LGRS.2020.3043710.
6. [6] Wang Y, Yu W, Fang Z. Multiple Kernel-based SVM classification of hyperspectral images by combining spectral, spatial, and semantic information. Remote Sens. 2020;12(1). https://doi: 10.3390/RS12010120.
7. [7] Ham JS, Chen Y, Crawford MM, Ghosh J. Investigation of the random forest framework for classification of hyperspectral data. IEEE Trans. Geosci. Remote Sens. 2005;43(3):492–501. https://doi: 10.1109/TGRS.2004.842481.
8. [8] Alcolea A, Paoletti ME, Haut JM, Resano J, Plaza A. Inference in supervised spectral classifiers for on-board hyperspectral imaging: An overview. Remote Sens. 2020;12(3):1–29. https://doi: 10.3390/rs12030534.

9. [9] Ghamisi P, Dalla Mura M, Benediktsson JA. A survey on spectral-spatial classification techniques based on attribute profiles. IEEE Trans. Geosci. Remote Sens. 2015;53(5):2335–53. https://doi: 10.1109/TGRS.2014.2358934.
10. [10] Tuia D, Volpi M, Mura MD, Rakotomamonjy A, Flamary R. Automatic feature learning for spatio-spectral image classification with sparse SVM. IEEE Trans. Geosci. Remote Sens. 2014;52(10):6062–74. https://doi: 10.1109/TGRS.2013.2294724.
11. [11] Dalla Mura M, Villa A, Benediktsson JA, Chanussot J, Bruzzone L. Classification of hyperspectral images by using extended morphological attribute profiles and independent component analysis. IEEE Geosci. Remote Sens. Lett. 2011;8(3):542–46. https://doi: 10.1109/LGRS.2010.2091253.
12. [12] Jia S, Zhang X, Li Q. Spectral–spatial hyperspectral image classification using regularized low-rank representation and sparse representation-based graph cuts. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2015;8(6):2473–84.
13. [13] Qian Y, Ye M, Zhou J. Hyperspectral image classification based on structured sparse logistic regression and three-dimensional wavelet texture features. IEEE Trans. Geosci. Remote Sens. 2013;51(4):2276–91. https://doi: 10.1109/TGRS.2012.2209657.
14. [14] Hanbay K. Hyperspectral image classification using convolutional neural network and two-dimensional complex Gabor transform. J. Fac. Eng. Archit. Gazi Univ. 2020;35(1): 443–56. https://doi: 10.17341/gazimmfd.479086.
15. [15] Tang YY, Lu Y, Yuan H. Hyperspectral image classification based on three-dimensional scattering wavelet transform. IEEE Trans. Geosci. Remote Sens. 2015;53(5):2467–80. https://doi: 10.1109/TGRS.2014.2360672.
16. [16] Zhang L, Kumar V. Deep learning for Remote Sensing Data. IEEE Geosci. Remote Sens. Mag. 2016;4(2):22–40.
17. [17] Chen Y, Zhao X, Jia X. Spectral-Spatial Classification of Hyperspectral Data Based on Deep Belief Network. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2015;8(6): 2381–92. https://doi: 10.1109/JSTARS.2015.2388577.
18. [18] Zhao W, Du S. Spectral-Spatial Feature Extraction for Hyperspectral Image Classification: A Dimension Reduction and Deep Learning Approach. IEEE Trans. Geosci. Remote Sens. 2016;54(8):4544–54. https://doi: 10.1109/TGRS.2016.2543748.
19. [19] Yue J, Zhao W, Mao S, Liu H. Spectral-spatial classification of hyperspectral images using deep convolutional neural networks. Remote Sens. Lett. 2015;6(6):468–77. https://doi: 10.1080/2150704X.2015.1047045.
20. [20] Mohan A, Venkatesan M. HybridCNN based hyperspectral image classification using multiscale spatiospectral features. Infrared Phys. Technol. 2020;108:103326 https://doi: 10.1016/j.infrared.2020.103326.
21. [21] Kingma DP, Ba JL. Adam: A method for stochastic optimization. 3rd International Conference for Learning Representations, ICLR 2015. San Diego: May; 2015. p. 1–15.
22. [22] Grana M, Veganzons MA, Ayerdi B. Hyperspectral Remote Sensing Scenes [Internet]. (Erişim tarihi: 17.03.2021). Available from: https://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes .
23. [23] Data H. Deep Learning-Based Classification of Hyperspectral Data. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2014;7(6):2094–107.