Classification of Hyperspectral Remote Sensing Images Using Hybrid 3D-2D CNN Architecture

Abstract

Hyperspectral remote sensing images (HRSI) are image cubes with two spatial and one spectral dimensions. Convolutional neural network (CNN) is one of the most effective deep learning methods for extracting spatial-spectral feature information in HRSI classification. Traditional CNN-based methods usually use 2D CNN for feature extraction. Because 2D CNN captures only spatial-dimensional features, it cannot extract good feature maps from spectral dimensions. 3D CNN can simultaneously extract spatial-spectral features in HRSI. However, 3D CNN is computationally complex. In this study, a hybrid method consisting of 3D-2D CNN combination is proposed. While 3D CNN extracts common spatial-spectral features, more spatial features are learned with 2D CNN used after 3D CNN. In addition, the proposed hybrid method reduces the computational complexity. Mish activation function is used to increase the classification performance of the proposed method. As a result of the applications performed with two commonly used datasets, it is seen that the proposed method has better classification performance.

Keywords

• Remote Sensing
• Hyperspectral Image Classification
• Deep Learning

Hibrid 3B-2B ESA Mimarisi Kullanılarak Hiperspektral Uzaktan Algılama Görüntülerinin Sınıflandırılması

Abstract

Hiperspektral uzaktan algılama görüntüleri (HUAG) iki uzamsal ve bir spektral boyuta sahip görüntü küpleridir. Evrişimsel sinir ağı (ESA), HUAG sınıflandırmada uzamsal-spektral özellik bilgilerinin çıkarılmasını sağlayan en etkili derin öğrenme yöntemlerinden biridir. Geleneksel ESA tabanlı yöntemler genellikle özellik çıkarımı için 2B ESA’yı kullanmaktadır. 2B ESA yalnızca uzamsal boyutttaki özellikleri yakaladığından dolayı tek başına spektral boyutlardan iyi ayırt edici özellik haritaları çıkaramaz. 3B ESA HUAG’deki uzamsal-spektral özellikleri eş zamanlı olarak çıkarabilmektedir. Ancak 3B ESA hesaplama açısından karmaşıktır. Bu çalışmada 3B-2B ESA birleşiminden oluşan hibrid bir yöntem önerilmiştir. 3B ESA ortak uzamsal-spektral özellikleri çıkarırken, 3B ESA’dan sonra kullanılan 2B ESA ile daha fazla uzamsal özellikler öğrenilir. Ayrıca önerilen hibrid yöntem hesaplama karmaşıklığını da azaltmaktadır. Önerilen yöntemin sınıflandırma performansını arttırmak için Mish aktivasyon fonksiyonu kullanılmaktadır. Yaygın olarak kullanılan iki veriseti ile gerçekleştirilen uygulamalar sonucunda önerilen yöntemin daha iyi sınıflandırma performansına sahip olduğu görülmektedir.

Keywords

• Uzaktan Algılama
• Hiperspektral Görüntü Sınıflandırma
• Derin öğrenme

References

1. Ahmad, M., Khan, A.M., Mazzara, M., Distefano, S., Ali, M., Sarfraz, M.S. (2020) A Fast and Compact 3-D CNN for Hyperspectral Image Classification. IEEE Geoscience Remote Sensing Letters 1–5. https://doi.org/10.1109/LGRS.2020.3043710
2. Chen, Y., Zhao, X., Jia, X. (2015) Spectral-Spatial Classification of Hyperspectral Data Based on Deep Belief Network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 8: 2381–2392. https://doi.org/10.1109/JSTARS.2015.2388577
3. Firat, H., Hanbay, D. (2021) 3B ESA Tabanlı ResNet50 Kullanılarak Hiperspektral Görüntülerin Sınıflandırılması Classification of Hyperspectral Images Using 3D CNN Based ResNet50. 2021 29th Signal Processing and Communications Applications Conference (SIU), Istanbul-Tukey, pp.6–9. https://doi.org/10.1109/SIU53274.2021.9477899
4. Ge, Z., Cao, G., Li, X., Fu, P. (2020) Hyperspectral Image Classification Method Based on 2D-3D CNN and Multibranch Feature Fusion. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 13: 5776–5788. https://doi.org/10.1109/JSTARS.2020.3024841
5. Ham, J.S., Chen, Y., Crawford, M.M., Ghosh, J. (2005) Investigation of the random forest framework for classification of hyperspectral data. IEEE Transactions on Geoscience and Remote Sensing 43: 492–501. https://doi.org/10.1109/TGRS.2004.842481
6. Jia, J., Wang, Y., Chen, J., Guo, R., Shu, R., Wang, J. (2020) Status and application of advanced airborne hyperspectral imaging technology: A review. Infrared Physics & Technology 104: 103115. https://doi.org/10.1016/j.infrared.2019.103115
7. Kang, X., Duan, P., Li, S. (2020) Hyperspectral image visualization with edge-preserving filtering and principal component analysis. Information Fusion 57: 130–143. https://doi.org/10.1016/j.inffus.2019.12.003
8. Meng, Z., Li, L., Tang, X., Feng, Z., Jiao, L., Liang, M. (2019) Multipath residual network for spectral-spatial hyperspectral image classification. Remote Sensing 11: 1–19. https://doi.org/10.3390/rs11161896

9. Mingyi He, Bo Li, H.C. (2017) Multi-scale 3D deep convolutional neural network for hyperspectral image classification. 2017 IEEE International Conference on Image Processing, Beijing-China, pp: 3904–3908.
10. Misra, D. (2019) Mish: A Self Regularized Non-Monotonic Activation Function.
11. Mohan, A., Venkatesan, M. (2020) HybridCNN based hyperspectral image classification using multiscale spatiospectral features. Infrared Physics & Technology 108. https://doi.org/10.1016/j.infrared.2020.103326
12. Roy, S.K., Krishna, G., Dubey, S.R., Chaudhuri, B.B. (2019) HybridSN: Exploring 3D-2D CNN Feature Hierarchy for Hyperspectral Image Classification. arXiv 17: 277–281.
13. Üzen, H., Türkoğlu, M., Hanbay, D. (2021) Derin U-Net Ağ Mimarileri Kullanarak Yüzey Hata Tespiti Surface Defect Detection Using Deep U-Net Network Architectures. 2021 29th Signal Processing and Communications Applications Conference (SIU), Istanbul-Tukey https://doi.org/10.1109/SIU53274.2021.9477790
14. Villa, A., Benediktsson, J.A., Chanussot, J., Jutten, C. (2011) Hyperspectral image classification with Independent component discriminant analysis. IEEE Transactions on Geoscience and Remote Sensing 49: 4865–4876. https://doi.org/10.1109/TGRS.2011.2153861
15. Wang, Y., Yu, W., Fang, Z. (2020) Multiple Kernel-based SVM classification of hyperspectral images by combining spectral, spatial, and semantic information. Remote Sensing 12. https://doi.org/10.3390/RS12010120
16. Xu, H., Zhang, H., He, W., Zhang, L. (2019) Superpixel-based spatial-spectral dimension reduction for hyperspectral imagery classification. Neurocomputing 360: 138–150. https://doi.org/10.1016/j.neucom.2019.06.023
17. Y. Chen, H. Jiang, C. Li, X. Jia, P.G. (2016) Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks. IEEE Transactions on Geoscience and Remote Sensing 54: 6232–6251. https://doi.org/10.1109/TGRS. 2016.2584107
18. Zhang, L., Zhang, L., Kumar, V. (2016) Deep learning for Remote Sensing Data. IEEE Geoscience and Remote Sensing Magazine 4: 22–40.
19. Zhong, Z., Li, J., Luo, Z., Chapman, M. (2018) Spectral-Spatial Residual Network for Hyperspectral Image Classification: A 3-D Deep Learning Framework. IEEE Transactions on Geoscience and Remote Sensing 56: 847–858. https://doi.org/10.1109/TGRS.2017.2755542