Learning Based Super Resolution Application for Hyperspectral Images

Hüseyin Aydilek; Nihat İnanç

doi:10.47897/bilmes.1049338

EN TR

Learning Based Super Resolution Application for Hyperspectral Images

Öz

Due to its spectral properties, hyperspectral imaging is superior to other types of imaging tools in identifying, distinguishing and classifying objects. Hyperspectral imaging instruments can detect light reflected from certain wavelengths between infrared and ultraviolet, apart from the wavelength that the human eye can distinguish on the electromagnetic spectrum. While this feature provides detailed information about the spectral feature of the object under investigation, it causes its spatial resolution to be low due to the technical overlap between spatial resolution and spectral resolution. Today, applications of hyperspectral images are increasing in important fields such as agriculture, mining, medicine and pharmacy, especially for military purposes. In order for applications to produce more precise results, high spatial resolution is required, as well as high spectral information. Hardware solving of low spatial resolution problem is a difficult and costly method. Therefore, software solution is an interesting area in the field of image processing. In this thesis, a hybrid solution method based on deep learning and sparse representation is proposed to increase the low spatial resolution of hyperspectral images. The method obtains a super-resolution image from a single hyperspectral image with a low spatial image with a deep convolutional mesh. Later, the super-resolution image obtained and the original low-spatial-resolution hyperspectral image are fused with the dictionary learning method, resulting in a new super-resolution image with high spectral and spatial resolutions. The application results show that our method achieves successful results compared to many super resolution applications in the literature.

Anahtar Kelimeler

Remote Sensing, Hyperspectral Imaging, Super-Resolution, Deep Learning, Convolutional Neural Networks, Sparse Representation, Dictionary Learning

Learning Based Super Resolution Application for Hyperspectral Images

Öz

Due to its spectral properties, hyperspectral imaging is superior to other types of imaging tools in identifying, distinguishing and classifying objects. Hyperspectral imaging instruments can detect light reflected from certain wavelengths between infrared and ultraviolet, apart from the wavelength that the human eye can distinguish on the electromagnetic spectrum. While this feature provides detailed information about the spectral feature of the object under investigation, it causes its spatial resolution to be low due to the technical overlap between spatial resolution and spectral resolution. Today, applications of hyperspectral images are increasing in important fields such as agriculture, mining, medicine and pharmacy, especially for military purposes. In order for applications to produce more precise results, high spatial resolution is required, as well as high spectral information. Hardware solving of low spatial resolution problem is a difficult and costly method. Therefore, software solution is an interesting area in the field of image processing. In this thesis, a hybrid solution method based on deep learning and sparse representation is proposed to increase the low spatial resolution of hyperspectral images. The method obtains a super-resolution image from a single hyperspectral image with a low spatial image with a deep convolutional mesh. Later, the super-resolution image obtained and the original low-spatial-resolution hyperspectral image are fused with the dictionary learning method, resulting in a new super-resolution image with high spectral and spatial resolutions. The application results show that our method achieves successful results compared to many super resolution applications in the literature.

Anahtar Kelimeler

Remote Sensing, Hyperspectral Imaging, Super-Resolution, Deep Learning, Convolutional Neural Networks, Sparse Representation, Dictionary Learning

Kaynakça

[1] D. Wallach, F. Lamare, G. Kontaxakis, and D. Visvikis, “Super-Resolution in Respiratory Synchronized Positron Emission Tomography,” IEEE Trans. Med. Imaging, vol. 31, no. 2, pp. 438–448, Feb. 2012, doi: 10.1109/TMI.2011.2171358.
[2] V. Sze, Y.-H. Chen, T.-J. Yang, and J. S. Emer, “Efficient Processing of Deep Neural Networks: A Tutorial and Survey,” Proc. IEEE, vol. 105, no. 12, pp. 2295–2329, Dec. 2017, doi: 10.1109/JPROC.2017.2761740.
[3] G. Ciaburro, V. K. Ayyadevara, and A. Perrier, Hands-on machine learning on Google cloud platform: implementing smart and efficient analytics using Cloud ML Engine. 2018.
[4] J. Li, T. Qiu, C. Wen, K. Xie, and F.-Q. Wen, “Robust Face Recognition Using the Deep C2D-CNN Model Based on Decision-Level Fusion,” Sensors, vol. 18, no. 7, p. 2080, Jun. 2018, doi: 10.3390/s18072080.
[5] H. Yakura, S. Shinozaki, R. Nishimura, Y. Oyama, and J. Sakuma, “Malware Analysis of Imaged Binary Samples by Convolutional Neural Network with Attention Mechanism,” in Proceedings of the Eighth ACM Conference on Data and Application Security and Privacy, Tempe AZ USA, Mar. 2018, pp. 127–134. doi: 10.1145/3176258.3176335.
[6] B. Park, W. R. Windham, K. C. Lawrence, and D. P. Smith, “Contaminant Classification of Poultry Hyperspectral Imagery using a Spectral Angle Mapper Algorithm,” Biosyst. Eng., vol. 96, no. 3, pp. 323–333, Mar. 2007, doi: 10.1016/j.biosystemseng.2006.11.012.
[7] B. Zhukov, D. Oertel, F. Lanzl, and G. Reinhackel, “Unmixing-based multisensor multiresolution image fusion,” IEEE Trans. Geosci. Remote Sens., vol. 37, no. 3, pp. 1212–1226, May 1999, doi: 10.1109/36.763276.
[8] R. B. Gomez, A. Jazaeri, and M. Kafatos, “Wavelet-based hyperspectral and multispectral image fusion,” Orlando, FL, Jun. 2001, pp. 36–42. doi: 10.1117/12.428249.
[9] M. T. Eismann and R. C. Hardie, “Application of the stochastic mixing model to hyperspectral resolution enhancement,” IEEE Trans. Geosci. Remote Sens., vol. 42, no. 9, pp. 1924–1933, Sep. 2004, doi: 10.1109/TGRS.2004.830644.
[10] Y. Zhang, S. De Backer, and P. Scheunders, “Bayesian Fusion of Multispectral and Hyperspectral Image in Wavelet Domain,” in IGARSS 2008 - 2008 IEEE International Geoscience and Remote Sensing Symposium, Boston, MA, USA, 2008, p. V-69-V–72. doi: 10.1109/IGARSS.2008.4780029.

[11] Yifan Zhang, S. De Backer, and P. Scheunders, “Noise-Resistant Wavelet-Based Bayesian Fusion of Multispectral and Hyperspectral Images,” IEEE Trans. Geosci. Remote Sens., vol. 47, no. 11, pp. 3834–3843, Nov. 2009, doi: 10.1109/TGRS.2009.2017737.
[12] Q. Wei, N. Dobigeon, and J.-Y. Tourneret, “Bayesian fusion of multispectral and hyperspectral images with unknown sensor spectral response,” in 2014 IEEE International Conference on Image Processing (ICIP), Paris, France, Oct. 2014, pp. 698–702. doi: 10.1109/ICIP.2014.7025140.
[13] N. Akhtar, F. Shafait, and A. Mian, “Bayesian sparse representation for hyperspectral image super resolution,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, Jun. 2015, pp. 3631–3640. doi: 10.1109/CVPR.2015.7298986.
[14] M. A. Bendoumi, Mingyi He, and Shaohui Mei, “Hyperspectral Image Resolution Enhancement Using High-Resolution Multispectral Image Based on Spectral Unmixing,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 10, pp. 6574–6583, Oct. 2014, doi: 10.1109/TGRS.2014.2298056.
[15] K. Zhang, M. Wang, S. Yang, and L. Jiao, “Spatial–Spectral-Graph-Regularized Low-Rank Tensor Decomposition for Multispectral and Hyperspectral Image Fusion,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 11, no. 4, pp. 1030–1040, Apr. 2018, doi: 10.1109/JSTARS.2017.2785411.
[16] Wei, “Fusion of multispectral and hyperspectral images based on sparse representation,” 2014.
[17] Bo Huang, Huihui Song, Hengbin Cui, Jigen Peng, and Zongben Xu, “Spatial and Spectral Image Fusion Using Sparse Matrix Factorization,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 3, pp. 1693–1704, Mar. 2014, doi: 10.1109/TGRS.2013.2253612.
[18] F. Palsson, J. R. Sveinsson, and M. O. Ulfarsson, “Multispectral and Hyperspectral Image Fusion Using a 3-D-Convolutional Neural Network,” IEEE Geosci. Remote Sens. Lett., vol. 14, no. 5, pp. 639–643, May 2017, doi: 10.1109/LGRS.2017.2668299.
[19] F. Zhou, R. Hang, Q. Liu, and X. Yuan, “Pyramid Fully Convolutional Network for Hyperspectral and Multispectral Image Fusion,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 12, no. 5, pp. 1549–1558, May 2019, doi: 10.1109/JSTARS.2019.2910990.
[20] X.-H. Han and Y.-W. Chen, “Deep Residual Network of Spectral and Spatial Fusion for Hyperspectral Image Super-Resolution,” in 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM), Singapore, Singapore, Sep. 2019, pp. 266–270. doi: 10.1109/BigMM.2019.00-13.
[21] X. Han, J. Yu, J. Luo, and W. Sun, “Hyperspectral and Multispectral Image Fusion Using Cluster-Based Multi-Branch BP Neural Networks,” Remote Sens., vol. 11, no. 10, p. 1173, May 2019, doi: 10.3390/rs11101173.
[22] F. A. Mianji, Y. Gu, Y. Zhang, and J. Zhang, “Enhanced Self-Training Superresolution Mapping Technique for Hyperspectral Imagery,” IEEE Geosci. Remote Sens. Lett., vol. 8, no. 4, pp. 671–675, Jul. 2011, doi: 10.1109/LGRS.2010.2102334.
[23] S. He, H. Zhou, Y. Wang, W. Cao, and Z. Han, “Super-resolution reconstruction of hyperspectral images via low rank tensor modeling and total variation regularization,” in 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Beijing, China, Jul. 2016, pp. 6962–6965. doi: 10.1109/IGARSS.2016.7730816.
[24] J. Hu, Y. Li, and W. Xie, “Hyperspectral Image Super-Resolution by Spectral Difference Learning and Spatial Error Correction,” IEEE Geosci. Remote Sens. Lett., vol. 14, no. 10, pp. 1825–1829, Oct. 2017, doi: 10.1109/LGRS.2017.2737637.
[25] Y. Li, J. Hu, X. Zhao, W. Xie, and J. Li, “Hyperspectral image super-resolution using deep convolutional neural network,” Neurocomputing, vol. 266, pp. 29–41, Nov. 2017, doi: 10.1016/j.neucom.2017.05.024.
[26] Y. Li, L. Zhang, C. Dingl, W. Wei, and Y. Zhang, “Single Hyperspectral Image Super-Resolution with Grouped Deep Recursive Residual Network,” in 2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM), Xi’an, Sep. 2018, pp. 1–4. doi: 10.1109/BigMM.2018.8499097.
[27] J. Jia, L. Ji, Y. Zhao, and X. Geng, “Hyperspectral image super-resolution with spectral–spatial network,” Int. J. Remote Sens., vol. 39, no. 22, pp. 7806–7829, Nov. 2018, doi: 10.1080/01431161.2018.1471546.
[28] W. Xie, X. Jia, Y. Li, and J. Lei, “Hyperspectral Image Super-Resolution Using Deep Feature Matrix Factorization,” IEEE Trans. Geosci. Remote Sens., vol. 57, no. 8, pp. 6055–6067, Aug. 2019, doi: 10.1109/TGRS.2019.2904108.
[29] Q. Wang, Q. Li, and X. Li, “Spatial-Spectral Residual Network for Hyperspectral Image Super-Resolution,” ArXiv200104609 Cs Eess, Jan. 2020, Accessed: Dec. 17, 2021. [Online]. Available: http://arxiv.org/abs/2001.04609
[30] J. Li et al., “Hyperspectral Image Super-Resolution by Band Attention Through Adversarial Learning,” IEEE Trans. Geosci. Remote Sens., vol. 58, no. 6, pp. 4304–4318, Jun. 2020, doi: 10.1109/TGRS.2019.2962713.
[31] Y. Fu, Z. Liang, and S. You, “Bidirectional 3D Quasi-Recurrent Neural Network for Hyperspectral Image Super-Resolution,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 14, pp. 2674–2688, 2021, doi: 10.1109/JSTARS.2021.3057936.
[32] S. Albawi, T. A. Mohammed, and S. Al-Zawi, “Understanding of a convolutional neural network,” in 2017 International Conference on Engineering and Technology (ICET), Antalya, Aug. 2017, pp. 1–6. doi: 10.1109/ICEngTechnol.2017.8308186.
[33] M. Elad, Sparse and redundant representations: from theory to applications in signal and image processing. New York, NY Heidelberg: Springer, 2010.
[34] J. M. Bioucas-Dias et al., “Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 5, no. 2, pp. 354–379, Apr. 2012, doi: 10.1109/JSTARS.2012.2194696.
[35] F. A. Kruse et al., “The spectral image processing system (SIPS)—interactive visualization and analysis of imaging spectrometer data,” Remote Sens. Environ., vol. 44, no. 2–3, pp. 145–163, May 1993, doi: 10.1016/0034-4257(93)90013-N.

Ayrıntılar

Birincil Dil

İngilizce

Konular

Mühendislik

Bölüm

Araştırma Makalesi

Yazarlar

Hüseyin Aydilek ^*
0000-0003-3051-4259
Türkiye

Nihat İnanç
0000-0003-2989-6632
Türkiye

Yayımlanma Tarihi

31 Aralık 2021

Gönderilme Tarihi

12 Ekim 2021

Kabul Tarihi

19 Aralık 2021

Yayımlandığı Sayı

Yıl 2021 Cilt: 5 Sayı: 2

DOI

https://doi.org/10.47897/bilmes.1049338

IZ

https://izlik.org/JA38RW39XF

APA

Aydilek, H., & İnanç, N. (2021). Learning Based Super Resolution Application for Hyperspectral Images. International Scientific and Vocational Studies Journal, 5(2), 210-217. https://doi.org/10.47897/bilmes.1049338

AMA

1.Aydilek H, İnanç N. Learning Based Super Resolution Application for Hyperspectral Images. ISVOS. 2021;5(2):210-217. doi:10.47897/bilmes.1049338

Chicago

Aydilek, Hüseyin, ve Nihat İnanç. 2021. “Learning Based Super Resolution Application for Hyperspectral Images”. International Scientific and Vocational Studies Journal 5 (2): 210-17. https://doi.org/10.47897/bilmes.1049338.

EndNote

Aydilek H, İnanç N (01 Aralık 2021) Learning Based Super Resolution Application for Hyperspectral Images. International Scientific and Vocational Studies Journal 5 2 210–217.

IEEE

[1]H. Aydilek ve N. İnanç, “Learning Based Super Resolution Application for Hyperspectral Images”, ISVOS, c. 5, sy 2, ss. 210–217, Ara. 2021, doi: 10.47897/bilmes.1049338.

ISNAD

Aydilek, Hüseyin - İnanç, Nihat. “Learning Based Super Resolution Application for Hyperspectral Images”. International Scientific and Vocational Studies Journal 5/2 (01 Aralık 2021): 210-217. https://doi.org/10.47897/bilmes.1049338.

JAMA

1.Aydilek H, İnanç N. Learning Based Super Resolution Application for Hyperspectral Images. ISVOS. 2021;5:210–217.

MLA

Aydilek, Hüseyin, ve Nihat İnanç. “Learning Based Super Resolution Application for Hyperspectral Images”. International Scientific and Vocational Studies Journal, c. 5, sy 2, Aralık 2021, ss. 210-7, doi:10.47897/bilmes.1049338.

Vancouver

1.Hüseyin Aydilek, Nihat İnanç. Learning Based Super Resolution Application for Hyperspectral Images. ISVOS. 01 Aralık 2021;5(2):210-7. doi:10.47897/bilmes.1049338

ULUSLARARASI BİLİMSEL VE MESLEKİ ÇALIŞMALAR DERGİSİ, Creative Commons Atıf-GayrıTicari 4.0 Uluslararası (CC BY-NC 4.0) lisansı ile yayınlamasına izin verir. Creative Commons Atıf-GayrıTicari 4.0 Uluslararası (CC BY-NC 4.0) lisansı, eserin ticari kullanım dışında her boyut ve formatta paylaşılmasına, kopyalanmasına, çoğaltılmasına ve orijinal esere uygun şekilde atıfta bulunmak kaydıyla yeniden düzenleme, dönüştürme ve eserin üzerine inşa etme dâhil adapte edilmesine izin verir.

Learning Based Super Resolution Application for Hyperspectral Images

Öz

Anahtar Kelimeler

Learning Based Super Resolution Application for Hyperspectral Images

Öz

Anahtar Kelimeler

Kaynakça

Ayrıntılar

Birincil Dil

Konular

Bölüm

Yazarlar

Yayımlanma Tarihi

Gönderilme Tarihi

Kabul Tarihi

Yayımlandığı Sayı

DOI

IZ

Kaynak Göster