A New and Efficient Pan Sharpening Method Based on Optimized Pixel Coefficients

Abstract

Pan sharpening aims to create a multispectral, high spatial resolution image by combining the multispectral image (MSI) with a high spatial resolution panchromatic image (PAN). Pan sharpening methods are performed between the MS image, which is the MSI image brought to PAN dimensions with the help of interpolation, and the PAN image. In this study, PAN sharpening is approached as an optimization problem. It is assumed that the optimal solution consists of multiplying the pixels of the MS image by optimized coefficients. It would be costly to optimize all the coefficients in this coefficient matrix one by one. For this reason, these coefficients were tried to be found with 5 different optimizationbased methods. It was also compared with 19 different methods commonly used in the literature. 6 different evaluation criteria were used for this comparison. These comparisons were made on 3 different datasets. It has been observed that the proposed methods are superior to other methods.

Keywords

• Pan Sharpening
• Image Fusion
• Remote Sensing
• Optimization

References

1. Aiazzi, B., Alparone, L., Baronti, S., & Garzelli, A. (2002). Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis. IEEE Transactions on geoscience and remote sensing, 40(10), 2300-2312. https://doi.org/10.1109/TGRS.2002.803623
2. Aiazzi, B., Alparone, L., Baronti, S., Garzelli, A., & Selva, M. (2006). MTF-tailored multiscale fusion of high-resolution MS and Pan imagery. Photogrammetric Engineering & Remote Sensing, 72(5), 591-596. https://doi.org/10.14358/PERS.72.5.591
3. Alparone, L., Aiazzi, B., Baronti, S., Garzelli, A., Nencini, F., & Selva, M. (2008). Multispectral and panchromatic data fusion assessment without reference. Photogrammetric Engineering & Remote Sensing, 74(2), 193-200. https://doi.org/10.14358/PERS.74.2.193
4. Amro, I., Mateos, J., Vega, M., Molina, R., & Katsaggelos, A. K. (2011). A survey of classical methods and new trends in pansharpening of multispectral images. EURASIP Journal on Advances in Signal Processing, 2011(1), 1-22. https://doi.org/10.1186/1687-6180-2011-79
5. Ciotola, M., Poggi, G., & Scarpa, G. (2023). Unsupervised Deep Learning-based Pansharpening with Jointly-Enhanced Spectral and Spatial Fidelity. IEEE Transactions on geoscience and remote sensing. https://doi.org/10.48550/arXiv.2307.14403
6. Civicioglu, P., & Besdok, E. (2022). Contrast stretching based pansharpening by using weighted differential evolution algorithm. Expert Systems with Applications, 208, 118144. https://doi.org/10.1016/j.eswa.2022.118144
7. Civicioglu, P., & Besdok, E. (2024). Pansharpening of remote sensing images using dominant pixels. Expert Systems with Applications, 242, 122783. https://doi.org/10.1016/j.eswa.2023.122783
8. Civicioglu, P., Besdok, E., Gunen, M. A., & Atasever, U. H. (2020). Weighted differential evolution algorithm for numerical function optimization: a comparative study with cuckoo search, artificial bee colony, adaptive differential evolution, and backtracking search optimization algorithms. Neural Computing and Applications, 32, 3923-3937. https://doi.org/10.1007/s00521-018-3822-5

9. Feng, Y., Yan, B., Jeon, S., & Yang, X. (2024). A hyperspectral pansharpening method using retrain transformer network for remote sensing images in UAV communications system. Wireless Networks, 1-14. https://doi.org/10.1007/s11276-023-03611-2
10. Garzelli, A. (2014). Pansharpening of multispectral images based on nonlocal parameter optimization. IEEE Transactions on geoscience and remote sensing, 53(4), 2096-2107. https://doi.org/10.1109/TGRS.2014.2354471
11. Gezgin. (2013). (Accessed:01/11/2023) https://gezgin.gov.tr/geoportal/app/main?execution=e3s1
12. Ghassemian, H. (2016). A review of remote sensing image fusion methods. Information Fusion, 32, 75-89. https://doi.org/10.1016/j.inffus.2016.03.003
13. Günen, M. A. (2021). Weighted differential evolution algorithm based pansharpening. International Journal of Remote Sensing, 42(22), 8468-8491. https://doi.org/10.1080/01431161.2021.1976874
14. Kallel, A. (2014). MTF-adjusted pansharpening approach based on coupled multiresolution decompositions. IEEE Transactions on geoscience and remote sensing, 53(6), 3124-3145. https://doi.org/10.1109/TGRS.2014.2369056
15. Kurban, T. (2022). Region based multi-spectral fusion method for remote sensing images using differential search algorithm and IHS transform. Expert Systems with Applications, 189, 116135. https://doi.org/10.1016/j.eswa.2021.116135
16. Lolli, S., Alparone, L., Garzelli, A., & Vivone, G. (2017). Haze correction for contrast-based multispectral pansharpening. IEEE Geoscience and Remote Sensing Letters, 14(12), 2255-2259. https://doi.org/10.1109/LGRS.2017.2761021
17. Masi, G., Cozzolino, D., Verdoliva, L., & Scarpa, G. (2016). Pansharpening by convolutional neural networks. Remote Sensing, 8(7), 594. https://doi.org/10.3390/rs8070594
18. Meng, X., Shen, H., Li, H., Zhang, L., & Fu, R. (2019). Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: Practical discussion and challenges. Information Fusion, 46, 102-113. https://doi.org/10.1016/j.inffus.2018.05.006
19. Meng, X., Xiong, Y., Shao, F., Shen, H., Sun, W., Yang, G., Yuan, Q., Fu, R., & Zhang, H. (2020). A large-scale benchmark data set for evaluating pansharpening performance: Overview and implementation. IEEE Geoscience and Remote Sensing Magazine, 9(1), 18-52. https://doi.org/10.1109/MGRS.2020.2976696
20. Open Remote Sensing. (2015). (Accessed:01/11/2023) https://openremotesensing.net/knowledgebase/a-critical-comparison-among-pansharpening-algorithms/
21. Otazu, X., González-Audícana, M., Fors, O., & Núñez, J. (2005). Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods. IEEE Transactions on geoscience and remote sensing, 43(10), 2376-2385. https://doi.org/10.1109/TGRS.2005.856106
22. Palsson, F., Sveinsson, J. R., & Ulfarsson, M. O. (2013). A new pansharpening algorithm based on total variation. IEEE Geoscience and Remote Sensing Letters, 11(1), 318-322. https://doi.org/10.1109/LGRS.2013.2257669
23. Restaino, R., Vivone, G., Dalla Mura, M., & Chanussot, J. (2016). Fusion of multispectral and panchromatic images based on morphological operators. IEEE Transactions on Image Processing, 25(6), 2882-2895. https://doi.org/10.1109/tip.2016.2556944
24. Scarpa, G., Vitale, S., & Cozzolino, D. (2018). Target-adaptive CNN-based pansharpening. IEEE Transactions on geoscience and remote sensing, 56(9), 5443-5457. https://doi.org/10.1109/TGRS.2018.2817393
25. Shahdoosti, H. R., & Ghassemian, H. (2014). Fusion of MS and PAN images preserving spectral quality. IEEE Geoscience and Remote Sensing Letters, 12(3), 611-615. https://doi.org/10.1109/LGRS.2014.2353135
26. Tu, T., Su, S., Shyn, H., & Huang, P. (2001). A new look at IHS-like image fusion methods, Information Fusion. https://doi.org/10.1016/S1566-2535(01)00036-7
27. Vivone, G. (2019). Robust band-dependent spatial-detail approaches for panchromatic sharpening. IEEE Transactions on geoscience and remote sensing, 57(9), 6421-6433. https://doi.org/10.1109/TGRS.2019.2906073
28. Vivone, G., Alparone, L., Chanussot, J., Dalla Mura, M., Garzelli, A., Licciardi, G. A., Restaino, R., & Wald, L. (2014a). A critical comparison among pansharpening algorithms. IEEE Transactions on geoscience and remote sensing, 53(5), 2565-2586. https://doi.org/10.1109/TGRS.2014.2361734
29. Vivone, G., Dalla Mura, M., Garzelli, A., Restaino, R., Scarpa, G., Ulfarsson, M. O., Alparone, L., & Chanussot, J. (2020). A new benchmark based on recent advances in multispectral pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods. IEEE Geoscience and Remote Sensing Magazine, 9(1), 53-81. https://doi.org/10.1109/MGRS.2020.3019315
30. Vivone, G., Restaino, R., & Chanussot, J. (2017). A regression-based high-pass modulation pansharpening approach. IEEE Transactions on geoscience and remote sensing, 56(2), 984-996. https://doi.org/10.1109/TGRS.2017.2757508
31. Vivone, G., Restaino, R., & Chanussot, J. (2018). Full scale regression-based injection coefficients for panchromatic sharpening. IEEE Transactions on Image Processing, 27(7), 3418-3431. https://doi.org/10.1109/TIP.2018.2819501
32. Vivone, G., Restaino, R., Dalla Mura, M., Licciardi, G., & Chanussot, J. (2013). Contrast and error-based fusion schemes for multispectral image pansharpening. IEEE Geoscience and Remote Sensing Letters, 11(5), 930-934. https://doi.org/10.1109/LGRS.2013.2281996
33. Vivone, G., Simões, M., Dalla Mura, M., Restaino, R., Bioucas-Dias, J. M., Licciardi, G. A., & Chanussot, J. (2014b). Pansharpening based on semiblind deconvolution. IEEE Transactions on geoscience and remote sensing, 53(4), 1997-2010. https://doi.org/10.1109/TGRS.2014.2351754
34. Wald, L., Ranchin, T., & Mangolini, M. (1997). Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images. Photogrammetric engineering and remote sensing, 63(6), 691-699.
35. Yilmaz, C. S., Yilmaz, V., & Gungor, O. (2022). A theoretical and practical survey of image fusion methods for multispectral pansharpening. Information Fusion, 79, 1-43. https://doi.org/10.1016/j.inffus.2021.10.001