Deep Learning Based Panchromatic2RGB Image Generation from VHR Images

Abstract

Image colorization is the process of obtaining colored images by assigning RGB color values to a grayscale or panchromatic image. This technique has an important place in the field of computer vision because colored images provide a better visual experience and are widely used in areas such as image recognition and object detection. It also has many practical applications such as coloring historical photographs, adding colors to be used in the analysis of medical images, and improving the analysis of satellite images. Colorization methods are divided into two main categories: brush coloring and sample-based coloring. Both methods have certain limitations. The performance of these methods depends on the selected reference images and may sometimes contain false colors or significant errors. While these methods require operator intervention or pre-defined rules, deep learning based methods are largely automated and uses neural networks to understand the global and local context of an image, leading to more realistic and contextually accurate colorizations. The presented study uses the Denoising Diffusion Null-Space Model (DDNM) architecture. DDNM is a method that aims to obtain more efficient and high-quality results compared to the coloring approaches available in literature. In the study, the weight data of the DDNM architecture was used to predict colored images from panchromatic images using the SpaceNet 6 open access dataset. The SpaceNet 6 dataset includes a combination of Capella Space 0.5m Synthetic Aperture Radar (SAR) imagery and Maxar's 0.5m electro-optical (EO) imagery. In order to assess the results, Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) accuracy metrics are calculated.

Keywords

• Image Colarization
• Deep Learning
• Remote Sensing
• SpaceNet 6

References

1. Zhang, R., Isola, P., Efros, A.A. (2016). Colorful Image Colorization. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds) Computer Vision – ECCV 2016. ECCV 2016. Lecture Notes in Computer Science(), vol 9907. Springer, Cham. https://doi.org/10.1007/978-3-319-46487-9_40
2. Huang, S., Jin, X., Jiang, Q., & Liu, L. (2022). Deep learning for image colorization: Current and future prospects. Engineering Applications of Artificial Intelligence, 114, 105006.
3. Li, F., Ma, L., & Cai, J. (2018). Multi-discriminator generative adversarial network for high resolution gray-scale satellite image colorization. In IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium (pp. 3489-3492). IEEE.
4. Wu, M., Jin, X., Jiang, Q., Lee, S. J., Guo, L., Di, Y., ... & Huang, J. (2019). Remote sensing image colorization based on multiscale SEnet GAN. In 2019 12th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI) (pp. 1-6). IEEE
5. Ji, G., Wang, Z., Zhou, L., Xia, Y., Zhong, S., & Gong, S. (2020). SAR image colorization using multidomain cycle-consistency generative adversarial network. IEEE Geoscience and Remote Sensing Letters, 18(2), 296-300
6. Wu, M., Jin, X., Jiang, Q., Lee, S. J., Liang, W., Lin, G., & Yao, S. (2021). Remote sensing image colorization using symmetrical multi-scale DCGAN in YUV color space. The Visual Computer, 37, 1707-1729.
7. Wang, Y., Yu, J., & Zhang, J. (2022). Zero-shot image restoration using denoising diffusion null-space model. arXiv preprint arXiv:2212.00490
8. Anwar, Saeed, Muhammad Tahir, Chongyi Li, Ajmal Mian, Fahad Shahbaz Khan, and Abdul Wahab Muzaffar. "Image colorization: A survey and dataset." Information Fusion 114 (2025): 102720.

9. Bose, R., Banerjee, A. S. B., & Chaudhuri, S. (2022). DARK: Few-shot remote sensing colorization using label conditioned color injection. IEEE Geoscience and Remote Sensing Lett
10. Fu, Q., Xia, S., Kang, Y., Sun, M., & Tan, K. (2024). Satellite Remote Sensing Grayscale Image Colorization Based on Denoising Generative Adversarial Network. Remote Sensing, 16(19), 3644
11. Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., & Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition (pp. 248-255)
12. Liu, Z., Luo, P., Wang, X., & Tang, X. (2015). Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision (pp. 3730-3738)
13. Wang, J., Nie, J., Chen, H., Xie, H., Zheng, C., Ye, M., & Wei, Z. (2022). Remote sensing image colorization based on joint stream deep convolutional generative adversarial networks. In Proceedings of the 4th ACM International Conference on Multimedia in Asia (pp. 1-8)
14. Ertan, Z., Korkut, B., Gördük, G., Kulavuz, B., Bakırman, T., & Bayram, B. (2024). Enhancement of underwater images with artificial intelligence. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 48, 149-156
15. Schwab, J., Antholzer, S., & Haltmeier, M. (2019). Deep null space learning for inverse problems: convergence analysis and rates. Inverse Problems, 35(2),
16. Shermeyer, J., Hogan, D., Brown, J., Van Etten, A., Weir, N., Pacifici, F., ... & Lewis, R. (2020). SpaceNet 6: Multi-sensor all weather mapping dataset. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops (pp. 196-197)
17. Unal, M., Yakar, M., & Yildiz, F. (2004, July). Discontinuity surface roughness measurement techniques and the evaluation of digital photogrammetric method. In Proceedings of the 20th international congress for photogrammetry and remote sensing, ISPRS (Vol. 1103, p. 1108).