Edge Boosted Global Awared Low-light Image Enhancement Network

Abstract

Low-light images are captured in situations where the lighting is poor or the camera hardware is not capable of producing good quality images. These types of images tend to have low contrast, blurry details, noise, and color distortion. In computer vision applications, image brightness plays a crucial role, and therefore, low-light image enhancement is used as a preprocessing step. In this study, we have improved the Low-Light Enhancement Network with Global Awareness (GLADNet) method by adding a UNet-based edge information extraction unit. The channel attention mechanism was also incorporated into the edge information extraction unit to achieve color preservation. Our experiments show that our proposed method has achieved higher PSNR, SSIM, and FSIM metrics compared to reference images. Additionally, it has produced lower NIQE and BRISQUE values for non-reference performance evaluation. Moreover, our proposed method removes noise better and produces visual results that are closer to the target images.

Keywords

• Low-light Image Enhancement
• U-net Based Edge Extraction
• Channel Attention
• Color Preservation
• Improved GladNET
• Canny

Kenar Destekli Global Farkındalı Düşük Aydınlıklı Görüntü İyileştirme Ağı

Öz

Düşük aydınlıklı görüntüler, ortam aydınlığının zayıf olduğu veya kamera donanımının iyi kalitede görüntüler üretemediği durumlarda yakalanır. Bu tür görüntüler düşük kontrast, bulanık ayrıntılar, gürültü ve renk bozulmasına sahip olma eğilimindedir. Bilgisayarlı görü uygulamalarında, görüntü parlaklığı çok önemli bir rol oynar ve bu nedenle, düşük ışıklı görüntü iyileştirme bir ön işleme adımı olarak kullanılır. Bu çalışmada, Küresel Farkındalık ile Düşük Işık İyileştirme Ağı (GLADNet) yöntemini UNet tabanlı bir kenar bilgisi çıkarma birimi ekleyerek geliştirdik. Renk korumasını sağlamak için de kanal dikkat mekanizması kenar bilgisi çıkarma birimine dahil ettik. Deneylerimiz, önerilen yöntemimizin referans görüntülere karşılaştırıldığında daha yüksek PSNR, SSIM ve FSIM metriklerine ulaştığını göstermektedir. Ayrıca, referans olmayan performans değerlendirmelerinde daha düşük NIQE ve BRISQUE değerlerine ulaşılmıştır. Önerdiğimiz yöntemin gürültüyü daha iyi gidermede ve hedef görüntülere daha yakın görsel sonuçlar ürettiği görülmüştür.

Anahtar Kelimeler

• Düşük Aydınlıklı Görüntü İyileştirme
• U-net Tabanlı Kenar Çıkarımı
• Kanal Dikkat
• Renk Korunumu
• Geliştirilmiş GladNET
• Canny

References

1. [1] C. Li et al., "Low-Light Image and Video En-hancement Using Deep Learning: A Survey," in IEEE Transactions on Pattern Analysis and Ma-chine Intelligence, vol. 44, no. 12, pp. 9396- 9416, 1 Dec. 2022, doi: 10.1109/TPAMI.2021.3126387.
2. [2] W. Wang, X. Wu, X. Yuan and Z. Gao, "An Ex-periment-Based Review of Low-Light Image En-hancement Methods," in IEEE Access, vol. 8, pp. 87884-87917, 2020, doi: 10.1109/ACCESS.2020.2992749.
3. [3] N. P. Galatsanos, C. A. Segall and A. K. Katsagge-los, "Digital image enhancement", Encyclopedia of Optical Engineering, pp. 388- 402, 2003.
4. [4] X. Liu, M. Pedersen and R. Wang, "Survey of natural image enhancement techniques: Classifi-cation evaluation challenges and perspectives", Digital Signal Processing, pp. 103547, 2022.
5. [5] S. M. Pizer et al., "Adaptive histogram equaliza-tion and its variations", Comput. Vis. Graph. Im-age Process., vol. 39, no. 3, pp. 355-368, 1987.
6. [6] K. Zuiderveld, "Contrast limited adaptive histo-gram equalization", Proc. Graph. Gems, pp. 474-485, 1994.
7. [7] H. Ibrahim and N. S. Pik Kong, "Brightness Pre-serving Dynamic Histogram Equalization for Im-age Contrast Enhancement," in IEEE Transactions on Consumer Electronics, vol. 53, no. 4, pp. 1752-1758, Nov. 2007, doi: 10.1109/TCE.2007.4429280.
8. [8] K. Nakai, Y. Hoshi and A. Taguchi, "Color image contrast enhacement method based on differen-tial intensity/saturation gray-levels histograms," 2013 International Symposium on Intelligent Sig-nal Processing and Communication Systems, Na-ha, Japan, 2013, pp. 445-449, doi: 10.1109/ISPACS.2013.6704591.

9. [9] Yu Wang, Qian Chen and Baeomin Zhang, "Image enhancement based on equal area dualistic subi-mage histogram equalization method," in IEEE Transactions on Consumer Electronics, vol. 45, no. 1, pp. 68-75, Feb. 1999, doi: 10.1109/30.754419.
10. [10] S. Rahman, M. M. Rahman, M. Abdullah-AlWadud, G. D. Al-Quaderi and M. Shoyaib, "An adaptive gamma correction for image enhance-ment", EURASIP J. Image Video Process., vol. 2016, no. 1, pp. 1-13, Dec. 2016.
11. [11] Z.-G. Wang, Z.-H. Liang and C.-L. Liu, "A realtime image processor with combining dynam-ic contrast ratio enhancement andinverse gamma correction for PDP", Displays, vol. 30, no. 3, pp. 133-139, Jul. 2009.
12. [12] S. -C. Huang, F. -C. Cheng and Y. -S. Chiu, "Effi-cient Contrast Enhancement Using Adaptive Gamma Correction with Weighting Distribution," in IEEE Transactions on Image Processing, vol. 22, no. 3, pp. 1032-1041, March 2013, doi: 10.1109/TIP.2012.2226047.
13. [13] D. J. Jobson, Z. Rahman and G. A. Woodell, "Properties and performance of a center/surround retinex," in IEEE Transactions on Image Pro-cessing, vol. 6, no. 3, pp. 451-462, March 1997, doi: 10.1109/83.557356.
14. [14] Z. Rahman, D. J. Jobson and G. A. Woodell, "Multi-scale retinex for color image enhance-ment", Proc. 3rd IEEE Int. Conf. Image Process., pp. 1003-1006, Sep. 1996.
15. [15] D. J. Jobson, Z. Rahman and G. A. Woodell, "A multiscale retinex for bridging the gap between color images and the human observation of scenes," in IEEE Transactions on Image Pro-cessing, vol. 6, no. 7, pp. 965-976, July 1997, doi: 10.1109/83.597272.
16. [16] S. Wang, J. Zheng, H. -M. Hu and B. Li, "Natural-ness Preserved Enhancement Algorithm for Non-Uniform Illumination Images," in IEEE Transac-tions on Image Processing, vol. 22, no. 9, pp. 3538-3548, Sept. 2013, doi: 10.1109/TIP.2013.2261309.
17. [17] K. G. Lore, A. Akintayo and S. Sarkar, "LLNet: A deep autoencoder approach to natural low-light image enhancement", Pattern Recognit., vol. 61, pp. 650-662, Jan. 2017.
18. [18] F. Lv, F. Lu and J. Wu, "MBLLEN: Low-light im-age/video enhancement using CNNs", Proc. Brit. Mach. Vis. Conf., pp. 1-13, 2018.
19. [19] C. Guo et al., "Zero-Reference Deep Curve Esti-mation for Low-Light Image Enhancement," 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 1777-1786, doi: 10.1109/CVPR42600.2020.00185.
20. [20] Soong-Der Chen and A. R. Ramli, "Contrast en-hancement using recursive mean-separate histo-gram equalization for scalable brightness preser-vation," in IEEE Transactions on Consumer Elec-tronics, vol. 49, no. 4, pp. 1301-1309, Nov. 2003, doi: 10.1109/TCE.2003.1261233.
21. [21] Y. Xiao, A. Jiang, J. Ye and M. -W. Wang, "Mak-ing of Night Vision: Object Detection Under Low-Illumination," in IEEE Access, vol. 8, pp. 123075-123086, 2020, doi: 10.1109/ACESS.2020.3007610.
22. [22] E. Land, "Lightness and retinex theory", J. Opt. Soc. Amer., vol. 58, pp. 1428A, 1967.
23. [23] S. Dong, P. Wang and K. Abbas, "A survey on deep learning and its applications", Comput. Sci. Rev., 2021.
24. [24] P. -M. -L. Nguyen, J. -H. Cho and S. B. Cho, "An architecture for real-time hardware co-simulation of edge detection in image processing using Prewitt edge operator," 2014 International Con-ference on Electronics, Information and Commu-nications (ICEIC), Kota Kinabalu, Malaysia, 2014, pp. 1-2, doi: 10.1109/ELINFOCOM.2014.6914387.
25. [25] W. Gao, X. Zhang, L. Yang, and H. Liu, "An im-proved Sobel edge detection," In International Conference on Computer Science and Infor-mation Technology, vol. 5, pp. 67– 71, IEEE, 2010, doi: 10.1109/ICCSIT.2010.5563693.
26. [26] K. Zhang, L. Zhang, K. -M. Lam and D. Zhang, "A Level Set Approach to Image Segmentation With Intensity Inhomogeneity," in IEEE Transactions on Cybernetics, vol. 46, no. 2, pp. 546-557, Feb. 2016, doi: 10.1109/TCYB.2015.2409119.
27. [27] Ming-Hsuan Yang, D. J. Kriegman and N. Ahuja, "Detecting faces in images: a survey," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 1, pp. 34-58, Jan. 2002, doi: 10.1109/34.982883.
28. [28] W. Wang, C. Wei, W. Yang and J. Liu, "GLAD-Net: Low-light enhancement network with global awareness", Proc. 13th IEEE Int. Conf. Autom. Face Gesture Recognit., pp. 751-755, May 2018.
29. [29] Y. Jiang et al., "EnlightenGAN: Deep Light En-hancement Without Paired Supervision," in IEEE Transactions on Image Processing, vol. 30, pp. 2340-2349, 2021, doi: 10.1109/TIP.2021.3051462.
30. [30] M. Abdullah-Al-Wadud, M. H. Kabir, M. A. A. Dewan and O. Chae, "A dynamic histogram equalization for image contrast enhance-ment",IEEE Trans. Consum. Electron., vol. 53, no. 2, pp. 593-600, May 2007.
31. [31] T. Celik and T. Tjahjadi, "Contextual and Varia-tional Contrast Enhancement," in IEEE Transac-tions on Image Processing, vol. 20, no. 12, pp. 3431-3441, Dec. 2011, doi: 10.1109/TIP.2011.2157513.
32. [32] C. Lee, C. Lee and C. -S. Kim, "Contrast En-hancement Based on Layered Difference Repre-sentation of 2D Histograms," in IEEE Transac-tions on Image Processing, vol. 22, no. 12, pp. 5372-5384, Dec. 2013, doi: 10.1109/TIP.2013.2284059.
33. [33] X. Guo, Y. Li and H. Ling, "LIME: Low-Light Im-age Enhancement via Illumination Map Estima-tion," in IEEE Transactions on Image Processing, vol. 26, no. 2, pp. 982-993, Feb. 2017, doi: 10.1109/TIP.2016.2639450.
34. [34] X. Ren, W. Yang, W. -H. Cheng and J. Liu, "LR3M: Robust Low-Light Enhancement via Low-Rank Regularized Retinex Model," in IEEE Transactions on Image Processing, vol. 29, pp. 5862-5876, 2020, doi: 10.1109/TIP.2020.2984098.
35. [35] J. Xu et al., "STAR: A Structure and Texture Aware Retinex Model," in IEEE Transactions on Image Processing, vol. 29, pp. 5022-5037, 2020, doi: 10.1109/TIP.2020.2974060.
36. [36] Li, W., Xiong, B., Ou, Q., Long, X., Zhu, J., Chen, J., and Wen, S. Zero-Shot Enhancement of Low-Light Image Based on Retinex Decomposition. arXiv preprint arXiv:2311.02995, 2023.
37. [37] M.-T. Duong, S. Lee and M.-C. Hong, "DMT-Net: Deep Multiple Networks for Low-Light Image Enhancement Based on Retinex Model," in IEEE Access, vol. 11, pp. 132147-132161, 2023, doi: 10.1109/ACCESS.2023.3336411.
38. [38] J. Wen, C. Wu, T. Zhang, Y. Yu and P. Swierczyn-ski, "Self-Reference Deep Adaptive Curve Esti-mation for Low-Light Image Enhancement," arXiv preprint arXiv:2308.08197, 2023.
39. [39] J. Hai, Z. Xuan, R. Yang, Y. Hao, F. Zou, F. Lin and S. Han, "R2rnet: Low-light image enhance-ment via real-low to real-normal network, " Jour-nal of Visual Communication and Image Repre-sentation, vol.90, February 2023.
40. [40] M. Zhu, P. Pan, W. Chen and Y. Yang, "EEMEFN: Low-light image enhancement via edge-enhanced multi-exposure fusion network", Proc. AAAI Conf. Artif. Intell., pp. 13 106-13 113, 2020.
41. [41] Y. Fu, Y. Hong, L. Chen and S. You, "LE-GAN: Unsupervised low-light image enhancement net-work using attention module and identity invari-ant loss, " Knowledge-Based Systems, vol.240, 2022.
42. [42] H. Shakibania, S. Raoufı, H. Khotanlou, "CDAN: Convolutional Dense Attention-guided Network for Low-light Image Enhancement," arXiv pre-print arXiv:2308.12902, 2023.
43. [43] Y. Zhang, X. Di, J. Wu, R. FU, Y. Li, Y. Wang, Y. Xu, G. YANG, and C. Wang, “A fast and light-weight network for low-light image enhance-ment,” arXiv preprint arXiv:2304.02978, 2023.
44. [44] S. Xie and Z. Tu, "Holistically-nested edge detec-tion", Proc. IEEE Int. Conf. Comput. Vis. (ICCV), pp. 1395-1403, Dec. 2015.
45. [45] J. Canny, "A Computational Approach to Edge Detection," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-8, no. 6, pp. 679-698, Nov. 1986, doi: 10.1109/TPAMI.1986.4767851.
46. [46] L. G. Roberts, “Machine perception of 3-D solids, optical and electro-optical information pro-cessing”, MIT Press, 1965.
47. [47] J. M. Prewitt, "Object enhancement and extrac-tion", Picture Process. Psychopictorics, vol. 10, no. 1, pp. 15-19, 1970.
48. [48] I.Sobel, Camera models and machine perception, 1970.
49. [49] X.-X. Yin, L. Sun, Y. Fu, R. Lu and Y. Zhang, "UNet-based medical image segmentation", J. Healthcare Eng., vol. 2022, pp. 1-16, Apr. 2022.
50. [50] O. Ronneberger, P. Fischer and T. Brox, "U-Net: Convolutional networks for biomedical image segmentation", Proc. 18th Int. Conf. Med. Image Comput. Comput. -Assist. Interv., pp. 234-241, 2015.
51. [51] Y. Zhang, J. Zhang and X. Guo, "Kindling the darkness: A practical low-light image enhancer" in arXiv:1905.04161, 2019, [online] Available: http://arxiv.org/abs/1905.04161.
52. [52] Z. Jiang et al., "A switched view of retinex: Deep self-regularized low-light image enhancement", Neurocomputing, vol. 454, pp. 361-372, 2021.
53. [53] C. C. Lim, Y. P. Loh, and L. K. Wong, "LAU-Net: A low light image enhancer with attention and resizing mechanisms, " Signal Processing: Image Communication, vol.115, 2023.
54. [54] X. Liang, X Chen, K. Ren, X. Miao, Z. Chen, and Y. Jin "Low-light image enhancement via adap-tive frequency decomposition network," Scien-tific Reports, 2023.
55. [55] S. W. Zamir et al., "Learning enriched features for real image restoration and enhancement", Proc. Eur. Conf. Comput. Vis., pp. 492-511, 2020.
56. [56] A. Horé and D. Ziou, "Image Quality Metrics: PSNR vs. SSIM," 2010 20th International Confer-ence on Pattern Recognition, Istanbul, Turkey, 2010, pp. 2366-2369, doi: 10.1109/ICPR.2010.579.
57. [57] U. Sara, M. Akter and M. S. Uddin, "Image quality assessment through FSIM SSIM MSE and PSNR—A comparative study", J. Comput. Com-mun., vol. 7, no. 3, pp. 8-18, 2019.
58. [58] A. Mittal, A. K. Moorthy and A. C. Bovik, "No-Reference Image Quality Assessment in the Spa-tial Domain," in IEEE Transactions on Image Pro-cessing, vol. 21, no. 12, pp. 4695-4708, Dec. 2012, doi: 10.1109/TIP.2012.2214050.
59. [59] A. Mittal, R. Soundararajan and A. C. Bovik, "Making a “Completely Blind” Image Quality Analyzer," in IEEE Signal Processing Letters, vol. 20, no. 3, pp. 209-212, March 2013, doi: 10.1109/LSP.2012.2227726.
60. [60] C. Wei, W. Wang and W. Yang, "Deep Retinex decomposition for low-light enhancement", Proc. Brit. Mach. Vis. Conf., pp. 1-12, 2018.
61. [61] D. Dang-Nguyen, C. Pasquini, V. Conotter and G. Boato, "RAISE: A raw images dataset for digital image forensics", Proc. 6th ACM Multimedia Syst. Conf., pp. 219-1224, 2015.
62. [62] Y. Zhang, X. Di, B. Zhang and C. Wang, "Self-supervised image enhancement network: Train-ing with low light images only", arXiv:2002.11300, 2020.