Research Article
BibTex RIS Cite

Estimation of Air Light With Deep Learning for a Near Real-Time Image Dehazing System

Year 2023, , 604 - 612, 15.10.2023

Yücel Çimtay

https://doi.org/10.34248/bsengineering.1349643

Abstract

Haze which can be created by natural or synthetic factors, degrades the visual quality and human sight distance. Visible objects become invisible or scarcely visible. The physics of the degrading function due to haze has been modelled by Atmospheric Light Scattering (ALS) Model. Therefore, from a single hazy image, by using proper methods, it is possible to recover the original scene. In dehazing methods, which solve the ALS function, there are basically two steps: First one is the estimation of the air light present at the time of the image capturing and the second one is the estimation of transmission of the corresponding scene. One of the most effective method which is used for air light estimation is QuadTree decomposition. For this method, tests show that the most amount of the dehazing time is consumed to estimate the air light. For the case of High Definition (HD) imagery, the estimation of air light consumes huge time. Therefore, it cannot be possible to achieve a real-time or near real-time dehazing on traditional hardware. In this study, a novel convolutional neural network model is developed to estimate the air light directly from the hazy image quickly. The estimated air light then is used with Atmospheric Light Scattering model to handle the recovered image. Results show that the time cost is reduced by 56.0% and 65% for image resolutions of (640x480) and (1920x1080) compared to the QuadTree Decomposition method used in ALS based dehazing methods, without losing the visual quality of the dehazed image.

Keywords

Depth map Image quality Distortion Image blur Real-time systems

Project Number

122E333

References

  • Al-Sammaraie, MF. 2015. Contrast enhancement of roads images with foggy scenes based on histogram equalization. Proceedings of 10th International Conference on Computer Science & Education (ICCSE), June 22-24, Cambridge, UK, pp: 95-101.
  • Ancuti C, Ancuti CO, Vleeschouwer CD. 2016. D-HAZY: A dataset to evaluate quantitatively dehazing algorithms journal. Proceedings of IEEE International Conference on Image Processing ICIP, September 25-28, Arizona, US, pp: 2226-2230.
  • Ancuti CO, Ancuti C, Sbert M, Timofte R. 2019. Dense haze: A benchmark for image dehazing with dense-haze and haze-free images. IEEE International Conference on Image Processing (ICIP), September 22-25, Taipei, Taiwan, pp: 1014-1018.
  • Ancuti CO, Ancuti C, Timofte R, Gool LV, Zhang L, Yang MH. 2019. NTIRE 2019 Image Dehazing Challenge Report. Proceedings of IEEE CVPR Workshop, June 16-17, Long Beach, CA, US, pp: 2241-2253.
  • Boyi L, Wenqi R, Dengpan F, Dacheng T, Feng D, Wenjun Z, Zhangyang W. 2017. Benchmarking Single-Image Dehazing and Beyond. IEEE Transact Image Proces, 28(1): 492-505.
  • C6748 pure DSP device data sheet. URL: https://www.ti.com/lit/ml/sprt6 33/sprt6 33.pdf?ts=15976 90676 332&ref_url=https %253A%252F%252Fw ww.googl e.com%252F (access date: June, 9, 2023).
  • Cai B, Xu X, Jia K, Qing C, Tao D. 2016. DehazeNet: An end-to-end system for single image haze removal. IEEE Transact Image Proces, 25(11): 5187-5198.
  • Chen C, Do MN, Wang J. 2016. Robust image and video dehazing with visual artifact suppression via gradient residual minimization. In: Proceedings of European Conference on Computer Vision, October 8-16, Amsterdam, Netherlands, pp: 576–591.
  • Cheng K, Yu Y, Zhou H, Zhou D. 2020. GPU fast restoration of nonuniform illumination images. J Real-Time Image Proces, 18(1): 75-83.
  • Cimtay Y. 2020. Towards real-time image dehazing on android operating system. Commun, Series A2-A3: Physical Sci Eng, 62(2): 177-188.
  • Cimtay Y. 2021. Smart and real-time image dehazing on mobile devices. J Real-Time Image Proces. 18: 2063-2072.
  • El Khoury J, Jean-Baptiste T, Alamin M. 2018. A database with reference for image dehazing evaluation. J Imag Sci Techn, 62(1): 010503-1-010503-13.
  • Guo C, Yan Q, Anwar S, Cong R, Ren W, Li C. 2022. Image dehazing transformer with transmission-aware 3D position embedding. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 19-24, New Orleans, LA, US, pp: 5802-5810.
  • Hao W, He M, Ge H, Wang C, Qing-Wei G. 2011. Retinex-like method for image enhancement in poor visibility conditions. Procedia Eng, 15(1): 2798-2803.
  • Haouassi S, Di W. 2020. Image dehazing based on (CMTnet) cascaded multi-scale convolutional neural networks and efficient light estimation algorithm. Appl Sci, 10(3): 1-21.
  • Hernandez-Beltran J, Diaz-Ramirez V, Juarez-Salazar, R. 2019. Realtime image dehazing using genetic programming. J Opt Photonics Inf Proces, 11136: 222-230.
  • Kaiming H, Jian S, Xiaoou, T. 2011. Single image haze removal using dark channel prior. IEEE Transact Pattern Analy Machine Intell, 33(12): 2341-2353.
  • Khatun A, Haque M, Basri R, Uddin M. 2020. Single image dehazing: an analysis on generative adversarial network. J Comput Commun, 8(4): 127-137.
  • Kim JH, Jang WD, Sim JY, Kim CS. 2013. Optimized contrast enhancement for real-time image and video dehazing. J Visual Commun Image Represent, 24(3): 410-425.
  • Kim JH, Sim JY, Kim CS. 2011. Single image dehazing based on contrast enhancement. Proceedings of IEEE International Conference Acoustics, Speech and Signal Processing(ICASSP), May 22-27, Prague, Czech Republic, pp: 1273-1276.
  • Kopf J, Neubert B, Chen B, Cohen M, Cohen-Or D, Deussen O, Uyttendaele M, Lischinski D. 2008. Deep photo: modelbased photograph enhancement and viewing. ACM Trans Graph, 27(5): 1-10.
  • Li B, Peng X, Wang Z, Xu J, Feng D. 2017. Aod-net: All-in-one dehazing network. Proceedings of the IEEE international conference on computer vision, October 22-29, Venice, Italy, pp: 4770-4778.
  • Li C, Guo J, Porikli F, Fu H, Pang Y. 2018. A cascaded convolutional neural network for single image dehazing. IEEE Access, 6(1): 24877-24887.
  • Li H, Zhang Y, Liu J, Ma Y. 2023. GTMNet: a vision transformer with guided transmission map for single remote sensing image dehazing. Scient Rep, 13(1): 9222.
  • Li J, Li G, Fan H. 2018. Image dehazing using residual-based deep CNN. IEEE Access, 6(1): 26831-26842.
  • Lu J, Dong C. 2019. DSP-based image real-time dehazing optimization for improved dark-channel prior algorithm. J Real-Time Image Proces, 17(1): 1675-1684.
  • Meng X, Feng Y, Su Z, Zhou F. 2022. Unsupervised domain adaptation image dehazing with contrastive nearest-farthest subspace distance. Proceedings of IEEE International Conference on Multimedia and Expo (ICME), July 18-22, Taipei, Taiwan, pp: 1-6.
  • Park D, Park, H, Han, DK, Ko, H. 2014. Single image dehazing with image entropy and information fidelity. IEEE International Conference on Image Processing (ICIP), October 27-30, Paris, France, pp: 4037-4041
  • Rashid H, Zafar N, Javed Iqbal M, Dawood H. 2019. Single Image Dehazing using CNN. Procedia Comput Sci, 147(1): 124-130.
  • Ren W, Zhou L, Chen J. 2022. Unsupervised single image dehazing with generative adversarial network. Multimed Syst, 2022: 1-11.
  • Shao Y, Li L, Ren W, Gao C, Sang N. 2020. Domain Adaptation for Image Dehazing. Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 16-18, Seattle, WA, US, pp: 2805-2814.
  • Shu-Juan P, Zhang H, Liu X, Wentao F, Zhong B, Ji-Xiang D. 2021. Real-time video dehazing via incremental transmission learning and spatial-temporally coherent regularization. Neurocomputing, 458: 602-614.
  • Singh A, Bhave A, Prasad DK. 2020. Single image dehazing for a variety of haze scenarios using back projected pyramid network. Proceedings of ECCV 2020 Workshops, August 23–28, Glasgow, UK, pp: 166-181.
  • Tan K, Oakley JP. 2001. Physics-based approach to color image enhancement in poor visibility conditions. J Optical Soc America, 18(10): 2460-2467.
  • Tran LA, Moon S, Park DC. 2022. A novel encoder-decoder network with guided transmission map for single image dehazing. Procedia Comput Sci, 204(1): 682-689.
  • Nguyen VT, Vien AG, Lee C. 2022. Real-time image and video dehazing based on multiscale guided filtering. Multimed Tools Appl, 8(1): 36567-36584.
  • Vazquez-Corral J, Galdran A, Cyriac P, Bertalmio M. 2020. A fast image dehazing method that does not introduce color artifacts. J Real-Time Image Proces, 17(1): 607-622
  • VisualKeras Library. 2023. URL: https://pypi.org/project/visualkeras/ (access date: June, 8, 2023).
  • Wang W, Yuan X. 2017. Recent advances in image dehazing. IEEE/CAA J Automatica Sinica, 4(3): 410-436.
  • Yang J, Jiang B, Lv Z, Jiang N. 2017. A real-time image dehazing method considering dark channel and statistics features. J Real-Time Image Proces, 13(1): 479-490.
  • Yuanyuan S, Yue M. 2015. Single image dehazing on mobile device based on GPU rendering technology. J Roboti Network Artificial Life, 2(2): 85-88.
  • Yuda S, Zhuqing H, Hui Q, Xin D. 2023. Vision transformers for single image dehazing. IEEE Transact Image Proces, 32(1): 1927-1941.
  • Zhang H, Patel VM. 2018. Densely connected pyramid dehazing network. Proceedings of the IEEE conference on computer vision and pattern recognition, June 18-23, Salt Lake City, UT, US, pp: 3194-3203.
  • Zhu Q, Mai J, Shao L. 2015. A fast single image haze removal algorithm using color attenuation prior. IEEE Transact Image Proces, 24(11): 3522-3533.

Estimation of Air Light With Deep Learning for a Near Real-Time Image Dehazing System

Year 2023, , 604 - 612, 15.10.2023

Yücel Çimtay

https://doi.org/10.34248/bsengineering.1349643

Abstract

Haze which can be created by natural or synthetic factors, degrades the visual quality and human sight distance. Visible objects become invisible or scarcely visible. The physics of the degrading function due to haze has been modelled by Atmospheric Light Scattering (ALS) Model. Therefore, from a single hazy image, by using proper methods, it is possible to recover the original scene. In dehazing methods, which solve the ALS function, there are basically two steps: First one is the estimation of the air light present at the time of the image capturing and the second one is the estimation of transmission of the corresponding scene. One of the most effective method which is used for air light estimation is QuadTree decomposition. For this method, tests show that the most amount of the dehazing time is consumed to estimate the air light. For the case of High Definition (HD) imagery, the estimation of air light consumes huge time. Therefore, it cannot be possible to achieve a real-time or near real-time dehazing on traditional hardware. In this study, a novel convolutional neural network model is developed to estimate the air light directly from the hazy image quickly. The estimated air light then is used with Atmospheric Light Scattering model to handle the recovered image. Results show that the time cost is reduced by 56.0% and 65% for image resolutions of (640x480) and (1920x1080) compared to the QuadTree Decomposition method used in ALS based dehazing methods, without losing the visual quality of the dehazed image.

Keywords

Depth map Image quality Distortion Image blur Real-time systems

Supporting Institution

Türkiye Bilimsel ve Teknolojik Araştırma Kurumu (TÜBİTAK)

Project Number

122E333

Thanks

This study is supported by The Scientific and Technological Research Council of Turkey (TUBITAK) within the Project number: 122E333.

References

  • Al-Sammaraie, MF. 2015. Contrast enhancement of roads images with foggy scenes based on histogram equalization. Proceedings of 10th International Conference on Computer Science & Education (ICCSE), June 22-24, Cambridge, UK, pp: 95-101.
  • Ancuti C, Ancuti CO, Vleeschouwer CD. 2016. D-HAZY: A dataset to evaluate quantitatively dehazing algorithms journal. Proceedings of IEEE International Conference on Image Processing ICIP, September 25-28, Arizona, US, pp: 2226-2230.
  • Ancuti CO, Ancuti C, Sbert M, Timofte R. 2019. Dense haze: A benchmark for image dehazing with dense-haze and haze-free images. IEEE International Conference on Image Processing (ICIP), September 22-25, Taipei, Taiwan, pp: 1014-1018.
  • Ancuti CO, Ancuti C, Timofte R, Gool LV, Zhang L, Yang MH. 2019. NTIRE 2019 Image Dehazing Challenge Report. Proceedings of IEEE CVPR Workshop, June 16-17, Long Beach, CA, US, pp: 2241-2253.
  • Boyi L, Wenqi R, Dengpan F, Dacheng T, Feng D, Wenjun Z, Zhangyang W. 2017. Benchmarking Single-Image Dehazing and Beyond. IEEE Transact Image Proces, 28(1): 492-505.
  • C6748 pure DSP device data sheet. URL: https://www.ti.com/lit/ml/sprt6 33/sprt6 33.pdf?ts=15976 90676 332&ref_url=https %253A%252F%252Fw ww.googl e.com%252F (access date: June, 9, 2023).
  • Cai B, Xu X, Jia K, Qing C, Tao D. 2016. DehazeNet: An end-to-end system for single image haze removal. IEEE Transact Image Proces, 25(11): 5187-5198.
  • Chen C, Do MN, Wang J. 2016. Robust image and video dehazing with visual artifact suppression via gradient residual minimization. In: Proceedings of European Conference on Computer Vision, October 8-16, Amsterdam, Netherlands, pp: 576–591.
  • Cheng K, Yu Y, Zhou H, Zhou D. 2020. GPU fast restoration of nonuniform illumination images. J Real-Time Image Proces, 18(1): 75-83.
  • Cimtay Y. 2020. Towards real-time image dehazing on android operating system. Commun, Series A2-A3: Physical Sci Eng, 62(2): 177-188.
  • Cimtay Y. 2021. Smart and real-time image dehazing on mobile devices. J Real-Time Image Proces. 18: 2063-2072.
  • El Khoury J, Jean-Baptiste T, Alamin M. 2018. A database with reference for image dehazing evaluation. J Imag Sci Techn, 62(1): 010503-1-010503-13.
  • Guo C, Yan Q, Anwar S, Cong R, Ren W, Li C. 2022. Image dehazing transformer with transmission-aware 3D position embedding. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 19-24, New Orleans, LA, US, pp: 5802-5810.
  • Hao W, He M, Ge H, Wang C, Qing-Wei G. 2011. Retinex-like method for image enhancement in poor visibility conditions. Procedia Eng, 15(1): 2798-2803.
  • Haouassi S, Di W. 2020. Image dehazing based on (CMTnet) cascaded multi-scale convolutional neural networks and efficient light estimation algorithm. Appl Sci, 10(3): 1-21.
  • Hernandez-Beltran J, Diaz-Ramirez V, Juarez-Salazar, R. 2019. Realtime image dehazing using genetic programming. J Opt Photonics Inf Proces, 11136: 222-230.
  • Kaiming H, Jian S, Xiaoou, T. 2011. Single image haze removal using dark channel prior. IEEE Transact Pattern Analy Machine Intell, 33(12): 2341-2353.
  • Khatun A, Haque M, Basri R, Uddin M. 2020. Single image dehazing: an analysis on generative adversarial network. J Comput Commun, 8(4): 127-137.
  • Kim JH, Jang WD, Sim JY, Kim CS. 2013. Optimized contrast enhancement for real-time image and video dehazing. J Visual Commun Image Represent, 24(3): 410-425.
  • Kim JH, Sim JY, Kim CS. 2011. Single image dehazing based on contrast enhancement. Proceedings of IEEE International Conference Acoustics, Speech and Signal Processing(ICASSP), May 22-27, Prague, Czech Republic, pp: 1273-1276.
  • Kopf J, Neubert B, Chen B, Cohen M, Cohen-Or D, Deussen O, Uyttendaele M, Lischinski D. 2008. Deep photo: modelbased photograph enhancement and viewing. ACM Trans Graph, 27(5): 1-10.
  • Li B, Peng X, Wang Z, Xu J, Feng D. 2017. Aod-net: All-in-one dehazing network. Proceedings of the IEEE international conference on computer vision, October 22-29, Venice, Italy, pp: 4770-4778.
  • Li C, Guo J, Porikli F, Fu H, Pang Y. 2018. A cascaded convolutional neural network for single image dehazing. IEEE Access, 6(1): 24877-24887.
  • Li H, Zhang Y, Liu J, Ma Y. 2023. GTMNet: a vision transformer with guided transmission map for single remote sensing image dehazing. Scient Rep, 13(1): 9222.
  • Li J, Li G, Fan H. 2018. Image dehazing using residual-based deep CNN. IEEE Access, 6(1): 26831-26842.
  • Lu J, Dong C. 2019. DSP-based image real-time dehazing optimization for improved dark-channel prior algorithm. J Real-Time Image Proces, 17(1): 1675-1684.
  • Meng X, Feng Y, Su Z, Zhou F. 2022. Unsupervised domain adaptation image dehazing with contrastive nearest-farthest subspace distance. Proceedings of IEEE International Conference on Multimedia and Expo (ICME), July 18-22, Taipei, Taiwan, pp: 1-6.
  • Park D, Park, H, Han, DK, Ko, H. 2014. Single image dehazing with image entropy and information fidelity. IEEE International Conference on Image Processing (ICIP), October 27-30, Paris, France, pp: 4037-4041
  • Rashid H, Zafar N, Javed Iqbal M, Dawood H. 2019. Single Image Dehazing using CNN. Procedia Comput Sci, 147(1): 124-130.
  • Ren W, Zhou L, Chen J. 2022. Unsupervised single image dehazing with generative adversarial network. Multimed Syst, 2022: 1-11.
  • Shao Y, Li L, Ren W, Gao C, Sang N. 2020. Domain Adaptation for Image Dehazing. Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 16-18, Seattle, WA, US, pp: 2805-2814.
  • Shu-Juan P, Zhang H, Liu X, Wentao F, Zhong B, Ji-Xiang D. 2021. Real-time video dehazing via incremental transmission learning and spatial-temporally coherent regularization. Neurocomputing, 458: 602-614.
  • Singh A, Bhave A, Prasad DK. 2020. Single image dehazing for a variety of haze scenarios using back projected pyramid network. Proceedings of ECCV 2020 Workshops, August 23–28, Glasgow, UK, pp: 166-181.
  • Tan K, Oakley JP. 2001. Physics-based approach to color image enhancement in poor visibility conditions. J Optical Soc America, 18(10): 2460-2467.
  • Tran LA, Moon S, Park DC. 2022. A novel encoder-decoder network with guided transmission map for single image dehazing. Procedia Comput Sci, 204(1): 682-689.
  • Nguyen VT, Vien AG, Lee C. 2022. Real-time image and video dehazing based on multiscale guided filtering. Multimed Tools Appl, 8(1): 36567-36584.
  • Vazquez-Corral J, Galdran A, Cyriac P, Bertalmio M. 2020. A fast image dehazing method that does not introduce color artifacts. J Real-Time Image Proces, 17(1): 607-622
  • VisualKeras Library. 2023. URL: https://pypi.org/project/visualkeras/ (access date: June, 8, 2023).
  • Wang W, Yuan X. 2017. Recent advances in image dehazing. IEEE/CAA J Automatica Sinica, 4(3): 410-436.
  • Yang J, Jiang B, Lv Z, Jiang N. 2017. A real-time image dehazing method considering dark channel and statistics features. J Real-Time Image Proces, 13(1): 479-490.
  • Yuanyuan S, Yue M. 2015. Single image dehazing on mobile device based on GPU rendering technology. J Roboti Network Artificial Life, 2(2): 85-88.
  • Yuda S, Zhuqing H, Hui Q, Xin D. 2023. Vision transformers for single image dehazing. IEEE Transact Image Proces, 32(1): 1927-1941.
  • Zhang H, Patel VM. 2018. Densely connected pyramid dehazing network. Proceedings of the IEEE conference on computer vision and pattern recognition, June 18-23, Salt Lake City, UT, US, pp: 3194-3203.
  • Zhu Q, Mai J, Shao L. 2015. A fast single image haze removal algorithm using color attenuation prior. IEEE Transact Image Proces, 24(11): 3522-3533.
There are 44 citations in total.

Details

Primary Language English
Subjects Circuits and Systems, Electrical Engineering (Other), Signal Processing
Journal Section Research Articles
Authors

Yücel Çimtay 0000-0002-2254-9307

Project Number 122E333
Early Pub Date October 5, 2023
Publication Date October 15, 2023
Submission Date August 25, 2023
Acceptance Date September 30, 2023
Published in Issue Year 2023

Cite

APA Çimtay, Y. (2023). Estimation of Air Light With Deep Learning for a Near Real-Time Image Dehazing System. Black Sea Journal of Engineering and Science, 6(4), 604-612. https://doi.org/10.34248/bsengineering.1349643
AMA Çimtay Y. Estimation of Air Light With Deep Learning for a Near Real-Time Image Dehazing System. BSJ Eng. Sci. October 2023;6(4):604-612. doi:10.34248/bsengineering.1349643
Chicago Çimtay, Yücel. “Estimation of Air Light With Deep Learning for a Near Real-Time Image Dehazing System”. Black Sea Journal of Engineering and Science 6, no. 4 (October 2023): 604-12. https://doi.org/10.34248/bsengineering.1349643.
EndNote Çimtay Y (October 1, 2023) Estimation of Air Light With Deep Learning for a Near Real-Time Image Dehazing System. Black Sea Journal of Engineering and Science 6 4 604–612.
IEEE Y. Çimtay, “Estimation of Air Light With Deep Learning for a Near Real-Time Image Dehazing System”, BSJ Eng. Sci., vol. 6, no. 4, pp. 604–612, 2023, doi: 10.34248/bsengineering.1349643.
ISNAD Çimtay, Yücel. “Estimation of Air Light With Deep Learning for a Near Real-Time Image Dehazing System”. Black Sea Journal of Engineering and Science 6/4 (October 2023), 604-612. https://doi.org/10.34248/bsengineering.1349643.
JAMA Çimtay Y. Estimation of Air Light With Deep Learning for a Near Real-Time Image Dehazing System. BSJ Eng. Sci. 2023;6:604–612.
MLA Çimtay, Yücel. “Estimation of Air Light With Deep Learning for a Near Real-Time Image Dehazing System”. Black Sea Journal of Engineering and Science, vol. 6, no. 4, 2023, pp. 604-12, doi:10.34248/bsengineering.1349643.
Vancouver Çimtay Y. Estimation of Air Light With Deep Learning for a Near Real-Time Image Dehazing System. BSJ Eng. Sci. 2023;6(4):604-12.

                                                24890