A generative adversarial network incorporating dark channel prior loss used for single image defogging

Cheng Deqiang; You Yangyang; Kou Qiqi; Xu Jinyang

doi:10.12086/oee.2022.210448

Article navigation > Opto-Electronic Engineering > 2022 Vol. 49 > No. 7 > 210448

Next Article Previous Article

Cheng D Q, You Y Y, Kou Q Q, et al. A generative adversarial network incorporating dark channel prior loss used for single image defogging[J]. Opto-Electron Eng, 2022, 49(7): 210448. doi: 10.12086/oee.2022.210448

Citation:

Cheng D Q, You Y Y, Kou Q Q, et al. A generative adversarial network incorporating dark channel prior loss used for single image defogging[J]. Opto-Electron Eng, 2022, 49(7): 210448. doi: 10.12086/oee.2022.210448

A generative adversarial network incorporating dark channel prior loss used for single image defogging

1.
Engineering Research Center of Underground Space Intelligent Control, Ministry of Education, Xuzhou, Jiangsu 221000, China
2.
School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, Jiangsu 221000, China
3.
School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221000, China

Fund Project: National Natural Science Foundation of China (51774281)

More Information

^*Corresponding author: 1120074179@qq.com

Received Date 19 January 2022

Revised Date 01 April 2022

Published Date 25 July 2022

Abstract

Abstract

Single image defogging using generative adversarial networks (GAN) relies on annotated datasets, which is easy to cause over-fitting of ground truth, and usually performs not well on natural images. To solve this problem, this paper designed a GAN network incorporating dark channel prior loss to defogging single image. This prior loss can influence the model prediction results in network training and correct the sparsity and skewness of the dark channel feature map. At the same time, it can definitely improve the actual defogging effect and prevent the model from over-fitting problem. In addition, in order to solve the problem that the extraction method of traditional dark channel feature has non-convex function and is difficult to be embedded into network training, this paper introduces a new extraction strategy which compresses pixel values instead of minimum filtering. The implementation function of this strategy is a convex function, which is conducive to embedded network training and enhances the overall robustness of the algorithm. Moreover, this strategy does not need to set a fixed scale to extract the dark channel feature map, and has good adaptability to images with different resolutions. Experimental results show that the proposed algorithm performs better on real images and synthetic test-sets like SOTS when compared with other sota algorithms.
- generative adversarial network /
- prior loss /
- sparsity /
- skewness

FullText(HTML)

References

[1]	吕晨, 程德强, 寇旗旗, 等. 基于YOLOv3和ASMS的目标跟踪算法[J]. 光电工程, 2021, 48(2): 200175. Google Scholar Lv C, Cheng D Q, Kou Q Q, et al. Target tracking algorithm based on YOLOv3 and ASMS[J]. Opto-Electron Eng, 2021, 48(2): 200175. Google Scholar
[2]	寇旗旗, 程德强, 于文洁, 等. 融合CLBP和局部几何特征的纹理目标分类[J]. 光电工程, 2019, 46(11): 180604. Google Scholar Kou Q Q, Cheng D Q, Yu W J, et al. Texture target classification with CLBP and local geometric features[J]. Opto-Electron Eng, 2019, 46(11): 180604. Google Scholar
[3]	江曼, 张皓翔, 程德强, 等. 融合HSV与方向梯度特征的多尺度图像检索[J]. 光电工程, 2021, 48(11): 210310. Google Scholar Jiang M, Zhang H X, Cheng D Q, et al. Multi-scale image retrieval based on HSV and directional gradient features[J]. Opto-Electron Eng, 2021, 48(11): 210310. Google Scholar
[4]	Kim T K, Paik J K, Kang B S. Contrast enhancement system using spatially adaptive histogram equalization with temporal filtering[J]. IEEE Trans Consum Electron, 1998, 44(1): 82−87. doi: 10.1109/30.663733 CrossRef Google Scholar
[5]	Zhou J C, Zhang D H, Zou P Y, et al. Retinex-based laplacian pyramid method for image defogging[J]. IEEE Access, 2019, 7: 122459−122472. doi: 10.1109/ACCESS.2019.2934981 CrossRef Google Scholar
[6]	Nayar S K, Narasimhan S G. Vision in bad weather[C]//Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, 1999: 820–827. Google Scholar
[7]	He K M, Sun J, Tang X O. Single image haze removal using dark channel prior[J]. IEEE Trans Pattern Anal Mach Intell, 2011, 33(12): 2341−2353. doi: 10.1109/TPAMI.2010.168 CrossRef Google Scholar
[8]	He K M, Sun J, Tang X O. Guided image filtering[J]. IEEE Trans Pattern Anal Mach Intell, 2013, 35(6): 1397−1409. doi: 10.1109/TPAMI.2012.213 CrossRef Google Scholar
[9]	Meng G F, Wang Y, Duan J Y, et al. Efficient image dehazing with boundary constraint and contextual regularization[C]//Proceedings of 2013 IEEE International Conference on Computer Vision, Sydney, 2013: 617–624. Google Scholar
[10]	Zhu Q S, Mai J M, Shao L. A fast single image haze removal algorithm using color attenuation prior[J]. IEEE Trans Image Process, 2015, 24(11): 3522−3533. doi: 10.1109/TIP.2015.2446191 CrossRef Google Scholar
[11]	王新, 张旭东, 张骏, 等. 结合光场多线索和大气散射模型的去雾算法[J]. 光电工程, 2020, 47(9): 190634. Google Scholar Wang X, Zhang X D, Zhang J, et al. Image dehazing algorithm by combining light field multi-cues and atmospheric scattering model[J]. Opto-Electron Eng, 2020, 47(9): 190634. Google Scholar
[12]	Zhao D, Xu L, Yan Y H, et al. Multi-scale Optimal Fusion model for single image dehazing[J]. Signal Process Image Commun, 2019, 74: 253−265. doi: 10.1016/j.image.2019.02.004 CrossRef Google Scholar
[13]	Yang Y, Wang Z W. Haze removal: push DCP at the edge[J]. IEEE Signal Process Lett, 2020, 27: 1405−1409. doi: 10.1109/LSP.2020.3013741 CrossRef Google Scholar
[14]	Wang S, Chen J Y. Single image dehazing using dark channel fusion and dark channel confidence[C]//Proceedings of 2020 International Conference on Big Data & Artificial Intelligence & Software Engineering, Bangkok, 2020: 439–444. Google Scholar
[15]	Ren W Q, Liu S, Zhang H, et al. Single image dehazing via multi-scale convolutional neural networks[C]//Proceedings of the 14th European Conference on Computer Vision, Amsterdam, 2016: 154–169. Google Scholar
[16]	Cai B L, Xu X M, Jia K, et al. DehazeNet: an end-to-end system for single image haze removal[J]. IEEE Trans Image Process, 2016, 25(11): 5187−5198. doi: 10.1109/TIP.2016.2598681 CrossRef Google Scholar
[17]	Li B Y, Peng X L, Wang Z Y, et al. AOD-Net: all-in-one dehazing network[C]//Proceedings of 2017 IEEE International Conference on Computer Vision, Venice, 2017: 4780–4788. Google Scholar
[18]	Chen D D, He M M, Fan Q N, et al. Gated context aggregation network for image dehazing and deraining[C]//Proceedings of 2019 IEEE Winter Conference on Applications of Computer Vision, Waikoloa, 2019: 1375–1383. Google Scholar
[19]	Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems, Montreal, 2014: 2672–2680. Google Scholar
[20]	Li R D, Pan J S, Li Z H, et al. Single image dehazing via conditional generative adversarial network[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 2018: 8202–8211. Google Scholar
[21]	Johnson J, Alahi A, Fei-Fei L. Perceptual losses for real-time style transfer and super-resolution[C]//Proceedings of the 14th European Conference on Computer Vision, Amsterdam, 2016: 694–711. Google Scholar
[22]	Engin D, Genc A, Ekenel H K. Cycle-dehaze: enhanced CycleGAN for single image dehazing[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, 2018: 939–946. Google Scholar
[23]	Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of 2017 IEEE International Conference on Computer Vision, Venice, 2017: 2242–2251. Google Scholar
[24]	Li L R H, Dong Y L, Ren W Q, et al. Semi-supervised image dehazing[J]. IEEE Trans Image Process, 2020, 29: 2766−2779. doi: 10.1109/TIP.2019.2952690 CrossRef Google Scholar
[25]	Pan J S, Sun D Q, Pfister H, et al. Blind image deblurring using dark channel prior[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 2016: 1628–1636. Google Scholar
[26]	Golts A, Freedman D, Elad M. Unsupervised single image dehazing using dark channel prior loss[J]. IEEE Trans Image Process, 2020, 29: 2692−2701. doi: 10.1109/TIP.2019.2952032 CrossRef Google Scholar
[27]	Shen Z Y, Lai W S, Xu T F, et al. Deep semantic face deblurring[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 2018: 8260–8269. Google Scholar
[28]	Tao X, Gao H Y, Shen X Y, et al. Scale-recurrent network for deep image deblurring[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 2018: 8174–8182. Google Scholar
[29]	Nah S, Kim T H, Lee K M. Deep multi-scale convolutional neural network for dynamic scene deblurring[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, 2017: 257–265. Google Scholar
[30]	Isola P, Zhu J Y, Zhou T H, et al. Image-to-image translation with conditional adversarial networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, 2017: 5967–5976. Google Scholar
[31]	Ledig C, Theis L, Huszár F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, 2017: 105–114. Google Scholar
[32]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 2016: 770–778. Google Scholar
[33]	Li B Y, Ren W Q, Fu D P, et al. Benchmarking single-image dehazing and beyond[J]. IEEE Trans Image Process, 2019, 28(1): 492−505. doi: 10.1109/TIP.2018.2867951 CrossRef Google Scholar
[34]	Silberman N, Hoiem D, Kohli P, et al. Indoor segmentation and support inference from RGBD images[C]//Proceedings of the 12th European Conference on Computer Vision, Florence, 2012. Google Scholar
[35]	Scharstein D, Szeliski R. High-accuracy stereo depth maps using structured light[C]//Proceedings of 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Madison, 2003: I-I. Google Scholar
[36]	Beijing realtime weather photos[EB/OL]. (2022-06). https://www.tour-beijing.com/real_time_weather_photo/. Google Scholar
[37]	Liu F Y, Shen C H, Lin G S, et al. Learning depth from single monocular images using deep convolutional neural fields[J]. IEEE Trans Pattern Anal Mach Intell, 2016, 38(10): 2024−2039. doi: 10.1109/TPAMI.2015.2505283 CrossRef Google Scholar
[38]	Hambarde P, Dudhane A, Patil P W, et al. Depth estimation from single image and semantic prior[C]//Proceedings of 2020 IEEE International Conference on Image Processing, Abu Dhabi, 2020: 1441–1445. Google Scholar
[39]	Zhao S Y, Zhang L, Shen Y, et al. RefineDNet: a weakly supervised refinement framework for single image dehazing[J]. IEEE Trans Image Process, 2021, 30: 3391−3404. doi: 10.1109/TIP.2021.3060873 CrossRef Google Scholar
[40]	Ancuti C, Ancuti C O, De Vleeschouwer C, et al. D-HAZY: a dataset to evaluate quantitatively dehazing algorithms[C]//Proceedings of 2016 IEEE International Conference on Image Processing, Phoenix, 2016: 2226–2230. Google Scholar
[41]	Zhang Y F, Ding L, Sharma G. HazeRD: an outdoor scene dataset and benchmark for single image dehazing[C]//Proceedings of 2017 IEEE International Conference on Image Processing, Beijing, 2017: 3205–3209. Google Scholar
[42]	Zhang L, Shen Y, Li H Y. VSI: a visual saliency-induced index for perceptual image quality assessment[J]. IEEE Trans Image Process, 2014, 23(10): 4270−4281. doi: 10.1109/TIP.2014.2346028 CrossRef Google Scholar
[43]	Zhao S Y, Zhang L, Huang S Y, et al. Dehazing evaluation: real-world benchmark datasets, criteria, and baselines[J]. IEEE Trans Image Process, 2020, 29: 6947−6962. doi: 10.1109/TIP.2020.2995264 CrossRef Google Scholar
[44]	Hautière N, Tarel J P, Aubert D, et al. Blind contrast enhancement assessment by gradient ratioing at visible edges[J]. Image Anal Stereol, 2008, 27(2): 87−95. Google Scholar
[45]	崔光茫, 张克奇, 毛磊, 等. 结合多尺度分解和梯度绝对值算子的显微图像清晰度评价方法[J]. 光电工程, 2019, 46(6): 180531. Google Scholar Cui G M, Zhang K Q, Mao L, et al. Micro-image definition evaluation using multi-scale decomposition and gradient absolute value[J]. Opto-Electron Eng, 2019, 46(6): 180531. Google Scholar

Overview

Overview

In the atmospheric environment, there are many fine particles in the air, which will lead to the absorption or refraction of light and affect the normal radiation of light. In this case, the color, contrast, saturation and detail of the image captured by the camera are often seriously affected. At present, computer vision needs to realize many high-level tasks such as pedestrian recognition, automatic driving, air navigation, remote sensing and telemetry, and these high-level tasks have a high demand for image quality. Therefore, it is of great significance to carry out single image defogging to obtain higher quality images before performing high-level tasks. In recent years, single image defogging using generative adversarial networks(GAN) has become a hot research aspect. However, the traditional GAN algorithms rely on annotated datasets, which is easy to cause over-fitting of ground truth, and usually performs not well on natural images. To solve this problem, this paper designed a GAN network incorporating dark channel prior loss to defogging single image. This prior loss can influence the model prediction results in network training and correct the sparsity and skewness of the dark channel feature map. At the same time, it can definitely improve the actual defogging effect and prevent the model from over-fitting problem. In addition, this paper introduced a new method to obtain dark channel feature map, which compresses pixel values instead of minimum filtering. This method does not need to set fixed scale to extract dark channel feature map, and has good adaptability to images with different resolutions. Moreover, the implementation function of this method is a convex function, which is conducive to embedded network training and enhances the overall robustness of the algorithm. The proposed algorithm is quantitatively analyzed in the comprehensive test set SOTS and the mixed subjective test set HSTS. The peak signal-to-noise ratio (PSNR), structural similarity SSIM and BCEA Metrics are used as the final evaluation indexes. The final result shows that our algorithm can raise PSNR up to 25.35 and raise SSIM up to 0.96 on HSTS test sets. While it comes to SOTS test sets, our method achieves the result of 24.44 PSNR and 0.89 SSIM. When we use BCEA metrics to evaluate our algorithm, we achieve the result of 0.8010 e,1.6672 r and 0.0123 p. In summary, Experimental results show that the proposed algorithm performs well on real images and synthetic test sets compared with other advanced algorithms.