Multi-image blind super-resolution in variational Bayesian framework

Min Lei; Yang Ping; Xu Bing; Liu Yong

doi:10.12086/oee.2019.180149

Article navigation > Opto-Electronic Engineering > 2019 Vol. 46 > No. 6 > 180149

Next Article Previous Article

Min Lei, Yang Ping, Xu Bing, et al. Multi-image blind super-resolution in variational Bayesian framework[J]. Opto-Electronic Engineering, 2019, 46(6): 180149. doi: 10.12086/oee.2019.180149

Citation:

Min Lei, Yang Ping, Xu Bing, et al. Multi-image blind super-resolution in variational Bayesian framework[J]. Opto-Electronic Engineering, 2019, 46(6): 180149. doi: 10.12086/oee.2019.180149

Multi-image blind super-resolution in variational Bayesian framework

1.
Key Laboratory of Adaptive Optics, Chinese Academy of Sciences, Chengdu, Sichuan 610209, China
2.
School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan 610054, China
3.
Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu, Sichuan 610209, China
4.
University of Chinese Academy of Sciences, Beijing 100049, China

Fund Project: Supported by the National Innovation Fund of Chinese Academy of Sciences (CXJJ-16M208), the Preeminent Youth Fund of Sichuan Province, China (2012JQ0012), and the Outstanding Youth Science Fund of Chinese Academy of Sciences

More Information

Corresponding author: E-mail: pingyang2516@163.com

Received Date 03 April 2018

Revised Date 31 May 2018

Published Date 01 June 2019

Abstract

Abstract

Multi-frame image super-resolution method fuses the information of multi-frame low-resolution images to reconstruct high-resolution images. For multi-frame image super-resolution, the accurate estimation of blur kernel of low-resolution image is prerequisite for efficiency information fusion. Traditional super-resolution method usually assumes a known blur kernel and uses the Gaussian filter blur kernel for the enhancement. It also needs to tune the parameters by time-consuming hand-tuning. The proposed method acquires the super-resolution method based on the variational Bayesian method. The high-resolution image, the blur kernel and the model parameters are estimated simultaneously and automatically in the optimal stochastic sense. Experiments and simulations demonstrate that the proposed blind super-resolution method based on blur kernel self-adaptive estimation outperforms the state-of-art super-resolution method in variational Bayesian framework, especially, for the high signal to noise ratio scenarios.
- blind super-resolution /
- resolution enhancement /
- variational Bayesian /
- blur kernel /
- Kullback-Leibler divergence

FullText(HTML)

References

[1]	郭迎辉, 蒲明博, 马晓亮, 等.电磁超构材料色散调控研究进展[J].光电工程, 2017, 44(1): 3-22. doi: 10.3969/j.issn.1003-501X.2017.01.001 CrossRef Google Scholar Guo Y H, Pu M B, Ma X L, et al. Advances of dispersion-engineered metamaterials[J]. Opto-Electronic Engineering, 2017, 44(1): 3-22. doi: 10.3969/j.issn.1003-501X.2017.01.001 CrossRef Google Scholar
[2]	Dong C, Loy C C, He K M, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295-307. doi: 10.1109/TPAMI.2015.2439281 CrossRef Google Scholar
[3]	Ledig C, Theis L, Huszár F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, 2017: 105-114. Google Scholar
[4]	汪荣贵, 汪庆辉, 杨娟, 等.融合特征分类和独立字典训练的超分辨率重建[J].光电工程, 2018, 45(1): 170542. doi: 10.12086/oee.2018.170542 CrossRef Google Scholar Wang R G, Wang Q H, Yang J, et al. Image super-resolution reconstruction by fusing feature classification and independent dictionary training[J]. Opto-Electronic Engineering, 2018, 45(1): 170542. doi: 10.12086/oee.2018.170542 CrossRef Google Scholar
[5]	Nasrollahi K, Moeslund T B. Super-resolution: a comprehensive survey[J]. Machine Vision and Applications, 2014, 25(6): 1423-1468. doi: 10.1007/s00138-014-0623-4 CrossRef Google Scholar
[6]	Yue L W, Shen H F, Li J, et al. Image super-resolution: the techniques, applications, and future[J]. Signal Processing, 2016, 128: 389-408. doi: 10.1016/j.sigpro.2016.05.002 CrossRef Google Scholar
[7]	Tsai R Y, Huang T S. Multi-frame image restoration and registration[M]//Huang T S. Advances in Computer Vision and Image Processing. Greenwich: JAI Press, 1984: 317-339. Google Scholar
[8]	Shen H F, Peng L, Yue L W, et al. Adaptive norm selection for regularized image restoration and super-resolution[J]. IEEE Transactions on Cybernetics, 2016, 46(6): 1388-1399. doi: 10.1109/TCYB.2015.2446755 CrossRef Google Scholar
[9]	Zeng X Y, Yang L H. A robust multiframe super-resolution algorithm based on half-quadratic estimation with modified BTV regularization[J]. Digital Signal Processing, 2013, 23(1): 98-109. doi: 10.1016/j.dsp.2012.06.013 CrossRef Google Scholar
[10]	Huang S Y, Sun J, Yang Y, et al. Multi-frame super-resolution reconstruction based on gradient vector flow hybrid field[J]. IEEE Access, 2017, 5: 21669-21683. doi: 10.1109/ACCESS.2017.2757239 CrossRef Google Scholar
[11]	Molina R, Vega M, Abad J, et al. Parameter estimation in Bayesian high-resolution image reconstruction with multisensors[J]. IEEE Transactions on Image Processing, 2003, 12(12): 1655-1667. doi: 10.1109/TIP.2003.818117 CrossRef Google Scholar
[12]	Babacan S D, Molina R, Katsaggelos A K. Variational Bayesian super resolution[J]. IEEE Transactions on Image Processing, 2011, 20(4): 984-999. doi: 10.1109/TIP.2010.2080278 CrossRef Google Scholar
[13]	Villena S, Vega M, Molina R, et al. A non-stationary image prior combination in super-resolution[J]. Digital Signal Processing, 2014, 32: 1-10. doi: 10.1016/j.dsp.2014.05.017 CrossRef Google Scholar
[14]	Liu C, Sun D Q. On Bayesian adaptive video super resolution[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(2): 346-360. doi: 10.1109/TPAMI.2013.127 CrossRef Google Scholar
[15]	Ng M K, Shen H F, Lam E Y, et al. A total variation regularization based super-resolution reconstruction algorithm for digital video[J]. EURASIP Journal on Advances in Signal Processing, 2007, 2007: 074585. doi: 10.1155/2007/74585 CrossRef Google Scholar
[16]	Beal M J. Variational algorithms for approximate Bayesian inference[D]. London, UK: University College London, 2003. Google Scholar
[17]	Bioucas-Dias J M, Figueiredo M A T, Oliveira J P. Total variation-based image deconvolution: a majorization-minimization approach[C]//Proceedings of 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, Toulouse, 2006: 861-864. Google Scholar
[18]	Dong C, Loy C C, He K M, et al. Image super-resolution using deep convolutional networks[EB/OL]. http://mmlab.ie.cuhk.edu.hk/projects/SRCNN.html. Google Scholar
[19]	Lucas B, Kanade T. An iterative image registration technique with an application to stereo vision[C]//Proceedings of the 7th International Joint Conference on Artificial Intelligence, San Francisco, 1981: 674-679. Google Scholar
[20]	Wang Z, Bovik A C. Mean squared error: love it or leave it? A new look at signal fidelity measures[J]. IEEE Signal Processing Magazine, 2009, 26(1): 98-117. doi: 10.1109/MSP.2008.930649 CrossRef Google Scholar
[21]	Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612. doi: 10.1109/TIP.2003.819861 CrossRef Google Scholar

Overview

Overview

Overview: Images with high spatial resolution are usually desirable in real applications. The most direct approaches to increase the spatial resolution are: 1) increasing the bandwidth of the optics; 2) increasing the sampling frequency of the image sensor. However, these hardware-based approaches usually increase the volume or cost of the imaging system. Different from the hardware-based approaches, we use digital signal processing algorithm to increase the spatial resolution by a set of low-resolution (LR) images of the same scene. This approach is termed as multi-frame image super-resolution (SR). Variational Bayesian framework has been used to derive the SR algorithms for its model flexibility and parameter adaptivity. For multi-frame image SR, the accurate estimation of blur kernel of LR image is prerequisite for high-efficiency SR reconstruction. However, in variational Bayesian SR framework, all the methods assume a known and fixed blur kernel for LR images. We propose a blind SR method containing blur kernel self-adaptive estimation. First, the desired high-resolution (HR) image and the blur kernel are modeled in the imaging degradation model. Next, the total variation model is used to model the HR image and the blur kernel, and the Gamma distribution is used to model the corresponding parameters. Finally, the variational Bayesian inference based on Kullback-Leibler divergence and majorization-minimization approach is utilized to derive the SR algorithm. For the proposed method, the HR image, the blur kernel and the model parameters are estimated simultaneously and automatically. Experiments demonstrate that the proposed method outperforms the state-of-art methods. For the experiments on simulated data, the performance of the resolution enhancement method is quantitatively measured by the peak signal-to-noise ratio (PSNR) and structural similarity measure (SSIM). For typical ground truth HR image and blur kernel setup, the proposed method has the highest PSNR and SSIM and improves the PSNR by at least 1 dB~5 dB. For the visual effect, the proposed method has better blur removing performance. For the real data experiments using resolution chart LR images, the proposed method has better performance in preserving image details, suppressing noise and removing artifacts. The comparison experiments demonstrate the advantage of the proposed method. Especially, for the high signal-to-noise ratio (SNR) scenarios, the accuracy of blur kernel estimation dominates the performance and the proposed method can improve the performance dramatically. By the visual effect, the proposed method has better trade-off in preserving image details and removing noise and artifacts.