Convolutional neural network (CNN) has recently achieved a great success for single image super-resolution (SISR). However, most deep CNN-based super-resolution models use chained stacking to build the network, which results in the fact that the relationship between layers is weak and does not make full use of hierarchical features. In this paper, a multi-path recursive convolutional network (MRCN) is designed to address these problems in SISR. By using multi-path structure to strengthen the relationship between layers, our network can ef-fectively utilize features and extract rich high-frequency components. At the same time, we also use recursive structure to alleviate training difficulty. In addition, by introducing the operation of feature fusion into the model, our network can make full use of the features extracted from each layer in the reconstruction process and select the effective features adaptively. Extensive experiments on benchmarks datasets have shown that MRCN has a significant performance improvement against existing methods.
Home > Journal Home > Opto-Electronic Engineering
Opto-Electronic Engineering
ISSN: 1003-501X
CN: 51-1346/O4
Monthly, included in CA, Scopus, CSCD
CN: 51-1346/O4
Monthly, included in CA, Scopus, CSCD
Image super-resolution via multi-path recursive convolutional network
Author Affiliations
Correspondence: yangjuan6985@163.com
First published at:Nov 15, 2019
Abstract
References
[1] Li X, Orchard M T. New edge-directed interpolation[J]. IEEE Transactions on Image Processing, 2001, 10(10): 1521–1527.
[2] Zhang L, Wu X L. An edge-guided image interpolation algo-rithm via directional filtering and data fusion[J]. IEEE Transactions on Image Processing, 2006, 15(8): 2226–2238.
[3] Dai S Y, Han M, Xu W, et al. SoftCuts: a soft edge smoothness prior for color image super-resolution[J]. IEEE Transactions on Image Processing, 2009, 18(5): 969–981.
[4] Sun J, Xu Z B, Shum H Y. Image super-resolution using gradient profile prior[C]//Proceedings of 2008 IEEE Conference on Computer Vision and Pattern Recognition, 2008: 1–8.
[5] Wu C Z, Hu C S, Zhang M J, et al. Single image su-per-resolution reconstruction via supervised multi-dictionary learning[J]. Opto-Electronic Engineering, 2016, 43(11): 69–75.
吴从中, 胡长胜, 张明君, 等. 有监督多类字典学习的单幅图像超分辨率重建[J]. 光电工程, 2016, 43(11): 69–75.
[6] Wang R G, Wang Q H, Yang J, et al. Image super-resolution reconstruction by fusing feature classification and in-dependent dictionary training[J]. Opto-Electronic Engineering, 2018, 45(1): 170542.
汪荣贵, 汪庆辉, 杨娟, 等. 融合特征分类和独立字典训练的超分辨率重建[J]. 光电工程, 2018, 45(1): 170542.
[7] Timofte R, De Smet V, Van Gool L. Anchored neighborhood regression for fast example-based super-resolution[C]// Pro-ceedings of 2013 IEEE International Conference on Computer Vision, 2013: 1920–1927.
[8] Timofte R, De Smet V, Van Gool L. A+: adjusted anchored neighborhood regression for fast su-per-resolution[M]//Cremers D, Reid I, Saito H, et al. Computer Vision--ACCV 2014. Cham: Springer, 2014: 111–126.
[9] Dong C, Loy C C, He K M, et al. Learning a deep convolutional network for image super-resolution[C]//Proceedings of the 13th European Conference on Computer Vision, 2014: 184–199.
[10] Dong C, Loy C C, He K M, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295–307.
[11] Yang J C, Wright J, Huang T S, et al. Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing, 2010, 19(11): 2861–2873.
[12] Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1646–1654.
[13] Kim J, Lee J K, Lee K M. Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1637–1645.
[14] Zhang K, Zuo W M, Gu S H, et al. Learning deep CNN Denoiser prior for image restoration[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2808–2817.
[15] Shi W Z, Jiang F, Zhao D B. Single image super-resolution with dilated convolution based multi-scale information learning in-ception module[C]//Proceedings of 2017 IEEE International Conference on Image Processing, 2017: 977–981.
[16] Ledig C, Theis L, Huszár F, et al. Photo-realistic single image super-resolution using a generative adversarial net-work[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 105–114.
[17] Tai Y, Yang J, Liu X M. Image super-resolution via deep recursive residual network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2790–2798.
[18] Mao X J, Shen C H, Yang Y B. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems, 2016: 2810–2818.
[19] Tai Y, Yang J, Liu X M, et al. MemNet: a persistent memory network for image restoration[C]//Proceedings of 2017 IEEE International Conference on Computer Vision, 2017: 4549–4557.
[20] Dong C, Loy C C, Tang X O. Accelerating the super-resolution convolutional neural network[C]//Proceedings of the 14th European Conference on Computer Vision, 2016: 391–407.
[21] Shi W Z, Caballero J, Huszár F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1874–1883.
[22] Lai W S, Huang J B, Ahuja N, et al. Deep Laplacian pyramid networks for fast and accurate super-resolution[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 5835–5843.
[23] Huang G, Liu Z, van der Maaten L, et al. Densely connected convolutional networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2261–2269.
[24] Chen Y P, Li J A, Xiao H X, et al. Dual path net-works[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 4470–4478.
[25] Hu J, Shen L, Albanie S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, doi: 10.1109/TPAMI.2019.2913372.
[26] Timofte R, Agustsson E, van Gool L, et al. NTIRE 2017 challenge on single image super-resolution: methods and results[C]//Proceedings of 2017 IEEE Conference on Com-puter Vision and Pattern Recognition Workshops, 2017: 1110–1121.
[27] Bevilacqua M, Roumy A, Guillemot C, et al. Low-complexity single-image super-resolution based on nonnegative neighbor embedding[C]//Proceedings of the British Machine Vision Conference, 2012: 135.1–135.10.
[28] Zeyde R, Elad M, Protter M. On single image scale-up using sparse-representations[C]//Proceedings of the 7th In-ternational Conference on Curves and Surfaces, 2010: 711–730.
[29] Huang J B, Singh A, Ahuja N. Single image super-resolution from transformed self-exemplars[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition, 2015: 5197–5206.
[30] Martin D, Fowlkes C, Tal D, et al. A database of human seg-mented natural images and its application to evaluating segmentation algorithms and measuring ecological statis-tics[C]//Proceedings of the 8th IEEE International Conference on Computer Vision, 2001: 416–423.
[31] He K M, Zhang X Y, Ren S Q, et al. Delving deep into rectifiers: surpassing human-level performance on ImageNet classification[C]//Proceedings of 2015 IEEE International Conference on Computer Vision, 2015: 1026–1034.
[32] Zhao H, Gallo O, Frosio I, et al. Loss functions for neural networks for image processing[OL]. arXiv:1511.08861[cs.CV], 2015.
[33] Bruhn A, Weickert J, Schn?rr C. Lucas/Kanade meets Horn/Schunck: combining local and global optic flow meth-ods[J]. International Journal of Computer Vision, 2005, 61(3): 211–231.
Keywords:
Funds:
National Natural Science Foundation of China (61672202)
Export Citations as:
For
Get Citation:
Shen Mingyu, Yu Pengfei, Wang Ronggui, et al. Image super-resolution via multi-path recursive convolutional network[J]. Opto-Electronic Engineering, 2019, 46(11): 180489.
Previous: An accurate measurement method for the spatial resolution of area array spectral imaging equipment
