﻿ 基于对象的三维图像颜色传递与视差优化
 光电工程  2019, Vol. 46 Issue (9): 180446      DOI: 10.12086/oee.2019.180446

Stereoscopic color transfer and disparity remapping based on selected object
Li Pengfei, Shao Feng
College of Information Science and Engineering, Ningbo University, Ningbo, Zhejiang 315211, China
Abstract: Color transfer has been a hot research issue in the field of image processing and computer vision in recent years. The main purpose is to transfer the color of a target image to source image so that the source image has the same or similar color features with the target image. In practical applications for the color transfer of binocular stereoscopic images, the user may only need to transfer the color of the selected object while keeping the background color unchanged. For this purpose, a color transfer method based on the selected object is proposed in this paper. In the method, by assigning the object of the image by user, the accurate object is segmented via graph cut, and the probability density curves of color distribution between the selected object and the target image are matched to accomplish the color transfer. In order to enhance the viewing experience provided for the user, a non-linear disparity optimization is performed after the color transfer operation. According to the histogram feature of disparity map, the disparity mapping function is calculated, and the target disparity is obtained to enhance the depth sensation of the selected object. The experimental results demonstrate that the combination of stereoscopic color transfer and disparity remapping effectively enhances the stereoscopic viewing experience.
Keywords: stereoscopic images    multivariate generalized Gaussian distribution (MGGD)    color transfer    disparity remapping

1 引言

2 基于对象的立体颜色传递与非线性视差优化方法

 图 1 基于对象的立体图像颜色传递与视差优化方法框图 Fig. 1 The proposed framework
2.1 基于用户选择对象的颜色传递

2.1.1 用户选取对象的分割

 图 3 颜色分布模型变换。 Fig. 3 Color distribution transformation. (a)源图像颜色特征模型；(b)目标图像颜色特征模型 (a) Source color distribution; (b) Target color distribution
2.2 非线性视差优化

 ${\phi _n}(d) = \log (1 + s \times d),$ (11)

 ${\phi_{\rm{a}}}(d) = \left\{ \begin{gathered} {\phi_0}(d), \;\;d \in {\mathit{\Omega} _0} \\ ... \\ {\phi_n}(d), \;\;d \in {\mathit{\Omega} _n} \\ \end{gathered} \right.。$ (12)

 ${\phi _{\rm{a}}}(d) = \int_0^d {{\phi _{\rm{a}}}^\prime (x)} {\rm{d}}x。$ (13)

 图 4 非线性视差映射关系示意。 Fig. 4 Nonlinear disparity mapping. (a)源视差图；(b)视差直方图；(c)视差映射关系 (a) Source disparity; (b) Disparity histogram; (c) Disparity mapping

 图 5 非线性视差映射效果。 Fig. 5 Nonlinear disparity mapping result. (a)立体图；(b)源视差；(c)视差映射关系；(d)目标视差 (a) Stereo images; (b) Source disparity; (c) Mapping function; (d) Target disparity
3 实验结果与分析

3.1 基于对象的颜色传递和视差优化结果分析

 图 6 基于用户选择对象的颜色传递。 Fig. 6 Color transfer based on selected objects. (a)源图像；(b)目标图像；(c)选取对象；(d)颜色传递 (a) Source images; (b) Target images; (c) Selected objects; (d) The results

 图 7 本文方法的结果。 Fig. 7 Results of the proposed method. (a)左视图；(b)右视图；(c)目标视差；(d)本文结果 (a) Left view image; (b) Right view image; (c) Target disparity; (d) Our results

3.2 与线性视差优化方法的比较

 图 8 视差优化方法对比。 Fig. 8 Comparison of disparity optimization methods. (a)源视差；(b)线性视差调整；(c)本文方法 (a) Source disparity; (b) Linear disparity mapping; (c) Our strategy
3.3 颜色传递方法对比

 图 9 颜色传递方法对比。 Fig. 9 Comparison of color transfer methods. (a)源图像；(b)目标图像；(c)文献[19]方法；(d)文献[20]方法；(e)本文方法 (a) Source image; (b) Target image; (c) Method [19]; (d) Method [20]; (e) Our method
3.4 本文方法的主观实验分析

 图 10 本文方法与源图像对比结果 Fig. 10 Comparison between proposed method and the source image

 图 11 本文方法与线性视差调整方法对比结果 Fig. 11 Comparison between proposed method and the linear disparity mapping
3.5 局限性

4 结论

 [1] Faridul H S, Pouli T, Chamaret C, et al. A survey of color mapping and its applications[C]//Eurographics 2014 State of the Art Reports, 2014. [2] Pouli T, Reinhard E. Progressive histogram reshaping for creative color transfer and tone reproduction[C]//Proceedings of the 8th International Symposium on Non-Photorealistic Animation and Rendering, 2010: 81–90. [Crossref] [3] Reinhard E, Adhikhmin M, Gooch B, et al. Color transfer between images[J]. IEEE Computer Graphics and Applications, 2001, 21(5): 34-41. [Crossref] [4] Pitie F, Kokaram A. The linear Monge-Kantorovitch linear colour mapping for example-based colour transfer[C]//Proceedings of the 4th European Conference on Visual Media Production, 2007. [5] Evans L C. Partial differential equations and monge-kantorovich mass transfer[J]. Current Developments in Mathematics, 1997, 1997: 65-126. [Crossref] [6] Hristova H, Le Meur O, Cozot R, et al. Style-aware robust color transfer[C]//Proceedings of the workshop on Computational Aesthetics, 2015. [Crossref] [7] Abadpour A, Kasaei S. A fast and efficient fuzzy color transfer method[C]//Proceedings of the 4th IEEE International Symposium on Signal Processing and Information Technology, 2004: 491–494. [Crossref] [8] Tai Y W, Jia J Y, Tang C K. Local color transfer via probabilistic segmentation by expectation-maximization[C]//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005: 747–754. [Crossref] [9] Hristova H, Le Meur O, Cozot R, et al. Transformation of the beta distribution for color transfer[C]//Proceedings of the 13th International Conference on Computer Graphics Theory and Applications, 2018: 112–121. [10] Agrawal A, Raskar R. Gradient domain manipulation techniques in vision and graphics[C]//Proceedings of the 11th International Conference on Computer Vision, 2007. [11] Shao F, Lin W S, Li Z T, et al. Toward simultaneous visual comfort and depth sensation optimization for stereoscopic 3-D experience[J]. IEEE Transactions on Cybernetics, 2017, 47(12): 4521-4533. [Crossref] [12] Pascal F, Bombrun L, Tourneret J Y, et al. Parameter estimation for multivariate generalized Gaussian distributions[J]. IEEE Transactions on Signal Processing, 2013, 61(23): 5960-5971. [Crossref] [13] Bombrun L, Pascal F, Tourneret J Y, et al. Performance of the maximum likelihood estimators for the parameters of multivariate generalized Gaussian distributions[C]//Proceedings of 2012 IEEE International Conference on Acoustics, Speech and Signal Processing, 2012: 3525–3528. [14] Hristova H, Le Meur O, Cozot R, et al. Transformation of the multivariate generalized Gaussian distribution for image editing[J]. IEEE Transactions on Visualization and Computer Graphics, 2018, 24(10): 2813-2826. [Crossref] [15] Li B F. The Research on image segmentation algorithms based on RGBD images[D]. Hefei: Hefei University of Technology, 2016. 李冰飞.基于RGBD图像的图像分割算法研究[D].合肥: 合肥工业大学, 2016. [Crossref] [16] Lang M, Hornung A, Wang O, et al. Nonlinear disparity mapping for stereoscopic 3D[J]. ACM Transactions on Graphics, 2010, 29(4): 75. [Crossref] [17] Sun D Q, Roth S, Black M J. Secrets of optical flow estimation and their principles[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010: 2432–2439. [Crossref] [18] Shamir A, Sorkine O. Visual media retargeting[C]//ACM SIGGRAPH ASIA 2009 Courses, 2009: 1–13. [Crossref] [19] Pitié F, Kokaram A C, Dahyot R. Automated colour grading using colour distribution transfer[J]. Computer Vision and Image Understanding, 2007, 107(1-2): 123-137. [Crossref] [20] Pitie F, Kokaram A C, Dahyot R. N-dimensional probability density function transfer and its application to color transfer[C]//Proceedings of the 10th International Conference on Computer Vision, 2005: 1434–1439. [Crossref]