E-mail Alert
RSS
| Citation: |
Dengmei Xie, Mei Yu, Zongju Peng, et al. High dynamic range imaging method based on image content adaptive matrix completion[J]. Opto-Electronic Engineering, 2017, 44(11): 1056-1065. doi: 10.3969/j.issn.1003-501X.2017.11.004
|
High dynamic range imaging method based on image content adaptive matrix completion
-
Abstract
High dynamic range (HDR) imaging usually produces ghosting artifacts, while the traditional matrix completion (MC) method may fail to completely remove the ghosts, without considering the motion characteristics of multi-exposure image. To solve this problem, this paper presents a new HDR imaging method based on content adaptive matrix completion of low dynamic range (LDR) image to remove the ghosts of HDR image. Firstly, according to the image luminance and chrominance information, the LDR image motion area is determined. Then, based on the priori information of motion, the regularization constraint intensity is adjusted in MC process to get each LDR image background information. Finally, a fusion strategy related to multiple exposures is proposed while the difference of details in each image area under different exposures is considered. Regular background sequences and cluttered background sequences are used for experiments. The experimental results demonstrate that, compared with the partial sum minimization of singular values-matrix completion method, the proposed method is more real-time and suitable for cluttered background sequences -
-
References
[1] Reinhard E, Heidrich W, Debevec P, et al. High dynamic range imaging: acquisition, display and image-based lighting. 2nd ed. [M]. Morgan Kaufmann Publishers Inc, 2010: 171–183. [2] Debevec P E, Malik J. Recovering high dynamic range radiance maps from photographs[C]// Proceedings of the 24th annual conference on Computer graphics and interactive techniques, New York: ACM, 1997: 369–378. [3] Mitsunaga T, Nayar S K. Radiometric self calibration[C]// Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, New York: ACM, 1999: 374–380. [4] Jacobs K, Loscos C, Ward G. Automatic high dynamic range image generation for dynamic scenes[J]. IEEE Computer Graphics and Applications, 2008, 28(2): 84–93. doi: 10.1109/MCG.2008.23 [5] Zheng J H, Li Z G, Zhu Z J, et al. Hybrid patching for a se-quence of differently exposed images with moving objects[J]. IEEE Transactions on Image Processing, 2013, 22(12): 5190–5201. doi: 10.1109/TIP.2013.2283401 [6] Zhang W, Cham W K. Gradient–directed multiexposure composition[J]. IEEE Transactions on Image Processing, 2012, 21(4): 2318–2323. doi: 10.1109/TIP.2011.2170079 [7] Khan E A, Akyuz A O, Reinhard E. Ghost removal in high dynamic range images[C]// IEEE International Conference on Image Processing, Atlanta, GA, USA: IEEE, 2006: 2005–2008. [8] Lee C, Li Y L, Monga V. Ghost–free high dynamic range imaging via rank minimization[J]. IEEE Signal Processing Letters, 2014, 21(9): 1045–1049. doi: 10.1109/LSP.2014.2323404 [9] Oh T H, Lee J Y, Tai Y W, et al. Robust high dynamic range imaging by rank minimization[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(6): 1219–1232. doi: 10.1109/TPAMI.2014.2361338 [10] Pece F, Kautz J. Bitmap movement detection: HDR for dynamic scenes[C]// Proceedings of the 2010 Conference on Visual Media Production, Washington, DC, USA: IEEE Computer Society, 2010. [11] Ma K, Li H, Yong H W, et al. Robust multi–exposure image fusion: a structural patch decomposition approach[J]. IEEE Transactions on Image Processing, 2017, 26(5): 2519–2532. doi: 10.1109/TIP.2017.2671921 [12] Lee J Y, Matsushita Y, Shi B X, et al. Radiometric calibration by rank minimization[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1): 144–156. doi: 10.1109/TPAMI.2012.66 [13] Hu Y, Zhang D B, Ye J P, et al. Fast and accurate matrix completion via truncated nuclear norm regularization[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(9): 2117–2130. doi: 10.1109/TPAMI.2012.271 [14] Gallo O, Gelfand N, Chen W C, et al. Artifact-free high dynamic range imaging[C]// IEEE International Conference on Computational Photography, San Francisco CA, USA: IEEE, 2009: 1–7. [15] [Online]. Available: . http://users.soe.ucsc.edu/orazio/deghost.html. [16] Heo Y S, Lee K M, Lee S U, et al. Ghost–free high dynamic range imaging[C]// Proceedings of the 10th Asian Conference of Computer Vision (ACCV), New Zealand: Queenstown, 2010: 486–500. [17] Hu J, Gallo O, Pulli K, et al. HDR deghosting: how to deal with saturation?[C]// IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA: IEEE, 2013: 1163–1170. [18] Sen P, Kalantari N K, Yaesoubi M, et al. Robust patch–based HDR reconstruction of dynamic scenes[J]. ACM Transactions on Graphics, 2012, 31(6): 1–11. [19] Oh T H, Lee J Y, Kweon I S. High dynamic range imaging by a rank-1 constraint[C]// IEEE International Conference on Image Processing (ICIP), Melbourne, VIC, Australia: IEEE, 2013: 790–794. [20] Lee Chul. Ghost-free high dynamic range imaging [EB/OL]. . http://cilab.pknu.ac.kr/research/rm_hdr.html. [21] Karađuzović-Hadžiabdić K, Telalović J H, Mantiuk R K. As-sessment of multi-exposure HDR image deghosting methods[J]. Computers & Graphics, 2017, 63: 1–17. -
Overview
High dynamic range (HDR) image is more consistent with human perception, and is being applied to many fields, such as consumer electronics, remote sensing system, intelligent transportation, security monitoring. So far, HDR imaging has usually produced ghosting artifacts, so there are many researchers carrying out the research about de-ghosting. Some researchers came up with the algorithms which can remove ghosts by detecting moving regions, and other researchers considered the fusion weight distribution to eliminate the artifacts. Recently, the algorithm that models the HDR imaging problem as a rank minimization problem is proposed. It is solved with the idea of matrix completion (MC). However, as no attention was paid to the motion characteristics of multi-exposure image, the traditional MC method could fail to completely remove the ghosts. Therefore, in order to solve this problem, according to the motion information of multi-exposure images and fusion strategy related to exposures, this paper presents a new HDR imaging method based on content adaptive MC of low dynamic range (LDR) image to remove the ghosts of HDR image. Firstly, the motion area of LDR image is obtained based on the luminance and color information of the image with the median threshold bitmap (MTB) features, the hue feature H and color saturation characteristics S. Then, based on the priori information of motion, regularization strength is adjusted in the MC process to obtain the low-rank background information of each LDR image. Finally, a fusion strategy related to multiple exposures is proposed to achieve a ghost-free HDR image, while the difference of details in each image area under different exposures is considered. Regular background sequences and cluttered background sequences are used for experiments. The experimental results demonstrate that, compared with the partial sum minimization of singular values-matrix completion method, the proposed method is more real-time and suitable for cluttered background sequences. The above figure is part of the experimental results, where (e) and (h) are the results of the proposed algorithm of Arch and Forest sequences. Compared with these algorithms, the images synthesized by the proposed algorithm not only have no ghost, but also have richer details. In addition, by calculating the processing speed of several classical algorithms and the proposed algorithm, it is shown that the proposed algorithm can reduce the computational complexity of the overall HDR imaging operation and run more efficiently.
-
Access History
Figures(11)
Tables(2)
Article Metrics
Export File
Citation
Dengmei Xie, Mei Yu, Zongju Peng, et al. High dynamic range imaging method based on image content adaptive matrix completion[J]. Opto-Electronic Engineering, 2017, 44(11): 1056-1065. doi: 10.3969/j.issn.1003-501X.2017.11.004
Format
Content
DownLoad:
-
Figure 1.
Influence of parameter λ on synthesis results. (a)λ=[max(r, c)]-1/2. (b) Turning up λ. (c) Turning down λ
-
Figure 2.
Farmework of the proposed algorithm.
-
Figure 3.
Feature figures of multi exposures sequence Arch. (a)Arch image sequence (5 LDR images with different exposures). (b) MTB features. (c) Hue feature H. (d) Color saturation characteristics S.
-
Figure 4.
Farmework of motion area detection.
-
Figure 5.
Relationship between normalized luminance and fusion weight and exposure time. (a) Normalized brightness. (b) Fusion weights.
-
Figure 6.
Moving region detection results corresponding to different feature combinations. (a) MTB. (b) H component. (c) S component. (d) MTB and H component. (e) MTB and S component. (f) H component and S component. (g) MTB, H component and S component. (h) GTM.
-
Figure 7.
Similarity between the moving region detection result and the GTM similarity curve under different feature combinations.
-
Figure 8.
Comparison of fusion results. (a) Regional division. (b) Average fusion strategy. (c) Proposed algorithm.
-
Figure 9.
Synthesis results of HDR images in static background. (a) Non-manually selected PSSV-MC. (b) Manually selected PSSV-MC. (c) The proposed algorithm.
-
Figure 10.
Comparison experiments of Arch sequences. (a) Debevec, et al[2]. (b) Zhang, et al[6]. (c) Gallo, et al[14]. (d) Oh, et al[19]. (e) Heo, et al[16]. (f) Zheng, et al[5]. (g) Hu, et al[17]. (h) Sen, et al[18]. (i) Photoshop CS6. (j) The proposed algorithm.
-
Figure 11.
Comparison experiment of Forest sequence. (a) Forest multi exposure image sequences. (b) Debevec, et al[2]. (c) Photoshop CS6. (d) Heo, et al[16]. (e) Gallo, et al[13]. (f) Sen, et al[18]. (g) The proposed algorithm.
