E-mail Alert
RSS
| Citation: |
Zheng H Y, Luo Y H, Li T L, et al. Confocal non-line-of-sight imaging based on spatial multiplexing detection[J]. Opto-Electron Eng, 2023, 50(5): 220256. doi: 10.12086/oee.2023.220256
|
Confocal non-line-of-sight imaging based on spatial multiplexing detection
-
Abstract
Non-line-of-sight imaging techniques can be used to capture images of objects that are hidden away in corners, and this technology has many useful applications. The captured signal is insufficient when the laser sends laser light to the intermediate surface, which is a problem when utilizing a digital micromirror device (DMD) in a confocal non-line-of-sight optical route because of the spectroscopic impact of the DMD. In order to increase the signal-to-noise ratio of the echo signal, the acquisition method of spatial multiplexing detection (SMD) is introduced into the confocal optical path in this study. SMD is integrated with the reconstruction algorithm of the light-cone transform (LCT). Based on the confocal non-line-of-sight imaging optical path, the laser initially emits laser pulses, followed by SMD collecting concealed object echoes, and the LCT algorithm concludes the reconstruction. The experimental results demonstrate that the single-pixel camera can complete the reconstruction using the LCT algorithm after employing the SMD acquisition method in the confocal non-line-of-sight optical route, which may improve the signal-to-noise ratio and the reconstruction quality. -
-
References
[1] Raskar R, Davis J. 5D time-light transport matrix: what can we reason about scene properties?[R/OL]. MIT Technical Report, (2008-03-01). https://dspace.mit.edu/handle/1721.1/67888. [2] Kirmani A, Hutchison T, Davis J, et al. Looking around the corner using ultrafast transient imaging[J]. Int J Comput Vis, 2011, 95(1): 13−28. doi: 10.1007/s11263-011-0470-y [3] Velten A, Willwacher T, Gupta O, et al. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging[J]. Nat Commun, 2012, 3: 745. doi: 10.1038/ncomms1747 [4] Arellano V, Gutierrez D, Jarabo A. Fast back-projection for non-line of sight reconstruction[J]. Opt Express, 2017, 25(10): 11574−11583. doi: 10.1364/OE.25.011574 [5] La Manna M, Kine F, Breitbach E, et al. Error backprojection algorithms for non-line-of-sight imaging[J]. IEEE Trans Pattern Anal Mach Intell, 2019, 41(7): 1615−1626. doi: 10.1109/TPAMI.2018.2843363 [6] O’Toole M, Lindell D B, Wetzstein G. Confocal non-line-of-sight imaging based on the light-cone transform[J]. Nature, 2018, 555(7696): 338−341. doi: 10.1038/nature25489 [7] Heide F, O’Toole M, Zang K, et al. Non-line-of-sight Imaging with partial occluders and surface normals[J]. ACM Trans Graph, 2019, 38(3): 22. doi: 10.1145/3269977 [8] Xin S M, Nousias S, Kutulakos K N, et al. A theory of Fermat paths for non-line-of-sight shape reconstruction[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. https://doi.org/10.1109/CVPR.2019.00696. [9] Lindell D B, Wetzstein G, O’Toole M. Wave-based non-line-of-sight imaging using fast f-k migration[J]. ACM Trans Graph, 2019, 38(4): 116. doi: 10.1145/3306346.3322937 [10] Liu X C, Bauer S, Velten A. Phasor field diffraction based reconstruction for fast non-line-of-sight imaging systems[J]. Nat Commun, 2020, 11(1): 1645. doi: 10.1038/s41467-020-15157-4 [11] 周强, 刘金璐, 谷远辉, 等. 量子保密通信用增益开关半导体脉冲激光器[J]. 中国激光, 2016, 43(5): 0502005. doi: 10.3788/CJL201643.0502005 Zhou Q, Liu J L, Gu Y H, et al. Gain-switched semiconductor pulsed laser for quantum secure communication[J]. Chin J Lasers, 2016, 43(5): 0502005. doi: 10.3788/CJL201643.0502005 [12] 彭柏语, 袁晨智, 张瑞明, 等. 用于量子通信的增益开关半导体激光器研究进展[J]. 光学学报, 2022, 42(3): 0327007. doi: 10.3788/AOS202242.0327007 Peng B Y, Yuan C Z, Zhang R M, et al. Progress in gain-switched semiconductor lasers for quantum communication[J]. Acta Opt Sin, 2022, 42(3): 0327007. doi: 10.3788/AOS202242.0327007 [13] Chopite J G, Hullin M B, Wand M, et al. Deep non-line-of-sight reconstruction[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020. [14] Heide F, Xiao L, Heidrich W, et al. Diffuse mirrors: 3D reconstruction from diffuse indirect illumination using inexpensive time-of-flight sensors[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014: 3222–3229. [15] Kadambi A, Zhao H, Shi B X, et al. Occluded imaging with time-of-flight sensors[J]. ACM Trans Graph, 2016, 35(2): 15. doi: 10.1145/2836164 [16] Chen W Z, Daneau S, Brosseau C, et al. Steady-state non-line-of-sight imaging[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. [17] Klein J, Peters C, Martín J, et al. Tracking objects outside the line of sight using 2D intensity images[J]. Sci Rep, 2016, 6: 32491. doi: 10.1038/srep32491 [18] Smith B M, O’Toole M, Gupta M. Tracking multiple objects outside the line of sight using speckle imaging[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 6258–6266. https://doi.org/10.1109/CVPR.2018.00655. [19] Gkioulekas I, Levin A, Durand F, et al. Micron-scale light transport decomposition using interferometry[J]. ACM Trans Graph, 2015, 34(4): 37. doi: 10.1145/2766928 [20] Willomitzer F, Li F Q, Rangarajan P, et al. Non-line-of-sight imaging using superheterodyne interferometry[C]//Proceedings of the Imaging and Applied Optics 2018, 2018. https://doi.org/10.1364/COSI.2018.CM2E.1. [21] 刘博, 姜朔, 于洋, 等. 宏/子脉冲编码光子计数激光雷达[J]. 光电工程, 2020, 47(10): 200265. doi: 10.12086/oee.2020.200265 Liu B, Jiang S, Yu Y, et al. Macro/sub-pulse coded photon counting LiDAR[J]. Opto-Electron Eng, 2020, 47(10): 200265. doi: 10.12086/oee.2020.200265 [22] Liu B, Yu Y, Chen Z, et al. True random coded photon counting Lidar[J]. Opto-Electron Adv, 2020, 3(2): 190044. doi: 10.29026/oea.2020.190044 [23] Wu C, Liu J J, Huang X, et al. Non–line-of-sight imaging over 1.43 km[J]. Proc Natl Acad Sci USA, 2021, 118(10): e2024468118. doi: 10.1073/pnas.2024468118 [24] 任禹, 罗一涵, 徐少雄, 等. 非视域定位中光子飞行时间提取方法对比研究[J]. 光电工程, 2021, 48(1): 200124. doi: 10.12086/oee.2021.200124 Ren Y, Luo Y H, Xu S X, et al. A comparative study of time of flight extraction methods in non-line-of-sight location[J]. Opto-Electron Eng, 2021, 48(1): 200124. doi: 10.12086/oee.2021.200124 [25] Nam J H, Brandt E, Bauer S, et al. Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second[J]. Nat Commun, 2021, 12(1): 6526. doi: 10.1038/s41467-021-26721-x [26] Musarra G, Lyons A, Conca E, et al. Non-line-of-sight three-dimensional imaging with a single-pixel camera[J]. Phys Rev Appl, 2019, 12(1): 011002. doi: 10.1103/PhysRevApplied.12.011002 [27] Yang W Q, Zhang C, Jiang W J, et al. None-line-of-sight imaging enhanced with spatial multiplexing[J]. Opt Express, 2022, 30(4): 5855−5867. doi: 10.1364/OE.450238 [28] Wiener N. Extrapolation, Interpolation, and Smoothing of Stationary Time Series: with Engineering Applications[M]. Cambridge: Technology Press of the Massachusetts Institute of Technology, 1949. -
Overview
It is usually challenging to image objects that are beyond sight, yet this technology has many possible applications. In situations like autonomous driving, counterterrorism, and rescue operations, analyzing the object information in the hidden area might help people make better decisions. Ultrafast lasers and detectors with high sensitivity and time resolution have been developed thanks to advances in photoelectric technology. The flight time of photons can be measured with a single photon detector. The intermediary surface receives laser pulses from the laser. Through the reflection of the intervening surface, the laser illuminates the concealed object. The detector gathers the three reflected echo photons that were sent back by the intermediary surface once the hidden object's reflection reaches it, extracts the data from the echo photon, and then uses an image technique to reconstruct the hidden target. In order to complete the reconstruction of hidden objects, the researchers have also successively proposed the reverse projection algorithm (BP), the iterative error anti-projection algorithm, the light-cone transform algorithm (LCT), the surface normal algorithm, and the non-line-of-sight reconstruction algorithm based on virtual waves. However, the scanning of the intermediary surface by the laser can lead to long data acquisition times. Later researchers have used a digital micromirror device (DMD) to flip the echo signals into a single photon detector, which allows the acquisition of signals to be finished without the scanning component. In earlier research, back-projection was used to reconstitute signals obtained with DMDs. In this paper, the spectroscopic effect of the DMD element in the optical route is resolved, and the signal acquisition method of SMD is brought into the non-line-of-sight imaging of the confocal optical path. The hidden object signal is captured by switching the micromirror, and the reconstruction is finished using LCT and BP algorithms, employing a confocal non-line-of-sight imaging optical route with a DMD element. By comparing the reconstruction quality of BP and LCT, SSIM and PSNR demonstrate that LCT can achieve greater reconstruction quality when using the SMD acquisition method. Additionally, research have shown that using this acquisition method will increase the quality of the reconstruction by increasing the laser intensity and the number of sampling points.
-
Access History
Export File
Citation
Zheng H Y, Luo Y H, Li T L, et al. Confocal non-line-of-sight imaging based on spatial multiplexing detection[J]. Opto-Electron Eng, 2023, 50(5): 220256. doi: 10.12086/oee.2023.220256
Format
Content
DownLoad:
-
Figure 1.
Schematics of the experimental set-up for non-line-of-sight imaging
-
Figure 2.
Hadamard observation pattern and matrix. (a) Hadamard observation pattern; (b) Hadamard observation matrix
-
Figure 3.
Principle of spatial multiplexing detection non-line-of-sight imaging. (a) Original signal of each detection point; (b) Four kinds of measurement patterns obtained from Hadamard matrix; (c) Time photon histogram in each measurement mode; (d) Recover the signal of each detection point
-
Figure 4.
Experimental light path diagram
-
Figure 5.
Comparison of reconstruction results. (a) LCT algorithm reconstruction results; (b) BP algorithm reconstruction results
-
Figure 6.
Reconstruction results of 16×16 pixel values for different hidden objects. (a) Rectangle result reconstructed by LCT algorithm; (b) Rectangle result reconstructed by BP algorithm; (c) Triangle result reconstructed by LCT algorithm; (d) Triangle result reconstructed by BP algorithm
-
Figure 7.
The reconstruction result of the 8×8 pixel value of the triangular object. (a) Triangle result reconstructed by LCT algorithm; (b) Triangle result reconstructed by BP algorithm
-
Figure 8.
The reconstruction results of the LCT algorithm obtained with different laser intensities. (a) Reconstruction results with an average laser power of 750 mW; (b) Reconstruction results with an average laser power of 500 mW
-
Figure 9.
Reconstruction results of the LCT algorithm obtained by different sampling arrays. (a) Reconstruction result of the sampling array being16×16; (b) Reconstruction result of the sampling array being 8×8;
- Figure .
