Non-line-of-sight detection makes it possible to observe round obstacles

Observing round obstacles is not only a superpower that human beings want to get, but also a topic that researchers interested in. we are not aim to get everything out of sight with this ability, but to recognize the hazard in advance which cannot be noticed normally. When driving a motor vehicle (especially autopilot), drivers can only notice the danger in his view. The traditional vision method and radar can only help to get a wider visual angle, but can't see the vehicles or pedestrians behind the corner. If the drivers are informed of the condition, there will be more time for them to make decision. Although corner-mirrors can be used to see the situation behind the corner, it is inflexible and may cause the light pollution and confuse drivers, in case a lot of corner-mirrors are installed. The case of rescue is similar with driving. It is dangerous for rescuers to search the trapped people by entering the dangerous areas. Therefore, observing the environment out of sight is helpful to reduce casualties and has become an international research hotspot.

With the development of optoelectronic technology, high sensitivity measurement methods, single photon avalanche diodes, superconducting nanowire single-photon detectors and streak cameras with ultra-high temporal and spatial resolution, have emerged one after another. This makes it possible to measure the spatial-temporal distribution of photons with high sensitivity and high spatial-temporal resolution. Foreign scholars use this kind of detector with ultra-high sensitivity and resolution to measure the weak optical signal produced by a paused laser. The signal is scattered from the invisible object (non-line-of-sight) to the area. Non-line-of-sight scene is reconstructed according to the time information (time of flight) and intensity information of the optical signal. However, the algorithm of 3D reconstruction takes several tens of minutes. The localization algorithm for non-line-of-sight objects runs fast, but it relies on the artificial selection of signal features. Therefore a fast and automatic detection of non-line-of-sight objects cannot be realized. In the actual application, the position of non-line-of-sight objects is more important than its details. So it is more urgent to achieve the fast and automatic non-line-of-sight location.

A method of modeling the non-line-of-sight detection process using photometry is proposed by the team of Researcher Luo Yihan from Institute of Optics and Electronics, Chinese Academy of Science. By using the photon flight model based on photometry, the energy variation of the pulse beam from its radiation and detection are studied, including the whole process of flight and scattering. In order to realize the fast and automatic non-line-of-sight detection, this team proposed a method of processing the weak optical signal using filtering methods instead of the traditional fitting method to extract the time of flight, and then locate the non-line-of-sight target more automatically and accurately. The team also proposed a probability weighted method to improve the positioning accuracy. The above methods have been verified in both the model and experiment based on photometry.

Figure 1 Principle of location algorithm

The results show that the mean filter and median filter are sensitive to the parameters of the filtering interval. The positioning results of Gauss filter algorithm tend to be stable with the increase of the interval, a larger interval can be selected without considering the amount of computation. The probability weighted method with a threshold can reduce the location error using Gauss filtering location method, but it is not suitable for mean filtering and median filtering location method. The distance between the laser source and the detectors is limited by the environment. In a narrow environment with the width less than 1 m, the median filtering algorithm can obtain a better positioning effect. When the object moves in a given range, the positioning stability of the mean filter and Gauss filter is better than that of the median filter, but considering the positioning effect reflected by the mean error, the median filter has more advantages.

Figure 2 The influence of (a) filter interval (b) probability threshold (c) equipment distance on positioning accuracy

Finally, the effectiveness of these three filtering algorithms to extract time-of-flight is verified by experiments. The experimental results demonstrate that these three filtering methods are more stable than the traditional Gauss fitting method. According to the comparison of experimental results, the positioning accuracy using Gauss filter is higher than others, and the positioning results obtained by the median filter method may be distorted.

Figure 3 (a) Experimental environment (b) Comparison of positioning results between traditional method and filtering methods

This work not only improves the accuracy and automation of non-line-of-sight positioning, but also proposes a new way to study non-line-of-sight problems, and provides a reference of algorithm selection in different situations for the better positioning effect. The related research work was published on the first issue of Opto-Electronic Engineering in 2021 with the title of "A comparative study of time of flight extraction methods in non-line-of-sight location".

About The Group

Professor Luo Yihan from Key Laboratory of Beam Control, Chinese Academy of Sciences (CAS) is also the member of Youth Innovation Promotion Association CAS, Sichuan Science and Technology Youth Association, and Western Light Program of CAS. The research group includes Ma Haotong, Xie Zongliang, Xu Shaoxiong, Ren Yu, Li Tailin, etc., and the research fields involve non-line-of-sight detection, weak-target detection, and precise pointing and tracking. So far, they have been in charge of more than 10 national projects and various funds, including the National 863 Program, Western Light Program, laboratory funds, etc., published more than 10 papers, and applied for more than 20 patents. In 2020, they won the "Significant Scientific Achievement Award of Institute of Optics and Electronics (IOE), CAS" and "Scientific Contribution Award of IOE, CAS".


Sun Rui, Zhang Han, Cheng Zhikang, et al. Super-resolution reconstruction of infrared image based on channel attention and transfer learning[J]. Opto-Electronic Engineering, 2020, 48(1): 200045.