3D shape measurement and texture mapping method based on structured light projection

Xiang Zhuolong; Zhang Qican; Wu Zhoujie

doi:10.12086/oee.2022.220169

Article navigation > Opto-Electronic Engineering > 2022 Vol. 49 > No. 12 > 220169

Next Article Previous Article

Xiang Z L, Zhang Q C, Wu Z J. 3D shape measurement and texture mapping method based on structured light projection[J]. Opto-Electron Eng, 2022, 49(12): 220169. doi: 10.12086/oee.2022.220169

Citation:

Xiang Z L, Zhang Q C, Wu Z J. 3D shape measurement and texture mapping method based on structured light projection[J]. Opto-Electron Eng, 2022, 49(12): 220169. doi: 10.12086/oee.2022.220169

3D shape measurement and texture mapping method based on structured light projection

College of Electronic Information Engineering, Sichuan University, Chengdu, Sichuan 610065, China

Fund Project: National Natural Science Fundation of China (62075143)

More Information

Corresponding author: zqc@scu.edu.cn

Received Date 15 July 2022

Revised Date 01 September 2022

Accepted Date 05 September 2022

Available Online 29 November 2022

Published Date 25 December 2022

Abstract

Abstract

After obtaining three-dimensional (3D) point cloud data of a measured scene using the binocular structured light projection measurement system, three texture (grayscale and color) acquisition and mapping methods have been explored in this paper for different scenes. Under the condition of no additional color imaging equipment, two texture acquisition methods are proposed respectively. On the premise of using an additional texture camera, the method using free texture mapping by adding marker points is proposed. Then, to get rid of the dependence on marker points, the unconstrained free texture mapping method using the object’s feature information is presented. This paper proposes three kinds of practical and feasible solutions from 3D point cloud data to mapping texture for different applicable scenes, and the feasibility of the proposed methods is proved by experiments. The free texture mapping method based on a binocular structured light system provides a simple and easy means of color 3D information acquirement for the fields of cultural relics digitization and reverse engineering.
- three-dimensional imaging /
- texture mapping /
- binocular vision

FullText(HTML)

References

[1]	Li Y X, Qian J M, Feng S J, et al. Deep-learning-enabled dual-frequency composite fringe projection profilometry for single-shot absolute 3D shape measurement[J]. Opto-Electron Adv, 2022, 5(5): 210021. doi: 10.29026/oea.2022.210021 CrossRef Google Scholar
[2]	应晓霖, 姚建云, 张晓松, 等. 采用LD的光源步进条纹投影三维测量系统[J]. 光电工程, 2021, 48(11): 210298. doi: 10.12086/oee.2021.210298 CrossRef Google Scholar Ying X L, Yao J Y, Zhang X S, et al. Fringe projection based three-dimensional measurement system by the light-source-stepping method using LD[J]. Opto-Electron Eng, 2021, 48(11): 210298. doi: 10.12086/oee.2021.210298 CrossRef Google Scholar
[3]	Wu Z J, Guo W B, Zhang Q C. Two-frequency phase-shifting method vs. Gray-coded-based method in dynamic fringe projection profilometry: a comparative review[J]. Opt Lasers Eng, 2022, 153: 106995. doi: 10.1016/J.OPTLASENG.2022.106995 CrossRef Google Scholar
[4]	左超, 张晓磊, 胡岩, 等. 3D真的来了吗?−三维结构光传感器漫谈[J]. 红外与激光工程, 2020, 49(3): 0303001. doi: 10.3788/IRLA202049.0303001 CrossRef Google Scholar Zuo C, Zhang X L, Hu Y, et al. Has 3D finally come of age?——An introduction to 3D structured-light sensor[J]. Infrared Laser Eng, 2020, 49(3): 0303001. doi: 10.3788/IRLA202049.0303001 CrossRef Google Scholar
[5]	张启灿, 吴周杰. 基于格雷码图案投影的结构光三维成像技术[J]. 红外与激光工程, 2020, 49(3): 0303004. doi: 10.3788/IRLA202049.0303004 CrossRef Google Scholar Zhang Q C, Wu Z J. Three-dimensional imaging technique based on gray-coded structured illumination[J]. Infrared Laser Eng, 2020, 49(3): 0303004. doi: 10.3788/IRLA202049.0303004 CrossRef Google Scholar
[6]	张宗华, 于瑾, 高楠, 等. 高反光表面三维形貌测量技术[J]. 红外与激光工程, 2020, 49(3): 0303006. doi: 10.3788/IRLA202049.0303006 CrossRef Google Scholar Zhang Z H, Yu J, Gao N, et al. Three-dimensional shape measurement techniques of shiny surfaces[J]. Infrared Laser Eng, 2020, 49(3): 0303006. doi: 10.3788/IRLA202049.0303006 CrossRef Google Scholar
[7]	侯冠宇, 吴斌, 何荣芳, 等. 基于双目光栅重建和纹理映射的缺陷三维测量方法[J]. 光学学报, 2022, 42(7): 0712003. doi: 10.3788/AOS202242.0712003 CrossRef Google Scholar Hou G Y, Wu B, He R F, et al. Three-dimensional measurement method of defects based on binocular grating reconstruction and texture mapping[J]. Acta Opt Sin, 2022, 42(7): 0712003. doi: 10.3788/AOS202242.0712003 CrossRef Google Scholar
[8]	左超, 陈钱. 计算光学成像: 何来, 何处, 何去, 何从?[J]. 红外与激光工程, 2022, 51(2): 20220110. doi: 10.3788/IRLA20220110 CrossRef Google Scholar Zuo C, Chen Q. Computational optical imaging: an overview[J]. Infrared Laser Eng, 2022, 51(2): 20220110. doi: 10.3788/IRLA20220110 CrossRef Google Scholar
[9]	张广军. 视觉测量[M]. 北京: 科学出版社, 2008. Google Scholar Zhang G J. Vision Measurement[M]. Beijing: Science Press, 2008. Google Scholar
[10]	何进英, 刘晓利, 彭翔, 等. 基于灰度约束的三维数字散斑整像素相关搜索[J]. 中国激光, 2017, 44(4): 0404003. doi: 10.3788/CJL201744.0404003 CrossRef Google Scholar He J Y, Liu X L, Peng X, et al. Integer pixel correlation searching for three-dimensional digital speckle based on gray constraint[J]. Chin J Lasers, 2017, 44(4): 0404003. doi: 10.3788/CJL201744.0404003 CrossRef Google Scholar
[11]	赵涵卓, 高楠, 孟召宗, 等. 双视角三维测量系统同时标定方法[J]. 光电工程, 2021, 48(3): 200127. doi: 10.12086/oee.2021.200127 CrossRef Google Scholar Zhao H Z, Gao N, Meng Z Z, et al. Method of simultaneous calibration of dual view 3D measurement system[J]. Opto-Electron Eng, 2021, 48(3): 200127. doi: 10.12086/oee.2021.200127 CrossRef Google Scholar
[12]	Catmull E E. A subdivision algorithm for computer display of curved surfaces[D]. Salt Lake City: The University of Utah, 1974. Google Scholar
[13]	Blinn J F. A scan line algorithm for displaying parametrically defined surfaces[J]. ACM SIGGRAPH Comput Graph, 1978, 12(SI): 1−7. doi: 10.1145/988437.988439 CrossRef Google Scholar
[14]	Bier E A, Sloan K R. Two-part texture mappings[J]. IEEE Comput Graph Appl, 1986, 6(9): 40−53. doi: 10.1109/MCG.1986.276545 CrossRef Google Scholar
[15]	张宗华, 彭翔, 胡小唐. 一种新型彩色三维光学成像系统[J]. 光学学报, 2002, 22(8): 994−998. doi: 10.3321/j.issn:0253-2239.2002.08.022 CrossRef Google Scholar Zhang Z H, Peng X, Hu X T. A new color 3-D optical imaging system[J]. Acta Opt Sin, 2002, 22(8): 994−998. doi: 10.3321/j.issn:0253-2239.2002.08.022 CrossRef Google Scholar
[16]	孙士杰, 翟爱平, 曹益平. 一种快速获取物体三维形貌和纹理信息的算法[J]. 光学学报, 2016, 36(3): 0312001. doi: 10.3788/AOS201636.0312001 CrossRef Google Scholar Sun S J, Zhai A P, Cao Y P. A fast algorithm for obtaining 3D shape and texture information of objects[J]. Acta Opt Sin, 2016, 36(3): 0312001. doi: 10.3788/AOS201636.0312001 CrossRef Google Scholar
[17]	Liu Y Z, Fu Y J, Zhou P X, et al. A real-time 3D shape measurement with color texture using a monochromatic camera[J]. Opt Commun, 2020, 474: 126088. doi: 10.1016/j.optcom.2020.126088 CrossRef Google Scholar
[18]	Zhang S, Huang P S. High-resolution, real-time three-dimensional shape measurement[J]. Opt Eng, 2006, 45(12): 123601. doi: 10.1117/1.2402128 CrossRef Google Scholar
[19]	Ou P, Li B W, Wang Y J, et al. Flexible real-time natural 2D color and 3D shape measurement[J]. Opt Express, 2013, 21(14): 16736−16741. doi: 10.1364/OE.21.016736 CrossRef Google Scholar
[20]	刘星明, 刘晓利, 殷永凯, 等. 真实感三维模型的纹理融合[J]. 计算机辅助设计与图形学学报, 2012, 24(11): 1440−1446. doi: 10.3969/j.issn.1003-9775.2012.11.008 CrossRef Google Scholar Liu X M, Liu X L, Yin Y K, et al. Texture blending of 3D photo-realistic model[J]. J Comput-Aided Des Comput Graph, 2012, 24(11): 1440−1446. doi: 10.3969/j.issn.1003-9775.2012.11.008 CrossRef Google Scholar
[21]	杜瑞建, 葛宝臻, 陈雷. 多视高分辨率纹理图像与双目三维点云的映射方法[J]. 中国光学, 2020, 13(5): 1055−1064. doi: 10.37188/CO.2020-0034 CrossRef Google Scholar Du R J, Ge B Z, Chen L. Texture mapping of multi-view high-resolution images and binocular 3D point clouds[J]. Chin Opt, 2020, 13(5): 1055−1064. doi: 10.37188/CO.2020-0034 CrossRef Google Scholar
[22]	赵琳敬. 三维点云智能进化拼接与表面纹理映射方法研究[D]. 天津: 天津大学, 2018. doi: 10.27356/d.cnki.gtjdu.2018.000668. Google Scholar Zhao L J. Research on 3D point cloud intelligent evolutionary registration and surface texture mapping method[D]. Tianjin: Tianjin University, 2018. doi: 10.27356/d.cnki.gtjdu.2018.000668. Google Scholar
[23]	向卓龙, 张启灿, 陈超文. 利用自由拍摄二维图像实现三维点云的纹理贴图[J]. 激光与光电子学进展, 2021, 58(18): 1811018. doi: 10.3788/LOP202158.1811018 CrossRef Google Scholar Xiang Z L, Zhang Q C, Chen C W. Texture mapping of 3D point clouds with freely recorded 2D images[J]. Laser Optoelectron Prog, 2021, 58(18): 1811018. doi: 10.3788/LOP202158.1811018 CrossRef Google Scholar
[24]	Lepetit V, Moreno-Noguer F, Fua P. EPnP: an accurate O(n) solution to the PnP problem[J]. Int J Comput Vis, 2009, 81(2): 155−166. doi: 10.1007/s11263-008-0152-6 CrossRef Google Scholar

Overview

Overview

In the traditional optical 3D measurement method, the ultimate goal is to obtain the 3D shape information of the measured object, but the 3D data that ignores the texture information often makes the measurement scene lack realism. To make obtained point cloud information more realistic and create an immersive feeling, the texture mapping technique is introduced to attach color information to the reconstructed 3D point cloud. Texture mapping also faces some technical problems. The first thing is how to complete color texture mapping without a color camera. Second, on the premise of using a color camera for texture recording, how to freely move the texture camera to capture the texture images from different perspectives without frequent calibration, so as to complete the accurate mapping from 3D point cloud to texture images.

This paper discusses the above two problems, and proposes a fixed-view grayscale texture mapping method and a color texture method for the case of no additional texture camera; for the case of using a color texture camera, this paper also proposes a free texture mapping method and an unconstrained mapping method. The specific content of the paper is as follows:

1) Under the condition that there is no color camera for texture capturing, a color projector is used to project sinusoidal fringes with three frequencies from three channels of R, G, and B, respectively. The deformed fringe images are collected, and the periodic intensity distribution of three-frequency fringes is eliminated by averaging phase-shifting patterns respectively. And the corresponding texture mapping can be completed after combining textures in three channels and color correction.

2) On the premise of taking an additional texture camera, marker points are added in the measured field, and the mapping relationship between the 3D point cloud and the 2D texture image can be obtained from the camera imaging model. To further get rid of the constraints of adding markers, an unconstrained free texture mapping method is proposed for objects with rich textures. The idea is to perform feature matching between the object images captured by the left and right cameras. According to the corresponding relationship between feature matching points and the 3D point cloud, the PnP problem is solved to obtain the pose relationship for the establishment of the mapping relationship between the 3D point cloud and the 2D texture image and finally realizes texture mapping. Experiments have proved the feasibility of these two methods. The research fruits of this paper could provide a simple and easy means of color 3D information acquirement for the fields of cultural relics digitization and reverse engineering.