A cylindrical image mosaic method based on fast camera calibration in multi-scene is proposed to solve the problems of scene limitation and complex calibration process in image mosaic using camera calibration parameter. Firstly, the accurate corner feature of checkerboard calibration board is used to make it in the overlapping field of view of two adjacent images. Then, the image sequence is pre-processed by corner extraction, precision and matching, so that the registration parameters between the images to be stitched can be solved accurately and quickly. After that, the cylindrical projection is used to maintain the visual consistency of the images, and the multi-band fusion is used to retain the details of the images. Subsequently, the images are stitched using registration parameters obtained by calibration. Finally, the whole system is built on a low-power embedded platform to accomplish fast calibration and mosaic process based on calibration parameters in multi-scene. The experiment results show that the proposed method can accomplish camera calibration quickly and accurately in indoor and tunnel scenarios, and the image mosaic process is time-consuming. Meanwhile, it can ensure better stitching accuracy and imaging effect, and has strong robustness.
Cylindrical image mosaic method based on fast camera calibration in multi-scene
First published at:Apr 13, 2020
1 Adel E, Elmogy M, Elbakry H. Image stitching based on feature extraction techniques: a survey[J]. International Journal of Computer Applications, 2014, 99(6): 1-8. DOI:10.5120/17374-7818
2 Cheng Y F, Jin S Y, Wang M, et al. Image mosaicking approach for a double-camera system in the GaoFen2 optical remote sensing satellite based on the big virtual camera[J]. Sensors, 2017, 17(6): E1441. DOI:10.3390/s17061441
3 Bosch J, Gracias N, Ridao P, et al. Omnidirectional underwater camera design and calibration[J]. Sensors, 2015, 15(3): 6033-6065. DOI:10.3390/s150306033
4 Chen C T, Pan Z W, Shen H L, et al. Image stitching and partitioning algorithms for infrared thermal human-body images[J]. Opto-Electronic Engineering, 2019, 46(9): 180689. DOI:10.12086/oee.2019.180689
陈晨涛, 潘之玮, 沈会良, 等.一种人体热红外图像拼接及部位划分方法[J].光电工程, 2019, 46(9): 180689. DOI:10.12086/oee.2019.180689
5 Kaynig V, Fischer B, Müller E, et al. Fully automatic stitching and distortion correction of transmission electron microscope images[J]. Journal of Structural Biology, 2010, 171(2): 163-173. DOI:10.1016/j.jsb.2010.04.012
6 Wu L P, Hu Y. Virtual reality technology of image smoothing in cylindrical panoramic image mosaic[J]. Science Technology and Engineering, 2017, 17(31): 271-276. DOI:10.3969/j.issn.1671-1815.2017.31.044
吴丽萍, 胡郁.柱面全景图图像拼接中图像平滑的虚拟现实技术[J].科学技术与工程, 2017, 17(31): 271-276. DOI:10.3969/j.issn.1671-1815.2017.31.044
7 Seo S, Jeonz S, Lee S. Efficient homography estimation method for panorama[C]//Proceedings of the 19th Korea-Japan Joint Workshop on Frontiers of Computer Vision, 2013: 209-212.
8 Szeliski R. Video mosaics for virtual environments[J]. IEEE Computer Graphics and Applications, 1996, 16(2): 22-30. DOI:10.1109/38.486677
9 Zou L H, Chen J, Zhang J, et al. An image mosaicing approach for video sequences based on space-time manifolds[C]//Proceedings of the 29th Chinese Control Conference, 2010.
10 Li L N, Geng N. Algorithm for sequence image automatic mosaic based on sift feature[C]//Proceedings of 2010 Wase International Conference on Information Engineering, 2010.
11 Wu Z W, Zhu L R, Sun X C. Expansion of the visual angle of a car rear-view image via an image mosaic algorithm[J]. Optical Engineering, 2015, 54(5): 053101. DOI:10.1117/1.OE.54.5.053101
12 Zhou A W, Shao W, Guo J J. An image mosaic method for defect inspection of steel rotary parts[J]. Journal of Nondestructive Evaluation, 2016, 35(4): 60. DOI:10.1007/s10921-016-0375-3
13 Alomran M, Chai D. Feature-based panoramic image stitching[C]//Proceedings of 2016 14th International Conference on Control, 2016.
14 Zhang J L, Sun H X, Jia Q X, et al. Image mosaic algorithm based on camera calibration[J]. Journal of North University of China (Natural Science Edition), 2008, 29(6): 575-579. DOI:10.3969/j.issn.1673-3193.2008.06.021
张金玲, 孙汉旭, 贾庆轩, 等.基于相机标定的图像拼接算法[J].中北大学学报(自然科学版), 2008, 29(6): 575-579. DOI:10.3969/j.issn.1673-3193.2008.06.021
15 Wang D, Liu F Y, Chen T E, et al. The method of cylinder panoramic image mosaic based on camera calibration parameters[J]. Science of Surveying and Mapping, 2016, 41(7): 150-154, 143.
16 Ma J L, Zhang J M, Sun W X. Research on panoramic image mosaic method based on camera calibration[J]. Journal of System Simulation, 2017, 29(5): 1112-1119.
17 Zhang Z Y. Flexible camera calibration by viewing a plane from unknown orientations[C]//Proceedings of the Seventh IEEE International Conference on Computer Vision, 1999.
18 Hartley R, Zisserman A. Multiple view geometry in computer vision[M]. Cambridge university press, 2003
19 Chuan Z, Long T D, Feng Z, et al. A planar homography estimation method for camera calibration[C]//Proceedings 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation. Computational Intelligence in Robotics and Automation for the New Millennium, 2003.
20 Lenth R V. Least-squares means: the R package lsmeans[J]. Journal of Statistical Software, 2016, 69(1): 1-33.
21 Janota A, Šimák V, Nemec D, et al. Improving the precision and speed of euler angles computation from low-cost rotation sensor data[J]. Sensors, 2015, 15(3): 7016-7039. DOI:10.3390/s150307016
22 Pulli K, Baksheev A, Kornyakov K, et al. Real-time computer vision with OpenCV[J]. Communications of the ACM, 2012, 55(6): 61-69. DOI:10.1145/2184319.2184337
23 Xu Y, Zhou Q H, Gong L W, et al. High-speed simultaneous image distortion correction transformations for a multicamera cylindrical panorama real-time video system using FPGA[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(6): 1061-1069. DOI:10.1109/TCSVT.2013.2290576
24 Guo H X, Guo H L, Xie K. Improved bilinear interpolation algorithm based on edge information[J]. Computer Engineering and Applications, 2011, 47(31): 171-174. DOI:10.3778/j.issn.1002-8331.2011.31.049
郭海霞, 郭海龙, 解凯.基于边缘信息改进的双线性插值算法[J].计算机工程与应用, 2011, 47(31): 171-174. DOI:10.3778/j.issn.1002-8331.2011.31.049
25 Guo Z C, Dang J W, Wang Y P, et al. Background modeling method based on multi-feature fusion[J]. Opto-Electronic Engineering, 2018, 45(12): 180206. DOI:10.12086/oee.2018.180206
郭治成, 党建武, 王阳萍, 等.基于多特征融合的背景建模方法[J].光电工程, 2018, 45(12): 180206. DOI:10.12086/oee.2018.180206
Supported by National Natural Science Foundation of China (51805280), the Public Technology Application Project of Zhejiang (2017C31094), and Natural Science Foundation of Zhejiang (LQ18E050005)
Get Citation: Fu Ziqiu, Zhang Xiaolong, Yu Chen, et al. Cylindrical image mosaic method based on fast camera calibration in multi-scene[J]. Opto-Electronic Engineering, 2020, 47(4): 190436.