Zhou Pengwei, Ji Yuanji, Dong Chao, et al. Video stabilization technique based on optical flow sensor[J]. Opto-Electronic Engineering, 2019, 46(11): 180581. doi: 10.12086/oee.2019.180581
Citation: Zhou Pengwei, Ji Yuanji, Dong Chao, et al. Video stabilization technique based on optical flow sensor[J]. Opto-Electronic Engineering, 2019, 46(11): 180581. doi: 10.12086/oee.2019.180581

Video stabilization technique based on optical flow sensor

    Fund Project: Supported by National Natural Science Foundation of China (61505192), Natural Science Foundation of Zhejiang Province (LQ15F050004, LY20F050008), and Open Foundation of Key Laboratory of Technology and Application for Safeguarding of Marine Rights and Interests, SOA (1705)
More Information
  • In allusion of the video jitter problem caused by platform motion, a video stabilization technique based on optical flow sensor is presented. Firstly, the scheme improves the general optical flow sensor to output accurate motion vectors under rotational motion, then motion vectors between adjacent frames are obtained by using the optical flow sensor. The real-time translation and rotation information of the main camera are calculated through coordinate transformation. Secondly, the method compensates the motion of video sequences to attain stable video sequences, and finally realizes video stabilization. Experimental results indicate that, compared with the unstable image, the peak signal-to-noise ratio (PSNR) is increased by 11.86 dB. In the case of obvious video jitter, the scheme can significantly reduce the jitter between video sequences. The method which has the characteristics of salutary video stabilization can meet the performance requirements of video stabilization and improve the capacity of disturbance resistance for platform.
  • 加载中
  • [1] 刘文.船载移动视频电子稳像方法研究[D].大连: 大连海事大学, 2017.

    Google Scholar

    Liu W. Research on the methods of electronic image stabilization for shipborne mobile video[D]. Dalian: Dalian Maritime University, 2017.http://cdmd.cnki.com.cn/Article/CDMD-10151-1017209269.htm

    Google Scholar

    [2] 张贝.基于特征提取与灰度投影联合稳像的算法研究[D].哈尔滨: 哈尔滨工程大学, 2010.

    Google Scholar

    Zhang B. Research on united image stabilization based on feature tracking and gray projection[D]. Harbin: Harbin Engineering University, 2010.

    Google Scholar

    [3] 冯筠, 延瑜瑜, 赵妍, 等.基于学习不变特征变换的兵马俑图像分区匹配[J].光学 精密工程, 2018, 26(7): 1774–1783.

    Google Scholar

    Feng J, Yan Y Y, Zhao Y, et al. A terracotta image partition matching method based on learned invariant feature transform[J]. Optics and Precision Engineering, 2018, 26(7): 1774–1783.

    Google Scholar

    [4] 曾峦, 翟优, 谭久彬.基于SIFT的自动匹配策略[J].光电工程, 2011, 38(2): 65–70. doi: 10.3969/j.issn.1003-501X.2011.02.011

    CrossRef Google Scholar

    Zeng L, Zhai Y, Tan J B. Automatic matching strategy based on SIFT[J]. Opto-Electronic Engineering, 2011, 38(2): 65–70. doi: 10.3969/j.issn.1003-501X.2011.02.011

    CrossRef Google Scholar

    [5] Honegger D, Meier L, Tanskanen P, et al. An open source and open hardware embedded metric optical flow CMOS camera for indoor and outdoor applications[C]//2013 IEEE International Conference on Robotics and Automation, Karlsruhe, 2013: 1736–1741.

    Google Scholar

    [6] 龙达峰, 刘俊, 张晓明, 等.基于椭球拟合的三轴陀螺仪快速标定方法[J].仪器仪表学报, 2013, 34(6): 1299–1305. doi: 10.3969/j.issn.0254-3087.2013.06.015

    CrossRef Google Scholar

    Long D F, Liu J, Zhang X M, et al. Triaxial gyroscope fast calibration method based on ellipsoid fitting[J]. Chinese Journal of Scientific Instrument, 2013, 34(6): 1299–1305. doi: 10.3969/j.issn.0254-3087.2013.06.015

    CrossRef Google Scholar

    [7] 纪慧中, 贾大宇, 董恩清, 等.基于图像特征和光流场的非刚性图像配准[J].光学 精密工程, 2017, 25(9): 2469–2482.

    Google Scholar

    Ji H Z, Jia D Y, Dong E Q, et al. Non-rigid registrations based on image characteristics and optical flows[J]. Optics and Precision Engineering, 2017, 25(9): 2469–2482.

    Google Scholar

    [8] Lan J H, Li J, Hu G D, et al. Vehicle speed measurement based on gray constraint optical flow algorithm[J]. Optik, 2014, 125(1): 289–295. doi: 10.1016/j.ijleo.2013.06.036

    CrossRef Google Scholar

    [9] Huang R M, Li J Q, Shi M Y, et al. Quadrotor hovering scheme based on improved optical flow[C]//2017 Chinese Automation Congress (CAC), Jinan, 2017: 2357–2362.

    Google Scholar

    [10] Tsai S E, Zhuang S H. Optical flow sensor integrated navigation system for quadrotor in GPS-denied environment[C]//2016 International Conference on Robotics and Automation Engineering (ICRAE), Jeju, 2016: 87–91.

    Google Scholar

    [11] 熊晶莹.基于特征提取与匹配的车载电子稳像方法研究[D].长春: 中国科学院长春光学精密机械与物理研究所, 2017.

    Google Scholar

    Xiong J Y. Study of electronic image stabilization based on feature detection and description for vehicles[D]. Changchun: Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, 2017.http://cdmd.cnki.com.cn/Article/CDMD-80139-1017064025.htm

    Google Scholar

    [12] 郑晓锋, 陈跃庭, 徐之海, 等.一种补偿平移与旋转运动的快速电子稳像算法[J].光子学报, 2008, 37(9): 1890–1894.

    Google Scholar

    Zheng X F, Chen Y T, Xu Z H, et al. A fast electronic image stabilization algorithm for translational and rotational motion compensation[J]. Acta Photonica Sinica, 2008, 37(9): 1890–1894.

    Google Scholar

    [13] 吕高杰, 张国华, 车宏.电子稳像算法精度评价方法研究[J].电光与控制, 2011, 18(3): 76–79. doi: 10.3969/j.issn.1671-637X.2011.03.019

    CrossRef Google Scholar

    Lv G J, Zhang G H, Che H. A method for evaluating the accuracy of electronic image stabilization system and algorithm[J]. Electronics Optics & Control, 2011, 18(3): 76–79. doi: 10.3969/j.issn.1671-637X.2011.03.019

    CrossRef Google Scholar

  • Overview: Video is one of the most intuitive and effective information carrier in the field of machine vision. Through the analysis of video information, the machine can perceive the surrounding environment and make relevant decisions. However, in the field of camera platform, such as shipborne, on-vehicle and hand-held, the bumping and jitter of the camera system which is due to the attitude change and vibration of the platform will cause the instability of the video sequences. It will seriously affect the information collection and decision for surrounding environment situation. Therefore, the video stabilization has important significance and value in practical engineering applications. The traditional video stabilization technique has mechanical image stabilization, optical image stabilization and electronic image stabilization. Mechanical image stabilization detects motion information and compensates the offset of video sequences by motor calibration system; optical image stabilization achieves image stabilization by adjusting the structure of the optical element to change the optical path and compensate the offset of the optical axis. The first two image stabilization technique are often expensive, difficult, and complicated to operate, which limits their application in small-sized and low-cost field of camera. The electronic image stabilization technique obtains the motion vectors of the video sequences by motion analysis and compensates the motion to obtain stable video sequences. However, this method tends to have a large amount of calculation and low precision, which limits its application range. In allusion of the above video stabilization problem, a video stabilization technique based on optical flow sensor is presented. Firstly, the scheme makes the general optical flow sensor overcome the shortcomings of the SAD algorithm under rotational motion and output accurate motion vectors. In addition, since the main camera and the optical flow sensor have a fixed spatial positional relationship, the real-time translation and rotation information of the main camera is calculated through coordinate transformation with the motion vectors obtained by the optical flow sensor. Then, the original unstable video sequences are compensated by frame, and finally the video stabilization is realized. Experimental results indicate that, compared with the unstable image, the peak signal-to-noise ratio (PSNR) is increased by 11.86 dB, and compared with the electronic image stabilization algorithm based on feature point matching, the processing time is decreased by 54.65% with similar image stabilization effect. In the case of obvious video jitter, the scheme can significantly reduce the jitter between video sequences. The method which has the characteristics of salutary video stabilization and high speed can meet the performance requirements of video stabilization and improve the capacity of disturbance resistance for platform.

  • 加载中
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Figures(9)

Tables(2)

Article Metrics

Article views(7466) PDF downloads(2413) Cited by(0)

Access History

Other Articles By Authors

Article Contents

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint