Citation: | Wang Zhenglai, Huang Min, Zhu Qibing, et al. The optical flow detection method of moving target using deep convolution neural network[J]. Opto-Electronic Engineering, 2018, 45(8): 180027. doi: 10.12086/oee.2018.180027 |
[1] | 黄凯奇, 任伟强, 谭铁牛.图像物体分类与检测算法综述[J].计算机学报, 2014, 37(6): 1225-1240. Huang K Q, Ren W Q, Tan T N. A review on image object classification and detection[J]. Chinese Journal of Computers, 2014, 37(6): 1225-1240. |
[2] | 卢宏涛, 张秦川.深度卷积神经网络在计算机视觉中的应用研究综述[J].数据采集与处理, 2016, 31(1): 1-17. Lu H T, Zhang Q C. Applications of deep convolutional neural network in computer vision[J]. Journal of Data Acquisition and Processing, 2016, 31(1): 1-17. |
[3] | Ren S Q, He K M, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1137. doi: 10.1109/TPAMI.2016.2577031 |
[4] | Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition, 2015: 3431-3440. |
[5] | 王顺飞, 闫钧华, 王志刚.改进的基于局部联合特征的运动目标检测方法[J].仪器仪表学报, 2015, 36(10): 2241-2248. doi: 10.3969/j.issn.0254-3087.2015.10.011 Wang S F, Yan J H, Wang Z G. Improved moving object detection algorithm based on local united feature[J]. Chinese Journal of Scientific Instrument, 2015, 36(10): 2241-2248. doi: 10.3969/j.issn.0254-3087.2015.10.011 |
[6] | 刘威, 赵文杰, 李成, 等.基于改进ORB特征匹配的运动小目标检测[J].光电工程, 2015, 42(10): 13-20. doi: 10.3969/j.issn.1003-501X.2015.10.003 Liu W, Zhao W J, Li C, et al. Detecting small moving target based on the improved ORB feature matching[J]. Opto-Electronic Engineering, 2015, 42(10): 13-20. doi: 10.3969/j.issn.1003-501X.2015.10.003 |
[7] |
黄超群. 基于混合高斯模型和Kalman滤波器的运动目标检测与跟踪[D]. 昆明: 云南大学, 2010.
Huang C Q. Motion detection and tracking based on gaussian mixture model and kalman filter[D]. Kunming: Yunnan University, 2010. |
[8] | 张金敏, 王斌.光照快速变化条件下的运动目标检测[J].光电工程, 2016, 43(2): 14-21. doi: 10.3969/j.issn.1003-501X.2016.02.003 Zhang J M, Wang B. Moving object detection under condition of fast illumination change[J]. Opto-Electronic Engineering, 2016, 43(2): 14-21. doi: 10.3969/j.issn.1003-501X.2016.02.003 |
[9] | 袁国武, 陈志强, 龚健, 等.一种结合光流法与三帧差分法的运动目标检测算法[J].小型微型计算机系统, 2013, 34(3): 668-671. Yuan G W, Chen Z Q, Gong J, et al. A moving object detection algorithm based on a combination of optical flow and three-frame difference[J]. Journal of Chinese Computer Systems, 2013, 34(3): 668-671. |
[10] | 罗胜, Jiang Y Z.视频检测烟雾的研究现状[J].中国图象图形学报, 2013, 18(10): 1225-1236. doi: 10.11834/jig.20131002 Luo S, Jiang Y Z. State-of-art of video based smoke detection algorithms[J]. Journal of Image and Graphics, 2013, 18(10): 1225-1236. doi: 10.11834/jig.20131002 |
[11] | Shi L F, Long F, Zhan Y J, et al. Video-based fire detection with spatio-temporal SURF and color features[C]//Proceedings of 2016 12th World Congress on Intelligent Control and Automation, 2016: 258-262. |
[12] | Dosovitskiy A, Fischery P, Ilg E, et al. Flownet: learning optical flow with convolutional networks[C]// Proceeding of 2015 IEEE International Conference on Computer Vision, 2015: 2758-2766. |
[13] | Ilg E, Mayer N, Saikia T, et al. Flownet 2. 0: evolution of optical flow estimation with deep networks[C]//Proceeding of 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1647-1655. |
[14] |
邓昊. 神经元图像识别相关方法探究[D]. 武汉: 湖北大学, 2016.
Deng H. Research on neuron image recognition[D]. Wuhan: Hubei University, 2016. |
[15] | Ouyang P, Hu H, Shi Z Z. Plankton classification with deep convolutional neural networks[C]//Proceeding of 2016 IEEE Information Technology, Networking, Electronic and Automation Control Conference, 2016: 132-136. |
[16] | He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceeding of 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778. |
[17] | He K M, Zhang X Y, Ren S Q, et al. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification[C]//Proceeding of 2015 IEEE International Conference on Computer Vision, 2015: 1026-1034. |
[18] | He K M, Sun J. Convolutional neural networks at constrained time cost[C]//Proceeding of 2015 IEEE Conference on Computer Vision and Pattern Recognition, 2015: 3992-4000. |
[19] | Horn B K P, Schunck B G. Determining optical flow[J]. Artificial Intelligence, 1981, 17(1-3): 185-203. doi: 10.1016/0004-3702(81)90024-2 |
[20] | Lucas B D, Kanade T. An iterative image registration technique with an application to stereo vision (DARPA)[C]//Proceedings of the 1981 DARPA Image Understanding Workshop, 1981: 121-130. |
Overview: Object detection is a hot research direction in the fields of pattern recognition and machine learning. It is used widely in areas such as security, traffic and the Internet. Computer realizes object detection automatically, which can reduce the burden of people in a certain extent. Moving target detection is an important research direction of object detection and plays an important role in target recognition. With the development of deep learning theory, the deep convolutional neural network has made remarkable achievements in the field of object detection by improving the generation strategy of candidate regions and by optimizing the network structure and training methods. Different from the usual definition, moving target detection is turned into focus on the detection of moving target that is needed rather than all of the moving targets. However, conventional convolution neural network cannot extract the dynamic characteristics of object and the accuracy of traditional motion detection methods is low and unable to only detect the required moving target. In this study, deep convolutional neural network is introduced into the optical flow detection of moving target. In this method, a pair of images and optical flow field of target are used as inputs of the network to adaptively study the target optical flow. Furthermore, through optimization of the expanding part of the network and the simplification of the network, and combined with many data augmentation technologies, the optical flow detection network of target object with both accuracy and real time is designed. Visualization technology is also applied to the network, which can directly compare the results with visual image. Experimental results show that the proposed method has better performance in the optical flow detection of moving target. The proposed network in this study retains the advantage of the deep convolutional neural network that only detect the optical flow of required target, which conforms to the practical application of object detection. Additionally, the proposed network achieves better performance in the both accuracy and the real-time of network. S-sp is improved by about 4.9% and 5.3%, respectively, on the AAE and AEE indicators compared to the original network, and SS-sp and CS-sp network are improved by about 5.0% on the AAE indicators compared to the original network. The runtime CS-sp and SS-sp network can be significantly reduced. The runtime of CS-sp network is reduced from about 179 ms to 34 ms and the runtime of the CS-sp network is reduced from 170 ms to 33 ms, which meet the requirements of real-time detection.
Network structure of FlowNet
Contracting structure. (a) FN-S contracting structure; (b) FN-C contracting structure
Expanding structure of FlowNet network
Structure of stack network
Part of experimental results 1. (a) Original image 1; (b) Original image 2; (c) Optical flow image of target; (d) Result of Horn-Schunck algorithm; (e) Result of Lucas-Kanade algorithm; (f) Result of CS-sp algorithm
Part of experimental results 2. (a) Original image 1; (b) Original image 2; (c) Optical flow image of target; (d) Result of Horn-Schunck algorithm; (e) Result of Lucas-Kanade algorithm; (f) Result of CS-sp algorithm