Sun R, Kan J S, Wu L W, et al. Rotating invariant face detection via cascaded networks and pyramidal optical flows[J]. Opto-Electron Eng, 2020, 47(1): 190135. doi: 10.12086/oee.2020.190135
Citation: Sun R, Kan J S, Wu L W, et al. Rotating invariant face detection via cascaded networks and pyramidal optical flows[J]. Opto-Electron Eng, 2020, 47(1): 190135. doi: 10.12086/oee.2020.190135

Rotating invariant face detection via cascaded networks and pyramidal optical flows

    Fund Project: Supported by National Natural Science Foundation of China (61471154) and Fundamental Research Funds for Central Universities (JZ2018YYPY0287)
More Information
  • In the unconstrained open-space, face detection is still a challenging task due to the facial posture changes, complex background environment, and motion blur. The rotation-invariant algorithm based on cascaded network and pyramid optical flow is proposed. Firstly, the cascading progressive convolutional neural network is adopted to locate the face position and facial landmark of the previous frame in the video stream. Secondly, the independent facial landmark detection network is used to reposition the current frame, and the optical flow mapping displacement of the facial landmark between the two frames is calculated afterwards. Finally, the detected face is corrected by the mapping relationship between the optical flow displacement of the facial landmark and the bounding box, thereby completing the rotation-invariant face detection. The experiment was tested on the FDDB public datasets, which proved that the method is more accurate. Moreover, the dynamic test on the Boston head tracking database proves that the face detection algorithm can effectively solve the problem of rotation-invariant face detection. Compared with other detection algorithms, the detection speed of the proposed algorithm has a great advantage, and the window jitter problem in the video is well solved.
  • 加载中
  • [1] Viola P, Jones M. Rapid object detection using a boosted cascade of simple features[C]//Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001.

    Google Scholar

    [2] Li B, Yang A M, Yang J. Rotated face detection using AdaBoost[C]//Proceedings of 2010 2nd International Conference on Information Engineering and Computer Science, 2010: 1-4.

    Google Scholar

    [3] Froba B, Ernst A. Face detection with the modified census transform[C]//Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004: 91-96.

    Google Scholar

    [4] Jin H L, Liu Q S, Lu H Q, et al. Face detection using improved LBP under Bayesian framework[C]//Proceedings of the Third International Conference on Image and Graphics, 2004: 306-309.

    Google Scholar

    [5] Farfade S S, Saberian M J, Li L J. Multi-view face detection using deep convolutional neural networks[C]//Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, 2015: 643-650.

    Google Scholar

    [6] Ranjan R, Patel V M, Chellappa R. A deep pyramid deformable part model for face detection[C]//Proceedings of 2015 IEEE 7th International Conference on Biometrics Theory, Applications and Systems, 2015.

    Google Scholar

    [7] Yang S, Luo P, Loy C C, et al. From facial parts responses to face detection: a deep learning approach[C]//Proceedings of 2015 IEEE International Conference on Computer Vision, 2015.

    Google Scholar

    [8] Bas A, Huber P, Smith W A P, et al. 3D morphable models as spatial transformer networks[C]//Proceedings of 2017 IEEE International Conference on Computer Vision Workshops, 2017.

    Google Scholar

    [9] 李小薪, 梁荣华.有遮挡人脸识别综述:从子空间回归到深度学习[J].计算机学报, 2018, 41(1): 177-207.

    Google Scholar

    Li X X, Liang R H. A review for face recognition with occlusion: from subspace regression to deep learning[J]. Chinese Journal of Computers, 2018, 41(1): 177-207.

    Google Scholar

    [10] Ren S Q, He K M, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems, 2015: 91-99.

    Google Scholar

    [11] Liu W, Anguelov D, Erhan D, et al. Single shot MultiBox detector[C]//Proceedings of the 14th European Conference on Compute Vision (ECCV), 2016: 21-37.

    Google Scholar

    [12] Redmon J, Divvala S, Girshick R, et al. You only look once: unified, real-time object detection[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016.

    Google Scholar

    [13] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[C]//Proceedings of the 3rd International Conference on Learning Representations, 2015.

    Google Scholar

    [14] Li H X, Lin Z, Shen X H, et al. A convolutional neural network cascade for face detection[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition, 2015: 5325-5334.

    Google Scholar

    [15] 潘榕, 魏慧琴.基于旋转不变性的人脸定位识别研究[J].计算机工程与设计, 2009, 30(8): 1941-1943, 1997.

    Google Scholar

    Pan R, Wei H Q. Research on human face detection and recognition based on rotation invariance[J]. Computer Engineering and Design, 2009, 30(8): 1941-1943, 1997.

    Google Scholar

    [16] 王炜强, 张晓阳, 曹春芹, 等.尺度不变单样本人脸识别方法[J].中国图象图形学报, 2012, 17(3): 380-386.

    Google Scholar

    Wang W Q, Zhang X Y, Gao C Q, et al. Scale invariant face recognition from single sample[J]. Journal of Image and Graphics, 2012, 17(3): 380-386.

    Google Scholar

    [17] 包晓安, 胡玲玲, 张娜, 等.基于级联网络的快速人脸检测算法[J].浙江理工大学学报, 2019, 41(3): 347-353. doi: 10.3969/j.issn.1673-3851(n).2019.03.011

    CrossRef Google Scholar

    Bao X A, Hu L L, Zhang N, et al. Fast face detection algorithm based on cascade network[J]. Journal of Zhejiang Sci-Tech University, 2019, 41(3): 347-353. doi: 10.3969/j.issn.1673-3851(n).2019.03.011

    CrossRef Google Scholar

    [18] 刘伟强.基于级联卷积神经网络的人脸检测算法的研究[D].厦门: 厦门大学, 2017.

    Google Scholar

    Liu W Q. Research on face detection algorithm based on cascaded convolutional neural networks[D]. Xiamen: Xiamen University, 2017.http://cdmd.cnki.com.cn/Article/CDMD-10384-1017120170.htm

    Google Scholar

    [19] 孙康, 李千目, 李德强.基于级联卷积神经网络的人脸检测算法[J].南京理工大学学报, 2018, 42(1): 40-47.

    Google Scholar

    Sun K, Li Q M, Li D Q. Face detection algorithm based on cascaded convolutional neural network[J]. Journal of Nanjing University of Science and Technology, 2018, 42(1): 40-47.

    Google Scholar

    [20] 林露樾.融合卷积神经网络以及光流法的目标跟踪方法[D].广州: 广东工业大学, 2018.

    Google Scholar

    Lin L Y. A visual object tracking method via CNN and optical flow with online learning[D]. Guangzhou: Guangdong University of Technology, 2018.

    Google Scholar

    [21] 王正来, 黄敏, 朱启兵, 等.基于深度卷积神经网络的运动目标光流检测方法[J].光电工程, 2018, 45(8): 38-47. doi: 10.12086/oee.2018.180027

    CrossRef Google Scholar

    Wang Z L, Huang M, Zhu Q B, et al. The optical flow detection method of moving target using deep convolution neural network[J]. Opto-Electronic Engineering, 2018, 45(8): 38-47. doi: 10.12086/oee.2018.180027

    CrossRef Google Scholar

    [22] Zhang K P, Zhang Z P, Li Z F, et al. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters, 2016, 23(10): 1499-1503. doi: 10.1109/LSP.2016.2603342

    CrossRef Google Scholar

    [23] Yang S, Luo P, Loy C C, et al. WIDER FACE: a face detection benchmark[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016.

    Google Scholar

    [24] Liu Z W, Luo P, Wang X G, et al. Deep learning face attributes in the wild[C]//Proceedings of 2015 IEEE International Conference on Computer Vision, 2015: 3730-3738.

    Google Scholar

    [25] Sun Y, Wang X G, Tang X O. Deep convolutional network cascade for facial point detection[C]//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition, 2013: 3476-3483.

    Google Scholar

    [26] Köstinger M, Wohlhart P, Roth P M, et al. Annotated facial landmarks in the wild: a large-scale, real-world database for facial landmark localization[C]//Proceedings of 2011 IEEE International Conference on Computer Vision Workshops, 2011: 2144-2151.

    Google Scholar

    [27] Jain V, Learned-Miller E G. FDDB: A benchmark for face detection in unconstrained settings[R]. UMass Amherst Technical Report, 2010.

    Google Scholar

    [28] Cascia M L, Sclaroff S. Fast, reliable head tracking under varying illumination[C]// Proceedings of 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1999: 604-610.

    Google Scholar

    [29] Wang H, Li Z F, Ji X, et al. Face R-CNN[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017.

    Google Scholar

  • Overview: In recent years, with the rapid deployment of video surveillance in urban space, the public security department has collected video in a massive unconstrained open environment. There are complex problems such as scale change, partial occlusion, motion blur and illumination change in the face detection of video stream. In particular, face rotation affects the performance and efficiency of the entire face recognition system. In this paper, the in-plane rotation problem of face detection in video stream is combined with the pyramid optical flow, and a rotating invariant face detection algorithm based on cascaded network and pyramid optical flow is proposed. Firstly, the cascading progressive convolutional neural network is used to locate the face position and facial landmark of the previous frame in the video stream. Secondly, the optical flow mapping between the facial landmark and the bounding box is obtained, and the independent facial landmark network is used to detect the current frame. After that, the optical flow displacement of the key points between the two frames is calculated. Finally, the detected face of the video is corrected by the mapping relationship between the optical flow displacement of the key point and the face candidate frame, thereby completing the rotation-invariant face detection. The experiments were tested on the FDDB public datasets. The ROC curve on the FDDB evaluates the performance of the face detection method. When the number of false positives is less than 160, the performance of our method is better than other methods. When the number of false positives is more than 160, the face detection result is close to Face R-CNN, which proves that the method has higher accuracy. Moreover, the dynamic test on the Boston head tracking database proves that the face detection algorithm can effectively solve the problem of rotation and scale change of the target area in the plane. The speed of this algorithm with other rotationally invariant face detectors on standard.mp4 video is compared. The minimum face size of these images is 100×100. The experimental video has a uniform length of 10 s, a frame rate of 30 frames/s, and a picture size of 640×480. Experiments show that the algorithm detection speed has a great advantage, and the window jitter problem in the video is well solved. The average detection rate of the algorithm in this paper is higher than the general video frame rate, and the model size is small, which is suitable for mobile devices. Time costs are greatly reduced compared to the methods of learning rotational invariant features and segmenting samples by highly complex networks.

  • 加载中
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Figures(9)

Tables(1)

Article Metrics

Article views(6677) PDF downloads(2652) Cited by(0)

Access History

Other Articles By Authors

Article Contents

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint