In order to meet the speed and accuracy requirements of face key point detection (face alignment) in application scenarios, firstly, cascaded prediction is carried out on the basis of SSD (single shot multibox detector), which combines more uniformly distributed feature layers to form MR-SSD (more robust SSD), a deep learning detector with more robust response to multi-scale faces. Secondly, based on the cascade shape regression method of local binary feature (LBF), a multi-angle initialization algorithm based on the difference between the facial pixels is proposed. Five groups of feature points in the 90 degree inclination range of positive and negative face are initialized to achieve excellent fitting effect for inclined face under multi angles. The mean square deviation of each group of feature points after regression is calculated and the maximum corresponding shape is used as the final regression shape. The optimal architecture proposed in this paper can obtain robust face bounding box and face alignment schemes against multi-angle tilt in real time.
Multi-angle key point detection of face based on deep learning detector
First published at:Feb 19, 2020
1 Wang Y M, Pan G, Wu Z H. A survey of 3D face recognition[J]. Journal of Computer-Aided Design & Computer Graphics, 2008, 20(7): 819–829. DOI:10.3745/JIPS.2009.5.2.041
2 Peng M C, Bao J, Ye M, et al. Face alignment algorithm based on shape parameter regression[J]. Pattern Recognition and Artificial Intelligence, 2016, 29(1): 63–71. DOI:10.16451/j.cnki.issn1003-6059.201601008
3 Zhu C R, Wang R S. Adaptive facial feature selection algorithm[J]. Journal of Computer-Aided Design & Computer Graphics, 2002, 14(1): 26–30. DOI:10.3321/j.issn:1003-9775.2002.01.007
4 Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014: 580–587.
5 Girshick R. Fast R-CNN[C]//Proceedings of 2015 IEEE International Conference on Computer Vision (ICCV), 2015: 1440–1448.
6 Ren S Q, He K M, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[C]//Advances in Neural Information Processing Systems, 2015: 91–99.
7 Uijlings J R R, van de Sande K E A, Gevers T, et al. Selective search for object recognition[J]. International Journal of Computer Vision, 2013, 104(2): 154–171.
8 Liu W, Anguelov D, Erhan D, et al. SSD: single shot multibox detector[C]//Proceedings of the 14thEuropean Conference on Computer Vision, 2016: 21–37.
9 Redmon J, Divvala S, Girshick R, et al. You only look once: unified, real-time object detection[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016: 779–788.
10 Cao X D, Wei Y C, Wen F, et al. Face alignment by explicit shape regression[J]. International Journal of Computer Vision, 2014, 107(2): 177–190. DOI:10.11772/j.issn.1001-9081.201711
11 Xiong X H, De la Torre F. Supervised descent method and its applications to face alignment[C]//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition, 2013: 532–539.
12 Ren S Q, Cao X D, Wei Y C, et al. Face alignment at 3000 FPS via regressing local binary features[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014: 1685–1692.
13 Li Z D, Zhong Y, Chen M, et al. PNMS algorithm based on penalty factors for face detection and alignment[J]. Advanced Engineering Sciences, 2018, 50(6): 225–231. DOI:10.15961/j.jsuese.201701086
李振东, 钟勇, 陈蔓, 等.基于惩罚因子的PNMS算法的人脸检测和对齐[J].工程科学与技术, 2018, 50(6): 225–231. DOI:10.15961/j.jsuese.201701086
14 Zhang K P, Zhang Z P, Li Z F, et al. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters, 2016,23(10): 1499–1503. DOI:10.1109/LSP.2016.2603342
15 Jiao F, Shan S G, Cui G Q, et al. Face recognition based on local feature analysis[J]. Journal of Computer-Aided Design & Computer Graphics, 2003, 15(1): 53–58.
16 Song H, Shi F. Multi-view face detection and pose discrimination in video[J]. Journal of Computer-Aided Design & Computer Graphics, 2007, 19(1): 90–95. DOI:10.3321/j.issn:1003-9775.2007.01.017
17 Zhang S F, Zhu X Y, Lei Z, et al. S3FD: single shot scale-invariant face detector[C]//Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV), 2017: 192–201.
18 Wan J, Li J, Chang J, et al. Face alignment on local-shape-based combined model[J]. Chinese Journal of Computers, 2018, 41(9): 2162–2174. DOI:10.11897/SP.J.1016.2018.02162
19 Bodini M. A review of facial landmark extraction in 2D images and videos using deep learning[J]. Big Data and Cognitive Computing, 2019, 3(1): 14. DOI:10.3390/bdcc3010014
National Natural Science Foundation of China (51777049) and Youth Science Foundation (51707051)
Get Citation: Zhao Xingwen, Hang Lijun, Gong Enlai, et al. Multi-angle key point detection of face based on deep learning detector[J]. Opto-Electronic Engineering, 2020, 47(1): 190299.
Previous: Anti-occlusion and re-tracking of real-time moving target based on kernelized correlation filter