Research on Multi-feature Human Pose Model Recognition Based on One-shot Learning

Human body gestures play a very important role in human emotion expression and perceptual communication, and have broad application prospects  in the fields of security surveillance, motion analysis, human-computer interaction and virtual reality.   Therefore, gesture recognition has become one of the most active research topics in computer vision. Human posture recognition mainly uses computer technologies such as image processing and analysis, machine learning and pattern recognition to perform feature detection, feature extraction and classification, feature tracking, and feature calculation and recognition for action sequences, and to achieve machine perception and understanding of body intentions. Human posture recognition can be regarded as a classification of action sequence data, that is, the test action sequence is incorporated into a pre-trained specific action sequence, and the similarity between the two sequences is checked. The current research in this direction is mainly based on human three-dimensional bones and human action sequences. The rapid developing of bone detection and tracking technology, coupled with the non-contact somatosensory cameras and software development kits, which provide technology bases for gesture description and recognition.However, gesture recognition still suffer from problems such as dependence on underlying data, missing feature information, low recognition accuracy and poor robustness. Therefore, human body gesture recognition is still a difficult task.

The research team of Prof. Li Guoyou from Hebei Provincial Key Laboratory proposed a one-shot learning model matching algorithm based on the angle and distance features of human posture joints for gesture feature extraction and data set model construction. They mainly uses KinectV2 sensors to obtain the 3D coordinate information of key joint points, perform feature calibration on the obtained joint point information, and construct the angle and distance characteristics of the used joint points, and then convert the feature information of the posture model to construct the human posture sample library. The pose feature characterization has high combinability, low computational complexity, effectively reduces data redundancy, and efficiently utilizes bone information.

Figure 1  Feature extraction. (a)Angle feature. (b)Distance feature

The one-shot learning model predefines a network, maps the labeled support set and unlabeled test set to the labels of the network, then performs data set sampling, and decomposes the data set into different support sets and test sets. Then import the decomposed support set data into the training embedding function for repeated optimization, and at the same time import the test data into the test embedding function for optimization, and use the gradient descent method to train the support set and the test set parameters to learn the different situations of each pose Generalization ability and get the model of each pose in different situations; finally the model matching algorithm builds different encoders for the support set and sample set, and uses the LSTM recurrent neural network to solve the problem of gradient disappearance and explosion. Then the model classifier quickly generates existing tags for unknown poses and performs parameter conversion, and performs a weighted sum calculation to select the type of the sample with the highest fit as the recognition result. This work eliminates the interference of undefined posture, and improves the anti-interference, accuracy, real-time and robustness of human posture recognition.

Figure 2  Single sample learning model

Figure 3  Flow chart of gesture recognition

About The Group

The research team of Prof. Li Guoyou fromHebei Provincial Key Laboratory of Industrial Computer Control Engineering of Yanshan University was established in 2011. The team members include Li Chenguang, Wang Weijiang, Yang Mengqi, Hang Bingpeng and others. The main research interests include servo control, intelligent control, robot fuzzy vision, chemical DCS control and simulation, photovoltaic power generation system research, and artificial intelligence. Presently presides over the research of Hebei Provincial Department of Education "Research on Image-based Robot Fuzzy Visual Servo Control", Hebei Province Science and Technology Development Project "DCS Controlled Catalytic Cracking Unit Simulation System",etc. He has published more than 60 paperswith 32 indexed EI and 7 indexed in SCIE, and applied for more than 30 invention patents. Prof. Li has won the second prize of China Machinery Industry Science and Technology Progress.


Li Guoyou, Li Chenguang, Wang Weijiang, et al. Research on Multi-feature Human Pose Model Recognition Based on One-shot Learning [J]. Opto-Electronic Engineering, 2021, 48(2): 200099.