4 research outputs found
Gait recognition and understanding based on hierarchical temporal memory using 3D gait semantic folding
Gait recognition and understanding systems have shown a wide-ranging application prospect. However, their use of unstructured data from image and video has affected their performance, e.g., they are easily influenced by multi-views, occlusion, clothes, and object carrying conditions. This paper addresses these problems using a realistic 3-dimensional (3D) human structural data and sequential pattern learning framework with top-down attention modulating mechanism based on Hierarchical Temporal Memory (HTM). First, an accurate 2-dimensional (2D) to 3D human body pose and shape semantic parameters estimation method is proposed, which exploits the advantages of an instance-level body parsing model and a virtual dressing method. Second, by using gait semantic folding, the estimated body parameters are encoded using a sparse 2D matrix to construct the structural gait semantic image. In order to achieve time-based gait recognition, an HTM Network is constructed to obtain the sequence-level gait sparse distribution representations (SL-GSDRs). A top-down attention mechanism is introduced to deal with various conditions including multi-views by refining the SL-GSDRs, according to prior knowledge. The proposed gait learning model not only aids gait recognition tasks to overcome the difficulties in real application scenarios but also provides the structured gait semantic images for visual cognition. Experimental analyses on CMU MoBo, CASIA B, TUM-IITKGP, and KY4D datasets show a significant performance gain in terms of accuracy and robustness
View and clothing invariant gait recognition via 3D human semantic folding
A novel 3-dimensional (3D) human semantic folding is introduced to provide a robust and efficient gait recognition method which is invariant to camera view and clothing style. The proposed gait recognition method comprises three modules: (1) 3D body pose, shape and viewing data estimation network (3D-BPSVeNet); (2) gait semantic parameter folding model; and (3) gait semantic feature refining network. First, 3D-BPSVeNet is constructed based on a convolution gated recurrent unit (ConvGRU) to extract 2-dimensional (2D) to 3D body pose and shape semantic descriptors (2D-3D-BPSDs) from a sequence of gait parsed RGB images. A 3D gait model with virtual dressing is then constructed by morphing the template of 3D body model using the estimated 2D-3D-BPSDs and the recognized clothing styles. The more accurate 2D-3D-BPSDs without clothes are then obtained by using the silhouette similarity function when updating the 3D body model to fit the 2D gait. Second, the intrinsic 2D-3D-BPSDs without interference from clothes are encoded by sparse distributed representation (SDR) to gain the binary gait semantic image (SD-BGSI) in a topographical semantic space. By averaging the SD-BGSIs in a gait cycle, a gait semantic folding image (GSFI) is obtained to give a high-level representation of gait. Third, a gait semantic feature refining network is trained to refine the semantic feature extracted directly from GSFI using three types of prior knowledge, i.e., viewing angles, clothing styles and carrying condition. Experimental analyses on CMU MoBo, CASIA B, KY4D, OU-MVLP and OU-ISIR datasets show a significant performance gain in gait recognition in terms of accuracy and robustness
Recommended from our members
Investigation of gait representations and partial body gait recognition
Recognising an individual by the way they walk is one of the most popular research subjects within
the field of soft biometrics in last few decades. The advancement of technology and equipment such
as Close Circuit Television (CCTV), wireless internet and wearable sensors makes it easier to obtain
gait data than ever before. The gait biometric can be used widely and in different areas such as
biomedical, forensic and surveillance. However, gait recognition still has many challenges and
fundamental issues. All of these problems only serve as a researcher’s motivation to learn more about
various gait topics to overcome the challenges and improve the field of gait recognition.
Gait recognition currently has high performance when carried out under very specific conditions such
as normal walking, obstruction from certain types of clothing and fixed camera view angles. When the
aforementioned conditions are changed, the classification rate dramatically drops. This study aims to
solve the problems of clothing, carrying objects and camera view angles within the indoor
environment and video-based data collection. Two gait related databases used for testing in this study
are CASIA dataset B and OU-ISIR Large population dataset with Bag (OU-LP-Bag). Three main tasks will
be tested with CASIA dataset B while only gait recognition is tested with OU-LP-Bag.
The gait recognition framework is developed to solve the three main tasks including gait recognition
by identical view, view classification and cross view recognition. This framework uses gait images
sequence as input to generate a gait compact image. Next, gait features are extracted with the optimal
feature map by Principal Component Analysis (PCA) and then a linear Support Vector Machine (SVM)
is used as the one-against-all multiclass classifier.
Four gait compact images including Gait Energy Image (GEI), Gait Entropy Image (GEnI), Gait Gaussian
Image (GGI) and the novel gait images called Gait Gaussian Entropy Image (GGEnI) are used as basic
gait representations. Then three secondary gait representations are generated from these basic
representations. These include Gradient Histogram Gait Image (GHGI) and two novel gait
representations called Convolutional Gait Image (CGI) and Convolutional Gradient Histogram Gait
Image (CGHGI). All representations are tested with three main tasks.
When people walk, each body part does not have the same locomotion information, for example,
there is much more motion in the leg than shoulder motion when walking. Moreover, clothing and
carrying objects do not have the same level of affect to every part of the body, for example, a handbag
does not generally affect leg motion. This study divides the human body into fourteen different body
parts based on height. Body parts and gait representations are combined to solve the three main tasks.
Three combined parts techniques which use two different parts to solve the problem are created. The
fist is Part Scores Fusion (PSF) which uses the summation score of two models based on each part. The
highest summation score model is chosen as the result. The second is Part Image Fusion (PIF) which
concatenates two parts into a single image with a 1:1 ratio. The highest scoring model which is
generated from image fusion is selected as the result. The third is Multi Region Duplication (MRD)
which uses the same idea as PIF, however, the second part’s ratio is increased to 1:2, 1:3 and 1:4.
These techniques are tested on the gait recognition by identical view.
In conclusion, the general framework is effectively for three main tasks. GHGI-GEI which is generated
from full silhouette is the most effective representation for gait recognition by identical view and cross
view recognition. GHGI-GGI with lower knee region is the most effective representation for view angle
classification. The GHGI-GEI CPI combination between full body and limb parts is the most effective
combination on OU-LP-Bag. A more detailed description of each aspect is in the following Chapters