7 research outputs found
Recommended from our members
View-invariant gait person re-identification with spatial and temporal attention
This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonPerson re-identification at a distance across multiple none overlapping cameras has
been an active research area for years. In the past ten years, Short term Person Re-Id
techniques have made great strides in terms of accuracy using only appearance features
in limited environments. However, massive intraclass variations and inter-class
confusion limit their ability to be used in practical applications. Moreover, appearance
consistency can only be assumed in a short time span from one camera to the other.
Since the holistic appearance will change drastically over days and weeks, the technique,
as mentioned above, will be ineffective. Practical applications usually require a
long-term solution in which the subject appearance and clothing might have changed
after a significant period has elapsed. Facing these problems, soft biometric features
such as Gait have been proposed in the past. Nevertheless, even Gait can vary with
illness, ageing and changes in the emotional state, changes in walking surfaces, shoe
type, clothes type, objects carried by the subject and even clutter in the scene. Therefore,
Gait is considered a temporal cue that could provide biometric motion information.
On the other hand, the shape of the human body could be viewed as a spatial signal
which can produce valuable information. So, extracting discriminative features from
both spatial and temporal domains would be very beneficial to this research. Therefore,
this thesis focuses on finding the best and most robust method to tackle the gait human Re-identification problem and solve it for practical applications. In real-world
surveillance scenarios, the human gait cycle is primarily abnormal. These abnormalities
include but not limited to temporal and spatial characteristics changes such as
walking speed, broken gait phase and most importantly, varied camera angles. Our
work performed an extensive literature study on spatial and temporal gait feature extraction
methods with a focus on deep learning. Next, we conducted a comparative
study and proposed a spatial-temporal approach for gait feature extraction using the
fusion of multiple modalities, including optical-flow, raw silhouettes and RGB images.
This approach was tested on two of the most challenging publicly available datasets for
gait recognition TUM-GAID and CASIA-B, with excellent results presented in chapter
3.
Furthermore, a modern spatial-temporal attention mechanism was proposed and
tested on CASIA-B and OULP datasets which learns salient features independent of
the gait cycle and view variations. The spatial attention layer in the proposed method
extracts the spatial feature maps using a two-layered architecture that are fused using
late fusion. It can pay attention to the identity-related salient regions in silhouette sequences
discriminatively using the spatial feature maps. The temporal attention layer
consists of an LSTM that encodes the temporal motion for silhouette sequences. It
uses the encoded output vectors in the temporal attention architecture to focus on the
most critical timesteps in the gait cycle and discard the rest. Furthermore, we improved
the performance of our method by mapping our extracted spatial-temporal gait
features to a discriminative null space for use in our Siamese architecture for crossmatching.
We also conducted an element removal experiment on each segment of our
spatial-temporal attentional network to gain insight into each component’s contribution to the performance. Our method showed outstanding robustness against abnormal
gait cycles as well as viewpoint variations on both benchmark datasets
Recommended from our members
Investigation of gait representations and partial body gait recognition
Recognising an individual by the way they walk is one of the most popular research subjects within
the field of soft biometrics in last few decades. The advancement of technology and equipment such
as Close Circuit Television (CCTV), wireless internet and wearable sensors makes it easier to obtain
gait data than ever before. The gait biometric can be used widely and in different areas such as
biomedical, forensic and surveillance. However, gait recognition still has many challenges and
fundamental issues. All of these problems only serve as a researcher’s motivation to learn more about
various gait topics to overcome the challenges and improve the field of gait recognition.
Gait recognition currently has high performance when carried out under very specific conditions such
as normal walking, obstruction from certain types of clothing and fixed camera view angles. When the
aforementioned conditions are changed, the classification rate dramatically drops. This study aims to
solve the problems of clothing, carrying objects and camera view angles within the indoor
environment and video-based data collection. Two gait related databases used for testing in this study
are CASIA dataset B and OU-ISIR Large population dataset with Bag (OU-LP-Bag). Three main tasks will
be tested with CASIA dataset B while only gait recognition is tested with OU-LP-Bag.
The gait recognition framework is developed to solve the three main tasks including gait recognition
by identical view, view classification and cross view recognition. This framework uses gait images
sequence as input to generate a gait compact image. Next, gait features are extracted with the optimal
feature map by Principal Component Analysis (PCA) and then a linear Support Vector Machine (SVM)
is used as the one-against-all multiclass classifier.
Four gait compact images including Gait Energy Image (GEI), Gait Entropy Image (GEnI), Gait Gaussian
Image (GGI) and the novel gait images called Gait Gaussian Entropy Image (GGEnI) are used as basic
gait representations. Then three secondary gait representations are generated from these basic
representations. These include Gradient Histogram Gait Image (GHGI) and two novel gait
representations called Convolutional Gait Image (CGI) and Convolutional Gradient Histogram Gait
Image (CGHGI). All representations are tested with three main tasks.
When people walk, each body part does not have the same locomotion information, for example,
there is much more motion in the leg than shoulder motion when walking. Moreover, clothing and
carrying objects do not have the same level of affect to every part of the body, for example, a handbag
does not generally affect leg motion. This study divides the human body into fourteen different body
parts based on height. Body parts and gait representations are combined to solve the three main tasks.
Three combined parts techniques which use two different parts to solve the problem are created. The
fist is Part Scores Fusion (PSF) which uses the summation score of two models based on each part. The
highest summation score model is chosen as the result. The second is Part Image Fusion (PIF) which
concatenates two parts into a single image with a 1:1 ratio. The highest scoring model which is
generated from image fusion is selected as the result. The third is Multi Region Duplication (MRD)
which uses the same idea as PIF, however, the second part’s ratio is increased to 1:2, 1:3 and 1:4.
These techniques are tested on the gait recognition by identical view.
In conclusion, the general framework is effectively for three main tasks. GHGI-GEI which is generated
from full silhouette is the most effective representation for gait recognition by identical view and cross
view recognition. GHGI-GGI with lower knee region is the most effective representation for view angle
classification. The GHGI-GEI CPI combination between full body and limb parts is the most effective
combination on OU-LP-Bag. A more detailed description of each aspect is in the following Chapters