3 research outputs found
Novel methods for posture-based human action recognition and activity anomaly detection
PhD ThesisArti cial Intelligence (AI) for Human Action Recognition (HAR) and Human
Activity Anomaly Detection (HAAD) is an active and exciting research
eld. Video-based HAR aims to classify human actions and video-based
HAAD aims to detect abnormal human activities within data. However, a
human is an extremely complex subject and a non-rigid object in the video,
which provides great challenges for Computer Vision and Signal Processing.
Relevant applications elds are surveillance and public monitoring, assisted
living, robotics, human-to-robot interaction, prosthetics, gaming, video captioning,
and sports analysis.
The focus of this thesis is on the posture-related HAR and HAAD. The
aim is to design computationally-e cient, machine and deep learning-based
HAR and HAAD methods which can run in multiple humans monitoring
scenarios.
This thesis rstly contributes two novel 3D Histogram of Oriented Gradient
(3D-HOG) driven frameworks for silhouette-based HAR. The 3D-HOG
state-of-the-art limitations, e.g. unweighted local body areas based processing
and unstable performance over di erent training rounds, are addressed.
The proposed methods achieve more accurate results than the
baseline, outperforming the state-of-the-art. Experiments are conducted on
publicly available datasets, alongside newly recorded data.
This thesis also contributes a new algorithm for human poses-based
HAR. In particular, the proposed human poses-based HAR is among the
rst, few, simultaneous attempts which have been conducted at the time.
The proposed HAR algorithm, named ActionXPose, is based on Convolutional
Neural Networks and Long Short-Term Memory. It turns out to be
more reliable and computationally advantageous when compared to human
silhouette-based approaches. The ActionXPose's
exibility also allows crossdatasets
processing and more robustness to occlusions scenarios. Extensive
evaluation on publicly available datasets demonstrates the e cacy of ActionXPose
over the state-of-the-art. Moreover, newly recorded data, i.e.
Intelligent Sensing Lab Dataset (ISLD), is also contributed and exploited to
further test ActionXPose in real-world, non-cooperative scenarios.
The last set of contributions in this thesis regards pose-driven, combined
HAR and HAAD algorithms. Motivated by ActionXPose achievements, this
thesis contributes a new algorithm to simultaneously extract deep-learningbased
features from human-poses, RGB Region of Interests (ROIs) and
detected objects positions. The proposed method outperforms the stateof-
the-art in both HAR and HAAD. The HAR performance is extensively
tested on publicly available datasets, including the contributed ISLD dataset.
Moreover, to compensate for the lack of data in the eld, this thesis
also contributes three new datasets for human-posture and objects-positions
related HAAD, i.e. BMbD, M-BMdD and JBMOPbD datasets