Human Motion Analysis for Efficient Action Recognition

Automatic understanding of human actions is at the core of several application domains, such as content-based indexing, human-computer interaction, surveillance, and sports video analysis. The recent advances in digital platforms and the exponential growth of video and image data have brought an urgent quest for intelligent frameworks to automatically analyze human motion and predict their corresponding action based on visual data and sensor signals. This thesis presents a collection of methods that targets human action recognition using different action modalities. The first method uses the appearance modality and classifies human actions based on heterogeneous global- and local-based features of scene and humanbody appearances. The second method harnesses 2D and 3D articulated human poses and analyizes the body motion using a discriminative combination of the parts’ velocities, locations, and correlations histograms for action recognition. The third method presents an optimal scheme for combining the probabilistic predictions from different action modalities by solving a constrained quadratic optimization problem. In addition to the action classification task, we present a study that compares the utility of different pose variants in motion analysis for human action recognition. In particular, we compare the recognition performance when 2D and 3D poses are used. Finally, we demonstrate the efficiency of our pose-based method for action recognition in spotting and segmenting motion gestures in real time from a continuous stream of an input video for the recognition of the Italian sign gesture language

Human Action Recognition Based on Temporal Pyramid of Key Poses Using RGB-D Sensors

Human action recognition is a hot research topic in computer vision, mainly due to the high number of related applications, such as surveillance, human computer interaction, or assisted living. Low cost RGB-D sensors have been extensively used in this field. They can provide skeleton joints, which represent a compact and effective representation of the human posture. This work proposes an algorithm for human action recognition where the features are computed from skeleton joints. A sequence of skeleton features is represented as a set of key poses, from which histograms are extracted. The temporal structure of the sequence is kept using a temporal pyramid of key poses. Finally, a multi-class SVM performs the classification task. The algorithm optimization through evolutionary computation allows to reach results comparable to the state-of-the-art on the MSR Action3D dataset.This work was supported by a STSM Grant from COST Action IC1303 AAPELE - Architectures, Algorithms and Platforms for Enhanced Living Environments

Discriminative joint non-negative matrix factorization for human action classification

This paper describes a supervised classification approach based on non-negative matrix factorization (NMF). Our classification framework builds on the recent expansions of non-negative matrix factorization to multiview learning, where the primary dataset benefits from auxiliary information for obtaining shared and meaningful spaces. For discrimination, we utilize data categories in a supervised manner as an auxiliary source of information in order to learn co-occurrences through a common set of basis vectors. We demonstrate the efficiency of our algorithm in integrating various image modalities for enhancing the overall classification accuracy over different benchmark datasets. Our evaluation considers two challenging image datasets of human action recognition. We show that our algorithm achieves superior results over state-of-the-art in terms of efficiency and overall classification accuracy

Learning spatial interest regions from videos to inform action recognition in still images

Common approaches to human action recognition from images rely on local descriptors for classification. Typically, these descriptors are computed in the vicinity of key points which either result from running a key point detector or from dense or random sampling of pixel coordinates. Such key points are not a-priori related to human activities and thus of limited information with regard to action recognition. In this paper, we propose to identify action-specific key points in images using information available from videos. Our approach does not require manual segmentation or templates but applies non-negative matrix factorization to optical flow fields extracted from videos. The resulting basisflows are found to to be indicative of action specific image regions and therefore allow for an informed sampling of key points. We also present a generative model that allows for characterizing joint distributions of regions of interest and a human actions. In practical experiments, we determine correspondences between regions of interest that were automatically learned from videos and manually annotated locations of human body parts available from independent benchmark image data sets. We observe high correlations between learned interest regions and body parts most relevant for different actions

Efficient pose-based action recognition

Action recognition from 3d pose data has gained increasing attention since the data is readily available for depth or RGB-D videos. The most successful approaches so far perform an expensive feature selection or mining approach for training. In this work, we introduce an algorithm that is very efficient for training and testing. The main idea is that rich structured data like 3d pose does not require sophisticated feature modeling or learning. Instead, we reduce pose data over time to histograms of relative location, velocity, and their correlations and use partial least squares to learn a compact and discriminative representation from it. Despite of its efficiency, our approach achieves state-of-the-art accuracy on four different benchmarks. We further investigate differences of 2d and 3d pose data for action recognition

Effects of catheter-based renal denervation on cardiac sympathetic activity and innervation in patients with resistant hypertension

To investigate, whether renal denervation (RDN) has a direct effect on cardiac sympathetic activity and innervation density. RDN demonstrated its efficacy not only in reducing blood pressure (BP) in certain patients, but also in decreasing cardiac hypertrophy and arrhythmias. These pleiotropic effects occur partly independent from the observed BP reduction. Eleven patients with resistant hypertension (mean office systolic BP 180 +/- A 18 mmHg, mean antihypertensive medications 6.0 +/- A 1.5) underwent I-123-mIBG scintigraphy to exclude pheochromocytoma. We measured cardiac sympathetic innervation and activity before and 9 months after RDN. Cardiac sympathetic innervation was assessed by heart to mediastinum ratio (H/M) and sympathetic activity by wash out ratio (WOR). Effects on office BP, 24 h ambulatory BP monitoring, were documented. Office systolic BP and mean ambulatory systolic BP were significantly reduced from 180 to 141 mmHg (p = 0.006) and from 149 to 129 mmHg (p = 0.014), respectively. Cardiac innervation remained unchanged before and after RDN (H/M 2.5 +/- A 0.5 versus 2.6 +/- A 0.4, p = 0.285). Cardiac sympathetic activity was significantly reduced by 67 % (WOR decreased from 24.1 +/- A 12.7 to 7.9 +/- A 25.3 %, p = 0.047). Both, responders and non-responders experienced a reduction of cardiac sympathetic activity. RDN significantly reduced cardiac sympathetic activity thereby demonstrating a direct effect on the heart. These changes occurred independently from BP effects and provide a pathophysiological basis for studies, investigating the potential effect of RDN on arrhythmias and heart failure

Exploiting Context Information for Image Description

Integrating ontological knowledge is a promising research direction to improve automatic image description. In particular, when probabilistic ontologies are available, the corresponding probabilities could be combined with the probabilities produced by a multi-class classifier applied to different parts in an image. This combination not only provides the relations existing between the different segments, but can also improve the classification accuracy. In fact, the context often gives cues suggesting the correct class of the segment. This paper discusses a possible implementation of this integration, and the first experimental results shows its effectiveness when the classifier accuracy is relatively low. For the assessment of the performance we constructed a simulated classifier which allows the a priori decision of its performance with a sufficient precision