77,636 research outputs found
Differential Recurrent Neural Networks for Action Recognition
The long short-term memory (LSTM) neural network is capable of processing
complex sequential information since it utilizes special gating schemes for
learning representations from long input sequences. It has the potential to
model any sequential time-series data, where the current hidden state has to be
considered in the context of the past hidden states. This property makes LSTM
an ideal choice to learn the complex dynamics of various actions.
Unfortunately, the conventional LSTMs do not consider the impact of
spatio-temporal dynamics corresponding to the given salient motion patterns,
when they gate the information that ought to be memorized through time. To
address this problem, we propose a differential gating scheme for the LSTM
neural network, which emphasizes on the change in information gain caused by
the salient motions between the successive frames. This change in information
gain is quantified by Derivative of States (DoS), and thus the proposed LSTM
model is termed as differential Recurrent Neural Network (dRNN). We demonstrate
the effectiveness of the proposed model by automatically recognizing actions
from the real-world 2D and 3D human action datasets. Our study is one of the
first works towards demonstrating the potential of learning complex time-series
representations via high-order derivatives of states
Action Recognition in Videos: from Motion Capture Labs to the Web
This paper presents a survey of human action recognition approaches based on
visual data recorded from a single video camera. We propose an organizing
framework which puts in evidence the evolution of the area, with techniques
moving from heavily constrained motion capture scenarios towards more
challenging, realistic, "in the wild" videos. The proposed organization is
based on the representation used as input for the recognition task, emphasizing
the hypothesis assumed and thus, the constraints imposed on the type of video
that each technique is able to address. Expliciting the hypothesis and
constraints makes the framework particularly useful to select a method, given
an application. Another advantage of the proposed organization is that it
allows categorizing newest approaches seamlessly with traditional ones, while
providing an insightful perspective of the evolution of the action recognition
task up to now. That perspective is the basis for the discussion in the end of
the paper, where we also present the main open issues in the area.Comment: Preprint submitted to CVIU, survey paper, 46 pages, 2 figures, 4
table
- …