312,378 research outputs found
Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification
Despite the steady progress in video analysis led by the adoption of
convolutional neural networks (CNNs), the relative improvement has been less
drastic as that in 2D static image classification. Three main challenges exist
including spatial (image) feature representation, temporal information
representation, and model/computation complexity. It was recently shown by
Carreira and Zisserman that 3D CNNs, inflated from 2D networks and pretrained
on ImageNet, could be a promising way for spatial and temporal representation
learning. However, as for model/computation complexity, 3D CNNs are much more
expensive than 2D CNNs and prone to overfit. We seek a balance between speed
and accuracy by building an effective and efficient video classification system
through systematic exploration of critical network design choices. In
particular, we show that it is possible to replace many of the 3D convolutions
by low-cost 2D convolutions. Rather surprisingly, best result (in both speed
and accuracy) is achieved when replacing the 3D convolutions at the bottom of
the network, suggesting that temporal representation learning on high-level
semantic features is more useful. Our conclusion generalizes to datasets with
very different properties. When combined with several other cost-effective
designs including separable spatial/temporal convolution and feature gating,
our system results in an effective video classification system that that
produces very competitive results on several action classification benchmarks
(Kinetics, Something-something, UCF101 and HMDB), as well as two action
detection (localization) benchmarks (JHMDB and UCF101-24).Comment: ECCV 2018 camera read
Supervised learning on graphs of spatio-temporal similarity in satellite image sequences
High resolution satellite image sequences are multidimensional signals
composed of spatio-temporal patterns associated to numerous and various
phenomena. Bayesian methods have been previously proposed in (Heas and Datcu,
2005) to code the information contained in satellite image sequences in a graph
representation using Bayesian methods. Based on such a representation, this
paper further presents a supervised learning methodology of semantics
associated to spatio-temporal patterns occurring in satellite image sequences.
It enables the recognition and the probabilistic retrieval of similar events.
Indeed, graphs are attached to statistical models for spatio-temporal
processes, which at their turn describe physical changes in the observed scene.
Therefore, we adjust a parametric model evaluating similarity types between
graph patterns in order to represent user-specific semantics attached to
spatio-temporal phenomena. The learning step is performed by the incremental
definition of similarity types via user-provided spatio-temporal pattern
examples attached to positive or/and negative semantics. From these examples,
probabilities are inferred using a Bayesian network and a Dirichlet model. This
enables to links user interest to a specific similarity model between graph
patterns. According to the current state of learning, semantic posterior
probabilities are updated for all possible graph patterns so that similar
spatio-temporal phenomena can be recognized and retrieved from the image
sequence. Few experiments performed on a multi-spectral SPOT image sequence
illustrate the proposed spatio-temporal recognition method
- …