2,681 research outputs found
Spatial-Aware Object Embeddings for Zero-Shot Localization and Classification of Actions
We aim for zero-shot localization and classification of human actions in
video. Where traditional approaches rely on global attribute or object
classification scores for their zero-shot knowledge transfer, our main
contribution is a spatial-aware object embedding. To arrive at spatial
awareness, we build our embedding on top of freely available actor and object
detectors. Relevance of objects is determined in a word embedding space and
further enforced with estimated spatial preferences. Besides local object
awareness, we also embed global object awareness into our embedding to maximize
actor and object interaction. Finally, we exploit the object positions and
sizes in the spatial-aware embedding to demonstrate a new spatio-temporal
action retrieval scenario with composite queries. Action localization and
classification experiments on four contemporary action video datasets support
our proposal. Apart from state-of-the-art results in the zero-shot localization
and classification settings, our spatial-aware embedding is even competitive
with recent supervised action localization alternatives.Comment: ICC
CycleACR: Cycle Modeling of Actor-Context Relations for Video Action Detection
The relation modeling between actors and scene context advances video action
detection where the correlation of multiple actors makes their action
recognition challenging. Existing studies model each actor and scene relation
to improve action recognition. However, the scene variations and background
interference limit the effectiveness of this relation modeling. In this paper,
we propose to select actor-related scene context, rather than directly leverage
raw video scenario, to improve relation modeling. We develop a Cycle
Actor-Context Relation network (CycleACR) where there is a symmetric graph that
models the actor and context relations in a bidirectional form. Our CycleACR
consists of the Actor-to-Context Reorganization (A2C-R) that collects actor
features for context feature reorganizations, and the Context-to-Actor
Enhancement (C2A-E) that dynamically utilizes reorganized context features for
actor feature enhancement. Compared to existing designs that focus on C2A-E,
our CycleACR introduces A2C-R for a more effective relation modeling. This
modeling advances our CycleACR to achieve state-of-the-art performance on two
popular action detection datasets (i.e., AVA and UCF101-24). We also provide
ablation studies and visualizations as well to show how our cycle actor-context
relation modeling improves video action detection. Code is available at
https://github.com/MCG-NJU/CycleACR.Comment: technical repor
- …