15 research outputs found
Late Temporal Modeling in 3D CNN Architectures with BERT for Action Recognition
In this work, we combine 3D convolution with late temporal modeling for
action recognition. For this aim, we replace the conventional Temporal Global
Average Pooling (TGAP) layer at the end of 3D convolutional architecture with
the Bidirectional Encoder Representations from Transformers (BERT) layer in
order to better utilize the temporal information with BERT's attention
mechanism. We show that this replacement improves the performances of many
popular 3D convolution architectures for action recognition, including ResNeXt,
I3D, SlowFast and R(2+1)D. Moreover, we provide the-state-of-the-art results on
both HMDB51 and UCF101 datasets with 85.10% and 98.69% top-1 accuracy,
respectively. The code is publicly available.Comment: Presented on the 2nd Workshop on Video Turing Test: Toward
Human-Level Video Story Understanding, ECCV 202
LSTA: Long Short-Term Attention for Egocentric Action Recognition
Egocentric activity recognition is one of the most challenging tasks in video analysis. It requires a fine-grained discrimination of small objects and their manipulation. While some methods base on strong supervision and attention mechanisms, they are either annotation consuming or do not take spatio-temporal patterns into account. In this paper we propose LSTA as a mechanism to focus on features from spatial relevant parts while attention is being tracked smoothly across the video sequence. We demonstrate the effectiveness of LSTA on egocentric activity recognition with an end-to-end trainable two-stream architecture, achieving state-of-the-art performance on four standard benchmarks