This paper strives to recognize individual actions and group activities from
videos. While existing solutions for this challenging problem explicitly model
spatial and temporal relationships based on location of individual actors, we
propose an actor-transformer model able to learn and selectively extract
information relevant for group activity recognition. We feed the transformer
with rich actor-specific static and dynamic representations expressed by
features from a 2D pose network and 3D CNN, respectively. We empirically study
different ways to combine these representations and show their complementary
benefits. Experiments show what is important to transform and how it should be
transformed. What is more, actor-transformers achieve state-of-the-art results
on two publicly available benchmarks for group activity recognition,
outperforming the previous best published results by a considerable margin.Comment: CVPR 202