150 research outputs found
Unified Keypoint-based Action Recognition Framework via Structured Keypoint Pooling
This paper simultaneously addresses three limitations associated with
conventional skeleton-based action recognition; skeleton detection and tracking
errors, poor variety of the targeted actions, as well as person-wise and
frame-wise action recognition. A point cloud deep-learning paradigm is
introduced to the action recognition, and a unified framework along with a
novel deep neural network architecture called Structured Keypoint Pooling is
proposed. The proposed method sparsely aggregates keypoint features in a
cascaded manner based on prior knowledge of the data structure (which is
inherent in skeletons), such as the instances and frames to which each keypoint
belongs, and achieves robustness against input errors. Its less constrained and
tracking-free architecture enables time-series keypoints consisting of human
skeletons and nonhuman object contours to be efficiently treated as an input 3D
point cloud and extends the variety of the targeted action. Furthermore, we
propose a Pooling-Switching Trick inspired by Structured Keypoint Pooling. This
trick switches the pooling kernels between the training and inference phases to
detect person-wise and frame-wise actions in a weakly supervised manner using
only video-level action labels. This trick enables our training scheme to
naturally introduce novel data augmentation, which mixes multiple point clouds
extracted from different videos. In the experiments, we comprehensively verify
the effectiveness of the proposed method against the limitations, and the
method outperforms state-of-the-art skeleton-based action recognition and
spatio-temporal action localization methods.Comment: CVPR 202
- …