2,437 research outputs found
Overcoming Topology Agnosticism: Enhancing Skeleton-Based Action Recognition through Redefined Skeletal Topology Awareness
Graph Convolutional Networks (GCNs) have long defined the state-of-the-art in
skeleton-based action recognition, leveraging their ability to unravel the
complex dynamics of human joint topology through the graph's adjacency matrix.
However, an inherent flaw has come to light in these cutting-edge models: they
tend to optimize the adjacency matrix jointly with the model weights. This
process, while seemingly efficient, causes a gradual decay of bone connectivity
data, culminating in a model indifferent to the very topology it sought to map.
As a remedy, we propose a threefold strategy: (1) We forge an innovative
pathway that encodes bone connectivity by harnessing the power of graph
distances. This approach preserves the vital topological nuances often lost in
conventional GCNs. (2) We highlight an oft-overlooked feature - the temporal
mean of a skeletal sequence, which, despite its modest guise, carries highly
action-specific information. (3) Our investigation revealed strong variations
in joint-to-joint relationships across different actions. This finding exposes
the limitations of a single adjacency matrix in capturing the variations of
relational configurations emblematic of human movement, which we remedy by
proposing an efficient refinement to Graph Convolutions (GC) - the BlockGC.
This evolution slashes parameters by a substantial margin (above 40%), while
elevating performance beyond original GCNs. Our full model, the BlockGCN,
establishes new standards in skeleton-based action recognition for small model
sizes. Its high accuracy, notably on the large-scale NTU RGB+D 120 dataset,
stand as compelling proof of the efficacy of BlockGCN
Understanding the Robustness of Skeleton-based Action Recognition under Adversarial Attack
Action recognition has been heavily employed in many applications such as
autonomous vehicles, surveillance, etc, where its robustness is a primary
concern. In this paper, we examine the robustness of state-of-the-art action
recognizers against adversarial attack, which has been rarely investigated so
far. To this end, we propose a new method to attack action recognizers that
rely on 3D skeletal motion. Our method involves an innovative perceptual loss
that ensures the imperceptibility of the attack. Empirical studies demonstrate
that our method is effective in both white-box and black-box scenarios. Its
generalizability is evidenced on a variety of action recognizers and datasets.
Its versatility is shown in different attacking strategies. Its deceitfulness
is proven in extensive perceptual studies. Our method shows that adversarial
attack on 3D skeletal motions, one type of time-series data, is significantly
different from traditional adversarial attack problems. Its success raises
serious concern on the robustness of action recognizers and provides insights
on potential improvements.Comment: Accepted in CVPR 2021. arXiv admin note: substantial text overlap
with arXiv:1911.0710
BASAR:Black-box Attack on Skeletal Action Recognition
Skeletal motion plays a vital role in human activity recognition as either an
independent data source or a complement. The robustness of skeleton-based
activity recognizers has been questioned recently, which shows that they are
vulnerable to adversarial attacks when the full-knowledge of the recognizer is
accessible to the attacker. However, this white-box requirement is overly
restrictive in most scenarios and the attack is not truly threatening. In this
paper, we show that such threats do exist under black-box settings too. To this
end, we propose the first black-box adversarial attack method BASAR. Through
BASAR, we show that adversarial attack is not only truly a threat but also can
be extremely deceitful, because on-manifold adversarial samples are rather
common in skeletal motions, in contrast to the common belief that adversarial
samples only exist off-manifold. Through exhaustive evaluation and comparison,
we show that BASAR can deliver successful attacks across models, data, and
attack modes. Through harsh perceptual studies, we show that it achieves
effective yet imperceptible attacks. By analyzing the attack on different
activity recognizers, BASAR helps identify the potential causes of their
vulnerability and provides insights on what classifiers are likely to be more
robust against attack. Code is available at
https://github.com/realcrane/BASAR-Black-box-Attack-on-Skeletal-Action-Recognition.Comment: Accepted in CVPR 202
- …