29 research outputs found
Breaking Temporal Consistency: Generating Video Universal Adversarial Perturbations Using Image Models
As video analysis using deep learning models becomes more widespread, the
vulnerability of such models to adversarial attacks is becoming a pressing
concern. In particular, Universal Adversarial Perturbation (UAP) poses a
significant threat, as a single perturbation can mislead deep learning models
on entire datasets. We propose a novel video UAP using image data and image
model. This enables us to take advantage of the rich image data and image
model-based studies available for video applications. However, there is a
challenge that image models are limited in their ability to analyze the
temporal aspects of videos, which is crucial for a successful video attack. To
address this challenge, we introduce the Breaking Temporal Consistency (BTC)
method, which is the first attempt to incorporate temporal information into
video attacks using image models. We aim to generate adversarial videos that
have opposite patterns to the original. Specifically, BTC-UAP minimizes the
feature similarity between neighboring frames in videos. Our approach is simple
but effective at attacking unseen video models. Additionally, it is applicable
to videos of varying lengths and invariant to temporal shifts. Our approach
surpasses existing methods in terms of effectiveness on various datasets,
including ImageNet, UCF-101, and Kinetics-400.Comment: ICCV 202
TubeR: Tubelet Transformer for Video Action Detection
We propose TubeR: a simple solution for spatio-temporal video action
detection. Different from existing methods that depend on either an off-line
actor detector or hand-designed actor-positional hypotheses like proposals or
anchors, we propose to directly detect an action tubelet in a video by
simultaneously performing action localization and recognition from a single
representation. TubeR learns a set of tubelet-queries and utilizes a
tubelet-attention module to model the dynamic spatio-temporal nature of a video
clip, which effectively reinforces the model capacity compared to using
actor-positional hypotheses in the spatio-temporal space. For videos containing
transitional states or scene changes, we propose a context aware classification
head to utilize short-term and long-term context to strengthen action
classification, and an action switch regression head for detecting the precise
temporal action extent. TubeR directly produces action tubelets with variable
lengths and even maintains good results for long video clips. TubeR outperforms
the previous state-of-the-art on commonly used action detection datasets AVA,
UCF101-24 and JHMDB51-21