16,770 research outputs found
SADA: Semantic Adversarial Diagnostic Attacks for Autonomous Applications
One major factor impeding more widespread adoption of deep neural networks
(DNNs) is their lack of robustness, which is essential for safety-critical
applications such as autonomous driving. This has motivated much recent work on
adversarial attacks for DNNs, which mostly focus on pixel-level perturbations
void of semantic meaning. In contrast, we present a general framework for
adversarial attacks on trained agents, which covers semantic perturbations to
the environment of the agent performing the task as well as pixel-level
attacks. To do this, we re-frame the adversarial attack problem as learning a
distribution of parameters that always fools the agent. In the semantic case,
our proposed adversary (denoted as BBGAN) is trained to sample parameters that
describe the environment with which the black-box agent interacts, such that
the agent performs its dedicated task poorly in this environment. We apply
BBGAN on three different tasks, primarily targeting aspects of autonomous
navigation: object detection, self-driving, and autonomous UAV racing. On these
tasks, BBGAN can generate failure cases that consistently fool a trained agent.Comment: Accepted at AAAI'2
Adv3D: Generating Safety-Critical 3D Objects through Closed-Loop Simulation
Self-driving vehicles (SDVs) must be rigorously tested on a wide range of
scenarios to ensure safe deployment. The industry typically relies on
closed-loop simulation to evaluate how the SDV interacts on a corpus of
synthetic and real scenarios and verify it performs properly. However, they
primarily only test the system's motion planning module, and only consider
behavior variations. It is key to evaluate the full autonomy system in
closed-loop, and to understand how variations in sensor data based on scene
appearance, such as the shape of actors, affect system performance. In this
paper, we propose a framework, Adv3D, that takes real world scenarios and
performs closed-loop sensor simulation to evaluate autonomy performance, and
finds vehicle shapes that make the scenario more challenging, resulting in
autonomy failures and uncomfortable SDV maneuvers. Unlike prior works that add
contrived adversarial shapes to vehicle roof-tops or roadside to harm
perception only, we optimize a low-dimensional shape representation to modify
the vehicle shape itself in a realistic manner to degrade autonomy performance
(e.g., perception, prediction, and motion planning). Moreover, we find that the
shape variations found with Adv3D optimized in closed-loop are much more
effective than those in open-loop, demonstrating the importance of finding
scene appearance variations that affect autonomy in the interactive setting.Comment: CoRL 2023. Project page: https://waabi.ai/adv3d
DeepPicar: A Low-cost Deep Neural Network-based Autonomous Car
We present DeepPicar, a low-cost deep neural network based autonomous car
platform. DeepPicar is a small scale replication of a real self-driving car
called DAVE-2 by NVIDIA. DAVE-2 uses a deep convolutional neural network (CNN),
which takes images from a front-facing camera as input and produces car
steering angles as output. DeepPicar uses the same network architecture---9
layers, 27 million connections and 250K parameters---and can drive itself in
real-time using a web camera and a Raspberry Pi 3 quad-core platform. Using
DeepPicar, we analyze the Pi 3's computing capabilities to support end-to-end
deep learning based real-time control of autonomous vehicles. We also
systematically compare other contemporary embedded computing platforms using
the DeepPicar's CNN-based real-time control workload. We find that all tested
platforms, including the Pi 3, are capable of supporting the CNN-based
real-time control, from 20 Hz up to 100 Hz, depending on hardware platform.
However, we find that shared resource contention remains an important issue
that must be considered in applying CNN models on shared memory based embedded
computing platforms; we observe up to 11.6X execution time increase in the CNN
based control loop due to shared resource contention. To protect the CNN
workload, we also evaluate state-of-the-art cache partitioning and memory
bandwidth throttling techniques on the Pi 3. We find that cache partitioning is
ineffective, while memory bandwidth throttling is an effective solution.Comment: To be published as a conference paper at RTCSA 201
Among Us: Adversarially Robust Collaborative Perception by Consensus
Multiple robots could perceive a scene (e.g., detect objects) collaboratively
better than individuals, although easily suffer from adversarial attacks when
using deep learning. This could be addressed by the adversarial defense, but
its training requires the often-unknown attacking mechanism. Differently, we
propose ROBOSAC, a novel sampling-based defense strategy generalizable to
unseen attackers. Our key idea is that collaborative perception should lead to
consensus rather than dissensus in results compared to individual perception.
This leads to our hypothesize-and-verify framework: perception results with and
without collaboration from a random subset of teammates are compared until
reaching a consensus. In such a framework, more teammates in the sampled subset
often entail better perception performance but require longer sampling time to
reject potential attackers. Thus, we derive how many sampling trials are needed
to ensure the desired size of an attacker-free subset, or equivalently, the
maximum size of such a subset that we can successfully sample within a given
number of trials. We validate our method on the task of collaborative 3D object
detection in autonomous driving scenarios
- …