Driving safety is a top priority for autonomous vehicles. Orthogonal to prior
work handling accident-prone traffic events by algorithm designs at the policy
level, we investigate a Closed-loop Adversarial Training (CAT) framework for
safe end-to-end driving in this paper through the lens of environment
augmentation. CAT aims to continuously improve the safety of driving agents by
training the agent on safety-critical scenarios that are dynamically generated
over time. A novel resampling technique is developed to turn log-replay
real-world driving scenarios into safety-critical ones via probabilistic
factorization, where the adversarial traffic generation is modeled as the
multiplication of standard motion prediction sub-problems. Consequently, CAT
can launch more efficient physical attacks compared to existing safety-critical
scenario generation methods and yields a significantly less computational cost
in the iterative learning pipeline. We incorporate CAT into the MetaDrive
simulator and validate our approach on hundreds of driving scenarios imported
from real-world driving datasets. Experimental results demonstrate that CAT can
effectively generate adversarial scenarios countering the agent being trained.
After training, the agent can achieve superior driving safety in both
log-replay and safety-critical traffic scenarios on the held-out test set. Code
and data are available at https://metadriverse.github.io/cat.Comment: 7th Conference on Robot Learning (CoRL 2023