6 research outputs found
Training Adversarial Agents to Exploit Weaknesses in Deep Control Policies
Deep learning has become an increasingly common technique for various control
problems, such as robotic arm manipulation, robot navigation, and autonomous
vehicles. However, the downside of using deep neural networks to learn control
policies is their opaque nature and the difficulties of validating their
safety. As the networks used to obtain state-of-the-art results become
increasingly deep and complex, the rules they have learned and how they operate
become more challenging to understand. This presents an issue, since in
safety-critical applications the safety of the control policy must be ensured
to a high confidence level. In this paper, we propose an automated black box
testing framework based on adversarial reinforcement learning. The technique
uses an adversarial agent, whose goal is to degrade the performance of the
target model under test. We test the approach on an autonomous vehicle problem,
by training an adversarial reinforcement learning agent, which aims to cause a
deep neural network-driven autonomous vehicle to collide. Two neural networks
trained for autonomous driving are compared, and the results from the testing
are used to compare the robustness of their learned control policies. We show
that the proposed framework is able to find weaknesses in both control policies
that were not evident during online testing and therefore, demonstrate a
significant benefit over manual testing methods.Comment: 2020 IEEE International Conference on Robotics and Automation (ICRA
Weakly Supervised Reinforcement Learning for Autonomous Highway Driving via Virtual Safety Cages
The use of neural networks and reinforcement learning has become increasingly
popular in autonomous vehicle control. However, the opaqueness of the resulting
control policies presents a significant barrier to deploying neural
network-based control in autonomous vehicles. In this paper, we present a
reinforcement learning based approach to autonomous vehicle longitudinal
control, where the rule-based safety cages provide enhanced safety for the
vehicle as well as weak supervision to the reinforcement learning agent. By
guiding the agent to meaningful states and actions, this weak supervision
improves the convergence during training and enhances the safety of the final
trained policy. This rule-based supervisory controller has the further
advantage of being fully interpretable, thereby enabling traditional validation
and verification approaches to ensure the safety of the vehicle. We compare
models with and without safety cages, as well as models with optimal and
constrained model parameters, and show that the weak supervision consistently
improves the safety of exploration, speed of convergence, and model
performance. Additionally, we show that when the model parameters are
constrained or sub-optimal, the safety cages can enable a model to learn a safe
driving policy even when the model could not be trained to drive through
reinforcement learning alone.Comment: Published in Sensor
Rethink the Adversarial Scenario-based Safety Testing of Robots: the Comparability and Optimal Aggressiveness
This paper studies the class of scenario-based safety testing algorithms in
the black-box safety testing configuration. For algorithms sharing the same
state-action set coverage with different sampling distributions, it is commonly
believed that prioritizing the exploration of high-risk state-actions leads to
a better sampling efficiency. Our proposal disputes the above intuition by
introducing an impossibility theorem that provably shows all safety testing
algorithms of the aforementioned difference perform equally well with the same
expected sampling efficiency. Moreover, for testing algorithms covering
different sets of state-actions, the sampling efficiency criterion is no longer
applicable as different algorithms do not necessarily converge to the same
termination condition. We then propose a testing aggressiveness definition
based on the almost safe set concept along with an unbiased and efficient
algorithm that compares the aggressiveness between testing algorithms.
Empirical observations from the safety testing of bipedal locomotion
controllers and vehicle decision-making modules are also presented to support
the proposed theoretical implications and methodologies