Deep Reinforcement Learning (RL) has shown promise in addressing complex
robotic challenges. In real-world applications, RL is often accompanied by
failsafe controllers as a last resort to avoid catastrophic events. While
necessary for safety, these interventions can result in undesirable behaviors,
such as abrupt braking or aggressive steering. This paper proposes two safety
intervention reduction methods: proactive replacement and proactive projection,
which change the action of the agent if it leads to a potential failsafe
intervention. These approaches are compared to state-of-the-art constrained RL
on the OpenAI safety gym benchmark and a human-robot collaboration task. Our
study demonstrates that the combination of our method with provably safe RL
leads to high-performing policies with zero safety violations and a low number
of failsafe interventions. Our versatile method can be applied to a wide range
of real-world robotic tasks, while effectively improving safety without
sacrificing task performance.Comment: 8 pages, 6 figure