Communication technologies enable coordination among connected and autonomous
vehicles (CAVs). However, it remains unclear how to utilize shared information
to improve the safety and efficiency of the CAV system. In this work, we
propose a framework of constrained multi-agent reinforcement learning (MARL)
with a parallel safety shield for CAVs in challenging driving scenarios. The
coordination mechanisms of the proposed MARL include information sharing and
cooperative policy learning, with Graph Convolutional Network (GCN)-Transformer
as a spatial-temporal encoder that enhances the agent's environment awareness.
The safety shield module with Control Barrier Functions (CBF)-based safety
checking protects the agents from taking unsafe actions. We design a
constrained multi-agent advantage actor-critic (CMAA2C) algorithm to train safe
and cooperative policies for CAVs. With the experiment deployed in the CARLA
simulator, we verify the effectiveness of the safety checking, spatial-temporal
encoder, and coordination mechanisms designed in our method by comparative
experiments in several challenging scenarios with the defined hazard vehicles
(HAZV). Results show that our proposed methodology significantly increases
system safety and efficiency in challenging scenarios.Comment: This paper has been accepted by the 2023 IEEE International
Conference on Robotics and Automation (ICRA 2023). 6 pages, 5 figure