A Learning-based Stochastic MPC Design for Cooperative Adaptive Cruise Control to Handle Interfering Vehicles

Vehicle to Vehicle (V2V) communication has a great potential to improve reaction accuracy of different driver assistance systems in critical driving situations. Cooperative Adaptive Cruise Control (CACC), which is an automated application, provides drivers with extra benefits such as traffic throughput maximization and collision avoidance. CACC systems must be designed in a way that are sufficiently robust against all special maneuvers such as cutting-into the CACC platoons by interfering vehicles or hard braking by leading cars. To address this problem, a Neural- Network (NN)-based cut-in detection and trajectory prediction scheme is proposed in the first part of this paper. Next, a probabilistic framework is developed in which the cut-in probability is calculated based on the output of the mentioned cut-in prediction block. Finally, a specific Stochastic Model Predictive Controller (SMPC) is designed which incorporates this cut-in probability to enhance its reaction against the detected dangerous cut-in maneuver. The overall system is implemented and its performance is evaluated using realistic driving scenarios from Safety Pilot Model Deployment (SPMD).Comment: 10 pages, Submitted as a journal paper at T-I

Multi-Timescale Control and Communications with Deep Reinforcement Learning -- Part I: Communication-Aware Vehicle Control

An intelligent decision-making system enabled by Vehicle-to-Everything (V2X) communications is essential to achieve safe and efficient autonomous driving (AD), where two types of decisions have to be made at different timescales, i.e., vehicle control and radio resource allocation (RRA) decisions. The interplay between RRA and vehicle control necessitates their collaborative design. In this two-part paper (Part I and Part II), taking platoon control (PC) as an example use case, we propose a joint optimization framework of multi-timescale control and communications (MTCC) based on Deep Reinforcement Learning (DRL). In this paper (Part I), we first decompose the problem into a communication-aware DRL-based PC sub-problem and a control-aware DRL-based RRA sub-problem. Then, we focus on the PC sub-problem assuming an RRA policy is given, and propose the MTCC-PC algorithm to learn an efficient PC policy. To improve the PC performance under random observation delay, the PC state space is augmented with the observation delay and PC action history. Moreover, the reward function with respect to the augmented state is defined to construct an augmented state Markov Decision Process (MDP). It is proved that the optimal policy for the augmented state MDP is optimal for the original PC problem with observation delay. Different from most existing works on communication-aware control, the MTCC-PC algorithm is trained in a delayed environment generated by the fine-grained embedded simulation of C-V2X communications rather than by a simple stochastic delay model. Finally, experiments are performed to compare the performance of MTCC-PC with those of the baseline DRL algorithms

Event-triggered autonomous platoon control against probabilistic sensor and actuator failures

This paper is concerned with event-triggered autonomous platoon control with probabilistic sensor and actuator failures. A new platoon model is established, in which the effect of eventtriggered scheme and probabilistic failures are involved. Based on the model, the criteria for the exponential stability and co-designing both the trigger parameters and the output feedback are derived by using the Lyapunov method. The theoretical results show that the proposed controller would be able to safely maintain a smaller inter-vehicle spacing and the platoon would be string stable. The effectiveness and advantage of the presented methodology are demonstrated by numerical simulations

Multi-Timescale Control and Communications with Deep Reinforcement Learning -- Part II: Control-Aware Radio Resource Allocation

In Part I of this two-part paper (Multi-Timescale Control and Communications with Deep Reinforcement Learning -- Part I: Communication-Aware Vehicle Control), we decomposed the multi-timescale control and communications (MTCC) problem in Cellular Vehicle-to-Everything (C-V2X) system into a communication-aware Deep Reinforcement Learning (DRL)-based platoon control (PC) sub-problem and a control-aware DRL-based radio resource allocation (RRA) sub-problem. We focused on the PC sub-problem and proposed the MTCC-PC algorithm to learn an optimal PC policy given an RRA policy. In this paper (Part II), we first focus on the RRA sub-problem in MTCC assuming a PC policy is given, and propose the MTCC-RRA algorithm to learn the RRA policy. Specifically, we incorporate the PC advantage function in the RRA reward function, which quantifies the amount of PC performance degradation caused by observation delay. Moreover, we augment the state space of RRA with PC action history for a more well-informed RRA policy. In addition, we utilize reward shaping and reward backpropagation prioritized experience replay (RBPER) techniques to efficiently tackle the multi-agent and sparse reward problems, respectively. Finally, a sample- and computational-efficient training approach is proposed to jointly learn the PC and RRA policies in an iterative process. In order to verify the effectiveness of the proposed MTCC algorithm, we performed experiments using real driving data for the leading vehicle, where the performance of MTCC is compared with those of the baseline DRL algorithms

On Resilient Control for Secure Connected Vehicles: A Hybrid Systems Approach

According to the Internet of Things Forecast conducted by Ericsson, connected devices will be around 29 billion by 2022. This technological revolution enables the concept of Cyber-Physical Systems (CPSs) that will transform many applications, including power-grid, transportation, smart buildings, and manufacturing. Manufacturers and institutions are relying on technologies related to CPSs to improve the efficiency and performances of their products and services. However, the higher the number of connected devices, the higher the exposure to cybersecurity threats. In the case of CPSs, successful cyber-attacks can potentially hamper the economy and endanger human lives. Therefore, it is of paramount importance to develop and adopt resilient technologies that can complement the existing security tools to make CPSs more resilient to cyber-attacks. By exploiting the intrinsically present physical characteristics of CPSs, this dissertation employs dynamical and control systems theory to improve the CPS resiliency to cyber-attacks. In particular, we consider CPSs as Networked Control Systems (NCSs), which are control systems where plant and controller share sensing and actuating information through networks. This dissertation proposes novel design procedures that maximize the resiliency of NCSs to network imperfections (i.e., sampling, packet dropping, and network delays) and denial of service (DoS) attacks. We model CPSs from a general point of view to generate design procedures that have a vast spectrum of applicability while creating computationally affordable algorithms capable of real-time performances. Indeed, the findings of this research aspire to be easily applied to several CPSs applications, e.g., power grid, transportation systems, and remote surgery. However, this dissertation focuses on applying its theoretical outcomes to connected and automated vehicle (CAV) systems where vehicles are capable of sharing information via a wireless communication network. In the first part of the dissertation, we propose a set of LMI-based constructive Lyapunov-based tools for the analysis of the resiliency of NCSs, and we propose a design approach that maximizes the resiliency. In the second part of the thesis, we deal with the design of DOS-resilient control systems for connected vehicle applications. In particular, we focus on the Cooperative Adaptive Cruise Control (CACC), which is one of the most popular and promising applications involving CAVs

Models and Performance of VANET based Emergency Braking

The network research community is working in the field of automotive to provide VANET based safety applications to reduce the number of accidents, deaths, injuries and loss of money. Several approaches are proposed and investigated in VANET literature, but in a completely network-oriented fashion. Most of them do not take into account application requirements and no one considers the dynamics of the vehicles. Moreover, message repropagation schemes are widely proposed without investigating their benefits and using very complicated approaches. This technical report, which is derived from the Master Thesis of Michele Segata, focuses on the Emergency Electronic Brake Lights (EEBL) safety application, meant to send warning messages in the case of an emergency brake, in particular performing a joint analysis of network requirements and provided application level benefits. The EEBL application is integrated within a Collaborative Adaptive Cruise Control (CACC) which uses network-provided information to automatically brake the car if the driver does not react to the warning. Moreover, an information aggregation scheme is proposed to analyze the benefits of repropagation together with the consequent increase of network load. This protocol is compared to a protocol without repropagation and to a rebroadcast protocol found in the literature (namely the weighted p-persistent rebroadcast). The scenario is a highway stretch in which a platoon of vehicles brake down to a complete stop. Simulations are performed using the NS_3 network simulation in which two mobility models have been embedded. The first one, which is called Intelligent Driver Model (IDM) emulates the behavior of a driver trying to reach a desired speed and braking when approaching vehicles in front. The second one (Minimizing Overall Braking Induced by Lane change (MOBIL)), instead, decides when a vehicle has to change lane in order to perform an overtake or optimize its path. The original simulator has been modified by - introducing real physical limits to naturally reproduce real crashes; - implementing a CACC; - implementing the driver reaction when a warning is received; - implementing different network protocols. The tests are performed in different situations, such as different number of lanes (one to five), different average speeds, different network protocols and different market penetration rates and they show that: - the adoption of this technology considerably decreases car accidents since the overall average maximum deceleration is reduced; - network load depends on application-level details, such as the implementation of the CACC; - VANET safety application can improve safety even with a partial market penetration rate; - message repropagation is important to reduce the risk of accidents when not all vehicles are equipped; - benefits are gained not only by equipped vehicles but also by unequipped ones