Cooperative Adaptive Cruise Control Based on Reinforcement Learning for Heavy-Duty BEVs

Advanced driver assistance systems (ADAS) are playing an increasingly important role in supporting the driver to create safer and more efficient driving conditions. Among all ADAS, adaptive cruise control (ACC) is a system that provides consistent aid, especially in highway mobility, guaranteeing safety by minimizing the possible risk of collision due to variations in the speed of the vehicle in front, automatically adjusting the vehicle velocity and maintaining the correct spacing. Theoretically, this type of system also makes it possible to optimize road throughput, increasing its capacity and reducing traffic congestion. However, it was found in practice that the current generation of ACC systems does not guarantee the so-called string stability of a vehicle platoon and can therefore lead to an actual decrease in traffic capacity. To overcome these issues, new cooperative adaptive cruise control (CACC) systems are being proposed that exploit vehicle-to-vehicle (V2V) connectivity, which can provide additional safety and robustness guarantees and introduce the possibility of concretely improving traffic flow stability

A generic architecture style for self-adaptive cyber-physical systems

Die aktuellen Konzepte zur Gestaltung von Regelungssystemen basieren auf dynamischen Verhaltensmodellen, die mathematische Ansätze wie Differentialgleichungen zur Ableitung der entsprechenden Funktionen verwenden. Diese Konzepte stoßen jedoch aufgrund der zunehmenden Systemkomplexität allmählich an ihre Grenzen. Zusammen mit der Entwicklung dieser Konzepte entsteht eine Architekturevolution der Regelungssysteme. In dieser Dissertation wird eine Taxonomie definiert, um die genannte Architekturevolution anhand eines typischen Beispiels, der adaptiven Geschwindigkeitsregelung (ACC), zu veranschaulichen. Aktuelle ACC-Varianten, die auf der Regelungstheorie basieren, werden in Bezug auf ihre Architekturen analysiert. Die Analyseergebnisse zeigen, dass das zukünftige Regelungssystem im ACC eine umfangreichere Selbstadaptationsfähigkeit und Skalierbarkeit erfordert. Dafür sind kompliziertere Algorithmen mit unterschiedlichen Berechnungsmechanismen erforderlich. Somit wird die Systemkomplexität erhöht und führt dazu, dass das zukünftige Regelungssystem zu einem selbstadaptiven cyber-physischen System wird und signifikante Herausforderungen für die Architekturgestaltung des Systems darstellt. Inspiriert durch Ansätze des Software-Engineering zur Gestaltung von Architekturen von softwareintensiven Systemen wird in dieser Dissertation ein generischer Architekturstil entwickelt. Der entwickelte Architekturstil dient als Vorlage, um vernetzte Architekturen mit Verfolgung der entwickelten Designprinzipien nicht nur für die aktuellen Regelungssysteme, sondern auch für selbstadaptiven cyber-physischen Systeme in der Zukunft zu konstruieren. Unterschiedliche Auslösemechanismen und Kommunikationsparadigmen zur Gestaltung der dynamischen Verhalten von Komponenten sind in der vernetzten Architektur anwendbar. Zur Bewertung der Realisierbarkeit des Architekturstils werden aktuelle ACCs erneut aufgenommen, um entsprechende logische Architekturen abzuleiten und die Architekturkonsistenz im Vergleich zu den originalen Architekturen basierend auf der Regelungstheorie (z. B. in Form von Blockdiagrammen) zu untersuchen. Durch die Anwendung des entwickelten generischen Architekturstils wird in dieser Dissertation eine künstliche kognitive Geschwindigkeitsregelung (ACCC) als zukünftige ACC-Variante entworfen, implementiert und evaluiert. Die Evaluationsergebnisse zeigen signifikante Leistungsverbesserungen des ACCC im Vergleich zum menschlichen Fahrer und aktuellen ACC-Varianten.Current concepts of designing automatic control systems rely on dynamic behavioral modeling by using mathematical approaches like differential equations to derive corresponding functions, and slowly reach limitations due to increasing system complexity. Along with the development of these concepts, an architectural evolution of automatic control systems is raised. This dissertation defines a taxonomy to illustrate the aforementioned architectural evolution relying on a typical example of control application: adaptive cruise control (ACC). Current ACC variants, with their architectures considering control theory, are analyzed. The analysis results indicate that the future automatic control system in ACC requires more substantial self-adaptation capability and scalability. For this purpose, more complicated algorithms requiring different computation mechanisms must be integrated into the system and further increase system complexity. This makes the future automatic control system evolve into a self-adaptive cyber-physical system and consistitutes significant challenges for the system’s architecture design. Inspired by software engineering approaches for designing architectures of software-intensive systems, a generic architecture style is proposed. The proposed architecture style serves as a template by following the developed design principle to construct networked architectures not only for the current automatic control systems but also for self-adaptive cyber-physical systems in the future. Different triggering mechanisms and communication paradigms for designing dynamic behaviors are applicable in the networked architecture. To evaluate feasibility of the architecture style, current ACCs are retaken to derive corresponding logical architectures and examine architectural consistency compared to the previous architectures considering the control theory (e.g., in the form of block diagrams). By applying the proposed generic architecture style, an artificial cognitive cruise control (ACCC) is designed, implemented, and evaluated as a future ACC in this dissertation. The evaluation results show significant performance improvements in the ACCC compared to the human driver and current ACC variants

Assessing the effectiveness of managed lane strategies for the rapid deployment of cooperative adaptive cruise control technology

Connected and Automated Vehicle (C/AV) technologies are fast expanding in the transportation and automotive markets. One of the highly researched examples of C/AV technologies is the Cooperative Adaptive Cruise Control (CACC) system, which exploits various vehicular sensors and vehicle-to-vehicle communication to automate vehicular longitudinal control. The operational strategies and network-level impacts of CACC have not been thoroughly discussed, especially in near-term deployment scenarios where Market Penetration Rate (MPR) is relatively low. Therefore, this study aims to assess CACC\u27s impacts with a combination of managed lane strategies to provide insights for CACC deployment. The proposed simulation framework incorporates 1) the Enhanced Intelligent Driver Model; 2) Nakagami-based radio propagation model; and 3) a multi-objective optimization (MOOP)-based CACC control algorithm. The operational impacts of CACC are assessed under four managed lane strategies (i.e., mixed traffic (UML), HOV (High Occupancy Vehicle)-CACC lane (MML), CACC dedicated lane (DL), and CACC dedicated lane with access control (DLA)). Simulation results show that the introduction of CACC, even with 10% MPR, is able to improve the network throughput by 7% in the absence of any managed lane strategies. The segment travel times for both CACC and non-CACC vehicles are reduced. The break-even point for implementing dedicated CACC lane is 30% MPR, below which the priority usage of the current HOV lane for CACC traffic is found to be more appropriate. It is also observed that DLA strategy is able to consistently increase the percentage of platooned CACC vehicles as MPR grows. The percentage of CACC vehicles within a platoon reaches 52% and 46% for DL and DLA, respectively. When it comes to the impact of vehicle-to-vehicle (V2V), it is found that DLA strategy provides more consistent transmission density in terms of median and variance when MPR reaches 20% or above. Moreover, the performance of the MOOP-based cooperative driving is examined. With average 75% likelihood of obtaining a feasible solution, the MOOP outperforms its counterpart which aims to minimize the headway objective solely. In UML, MML, and DL strategy, the proposed control algorithm achieves a balance spread among four objectives for each CACC vehicle. In the DLA strategy, however, the probability of obtaining feasible solution falls to 60% due to increasing size of platoon owing to DLA that constraints the feasible region by introduction more dimensions in the search space. In summary, UML or MML is the preferred managed lane strategy for improving traffic performance when MPR is less than 30%. When MRP reaches 30% or above, DL and DLA could improve the CACC performance by facilitating platoon formation. If available, priority access to an existing HOV lane can be adopted to encourage adaptation of CACC when CACC technology becomes publically available

Communication-Efficient Decentralized Multi-Agent Reinforcement Learning for Cooperative Adaptive Cruise Control

Connected and autonomous vehicles (CAVs) promise next-gen transportation systems with enhanced safety, energy efficiency, and sustainability. One typical control strategy for CAVs is the so-called cooperative adaptive cruise control (CACC) where vehicles drive in platoons and cooperate to achieve safe and efficient transportation. In this study, we formulate CACC as a multi-agent reinforcement learning (MARL) problem. Diverging from existing MARL methods that use centralized training and decentralized execution which require not only a centralized communication mechanism but also dense inter-agent communication, we propose a fully-decentralized MARL framework for enhanced efficiency and scalability. In addition, a quantization-based communication scheme is proposed to reduce the communication overhead without significantly degrading the control performance. This is achieved by employing randomized rounding numbers to quantize each piece of communicated information and only communicating non-zero components after quantization. Extensive experimentation in two distinct CACC settings reveals that the proposed MARL framework consistently achieves superior performance over several contemporary benchmarks in terms of both communication efficiency and control efficacy.Comment: 11 pages, 7 figure

Leveraging Connected Highway Vehicle Platooning Technology to Improve the Efficiency and Effectiveness of Train Fleeting Under Moving Blocks

Future advanced Positive Train Control systems may allow North American railroads to introduce moving blocks with shorter train headways. This research examines how closely following trains respond to different throttle and brake inputs. Using insights from connected automobile and truck platooning technology, six different following train control algorithms were developed, analyzed for stability, and evaluated with simulated fleets of freight trains. While moving blocks require additional train spacing beyond minimum safe braking distance to account for train control actions, certain following train algorithms can help minimize this distance and balance fuel efficiency and train headway by changing control parameters