29 research outputs found
Towards Learning Feasible Hierarchical Decision-Making Policies in Urban Autonomous Driving
Modern learning-based algorithms, powered by advanced deep structured neural nets, have multifacetedly facilitated automated driving platforms, spanning from scene characterization and perception to low-level control and state estimation schemes. Nonetheless, urban autonomous driving is regarded as a challenging application for machine learning (ML) and artificial intelligence (AI) since the learnt driving policies must handle complex multi-agent driving scenarios with indeterministic intentions of road participants. In the case of unsignalized intersections, automating the decision-making process at these safety-critical environments entails comprehending numerous layers of abstractions associated with learning robust driving behaviors to allow the vehicle to drive safely and efficiently.
Based on our in-depth investigation, we discern that an efficient, yet safe, decision-making scheme for navigating real-world unsignalized intersections does not exist yet. The state-of-the-art schemes lacked practicality to handle real-life complex scenarios as they utilize Low-fidelity vehicle dynamic models which makes them incapable of simulating the real dynamic motion in real-life driving applications. In addition, the conservative behavior of autonomous vehicles, which often overreact to threats which have low likelihood, degrades the overall driving quality and jeopardizes safety. Hence, enhancing driving behavior is essential to attain agile, yet safe, traversing maneuvers in such multi-agent environments. Therefore, the main goal of conducting this PhD research is to develop high-fidelity learning-based frameworks to enhance the autonomous decision-making process at these safety-critical environments.
We focus this PhD dissertation on three correlated and complementary research challenges. In our first research challenge, we conduct an in-depth and comprehensive survey on the state-of-the-art learning-based decision-making schemes with the objective of identifying the main shortcomings and potential research avenues. Based on the research directions concluded, we propose, in Problem II and Problem III, novel learning-based frameworks with the objective of enhancing safety and efficiency at different decision-making levels. In Problem II, we develop a novel sensor-independent state estimation for a safety-critical system in urban driving using deep learning techniques. A neural inference model is developed and trained via deep-learning training techniques to obtain accurate state estimates using indirect measurements of vehicle dynamic states and powertrain states. In Problem III, we propose a novel hierarchical reinforcement learning-based decision-making architecture for learning left-turn policies at four-way unsignalized intersections with feasibility guarantees. The proposed technique involves an integration of two main decision-making layers; a high-level learning-based behavioral planning layer which adopts soft actor-critic principles to learn high-level, non-conservative yet safe, driving behaviors, and a motion planning layer that uses low-level Model Predictive Control (MPC) principles to ensure feasibility of the two-dimensional left-turn maneuver. The high-level layer generates reference signals of velocity and yaw angle for the ego vehicle taking into account safety and collision avoidance with the intersection vehicles, whereas the low-level planning layer solves an optimization problem to track these reference commands considering several vehicle dynamic constraints and ride comfort
Multi-Agent Chance-Constrained Stochastic Shortest Path with Application to Risk-Aware Intelligent Intersection
In transportation networks, where traffic lights have traditionally been used
for vehicle coordination, intersections act as natural bottlenecks. A
formidable challenge for existing automated intersections lies in detecting and
reasoning about uncertainty from the operating environment and human-driven
vehicles. In this paper, we propose a risk-aware intelligent intersection
system for autonomous vehicles (AVs) as well as human-driven vehicles (HVs). We
cast the problem as a novel class of Multi-agent Chance-Constrained Stochastic
Shortest Path (MCC-SSP) problems and devise an exact Integer Linear Programming
(ILP) formulation that is scalable in the number of agents' interaction points
(e.g., potential collision points at the intersection). In particular, when the
number of agents within an interaction point is small, which is often the case
in intersections, the ILP has a polynomial number of variables and constraints.
To further improve the running time performance, we show that the collision
risk computation can be performed offline. Additionally, a trajectory
optimization workflow is provided to generate risk-aware trajectories for any
given intersection. The proposed framework is implemented in CARLA simulator
and evaluated under a fully autonomous intersection with AVs only as well as in
a hybrid setup with a signalized intersection for HVs and an intelligent scheme
for AVs. As verified via simulations, the featured approach improves
intersection's efficiency by up to while also conforming to the
specified tunable risk threshold
EMVLight: a Multi-agent Reinforcement Learning Framework for an Emergency Vehicle Decentralized Routing and Traffic Signal Control System
Emergency vehicles (EMVs) play a crucial role in responding to time-critical
calls such as medical emergencies and fire outbreaks in urban areas. Existing
methods for EMV dispatch typically optimize routes based on historical
traffic-flow data and design traffic signal pre-emption accordingly; however,
we still lack a systematic methodology to address the coupling between EMV
routing and traffic signal control. In this paper, we propose EMVLight, a
decentralized reinforcement learning (RL) framework for joint dynamic EMV
routing and traffic signal pre-emption. We adopt the multi-agent advantage
actor-critic method with policy sharing and spatial discounted factor. This
framework addresses the coupling between EMV navigation and traffic signal
control via an innovative design of multi-class RL agents and a novel
pressure-based reward function. The proposed methodology enables EMVLight to
learn network-level cooperative traffic signal phasing strategies that not only
reduce EMV travel time but also shortens the travel time of non-EMVs.
Simulation-based experiments indicate that EMVLight enables up to a
reduction in EMV travel time as well as an shorter average travel time
compared with existing approaches.Comment: 19 figures, 10 tables. Manuscript extended on previous work
arXiv:2109.05429, arXiv:2111.0027
Developments in Estimation and Control for Cloud-Enabled Automotive Vehicles.
Cloud computing is revolutionizing access to distributed information and computing resources that can facilitate future data and computation intensive vehicular control functions and improve vehicle driving comfort and safety. This dissertation investigates several potential Vehicle-to-Cloud-to-Vehicle (V2C2V) applications that can enhance vehicle control and enable additional functionalities by integrating onboard and cloud resources.
Firstly, this thesis demonstrates that onboard vehicle sensors can be used to sense road profiles and detect anomalies. This information can be shared with other vehicles and transportation authorities within a V2C2V framework. The response of hitting a pothole is characterized by a multi-phase dynamic model which is validated by comparing simulation results with a higher-fidelity commercial modeling package. A novel framework of simultaneous road profile estimation and anomaly detection is developed by combining a jump diffusion process (JDP)-based estimator and a multi-input observer. The performance of this scheme is evaluated in an experimental vehicle. In addition, a new clustering algorithm is developed to compress anomaly information by processing anomaly report streams.
Secondly, a cloud-aided semi-active suspension control problem is studied demonstrating for the first time that road profile information and noise statistics from the cloud can be used to enhance suspension control. The problem of selecting an optimal damping mode from a finite set of damping modes is considered and the best mode is selected based on performance prediction on the cloud.
Finally, a cloud-aided multi-metric route planner is investigated in which safety and comfort metrics augment traditional planning metrics such as time, distance, and fuel economy. The safety metric is developed by processing a comprehensive road and crash database while the comfort metric integrates road roughness and anomalies. These metrics and a planning algorithm can be implemented on the cloud to realize the multi-metric route planning. Real-world case studies are presented. The main contribution of this part of the dissertation is in demonstrating the feasibility and benefits of enhancing the existing route planning algorithms with safety and comfort metrics.PhDAerospace EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/120710/1/zhaojli_1.pd
New Perspectives on Modelling and Control for Next Generation Intelligent Transport Systems
This PhD thesis contains 3 major application areas all within an Intelligent Transportation
System context.
The first problem we discuss considers models that make beneficial use of the large
amounts of data generated in the context of traffic systems. We use a Markov chain
model to do this, where important data can be taken into account in an aggregate form.
The Markovian model is simple and allows for fast computation, even on low end computers,
while at the same time allowing meaningful insight into a variety of traffic system
related issues. This allows us to both model and enable the control of aggregate, macroscopic
features of traffic networks. We then discuss three application areas for this model:
the modelling of congestion, emissions, and the dissipation of energy in electric vehicles.
The second problem we discuss is the control of pollution emissions in
eets of hybrid
vehicles. We consider parallel hybrids that have two power units, an internal combustion
engine and an electric motor. We propose a scheme in which we can in
uence the mix
of the two engines in each car based on simple broadcast signals from a central infrastructure.
The infrastructure monitors pollution levels and can thus make the vehicles
react to its changes. This leads to a context aware system that can be used to avoid pollution
peaks, yet does not restrict drivers unnecessarily. In this context we also discuss
technical constraints that have to be taken into account in the design of traffic control
algorithms that are of a microscopic nature, i.e. they affect the operation of individual
vehicles. We also investigate ideas on decentralised trading of emissions. The goal here
is to allocate the rights to pollute fairly among the
eet's vehicles.
Lastly we discuss the usage of decentralised stochastic assignment strategies in traffic
applications. Systems are considered in which reservation schemes can not reliably be
provided or enforced and there is a signifficant delay between decisions and their effect. In
particular, our approach facilitates taking into account the feedback induced into traffic
systems by providing forecasts to large groups of users. This feedback can invalidate the
predictions if not modelled carefully. At the same time our proposed strategies are simple
rules that are easy to follow, easy to accept, and significantly improve the performance
of the systems under study. We apply this approach to three application areas, the assignment
of electric vehicles to charging stations, the assignment of vehicles to parking
facilities, and the assignment of customers to bike sharing stations.
All discussed approaches are analysed using mathematical tools and validated through
extensive simulations
Recommended from our members
Constructive Formal Control Synthesis through Abstraction and Decomposition
Control synthesis is the problem of automatically constructing a control strategy that induces a system to exhibit a declared behavior. Synthesis algorithms vary widely across different classes of system dynamics and specifications.While continuous optimization has traditionally been used to construct stabilizing controllers for physical systems modeled with differential equations, temporal logic synthesis for finite state machines heavily leverages discrete algorithms and data structures.Hybrid systems are a class of systems that exhibit both continuous and discrete behaviors, which are necessary to capture phenomena such as impacts for legged robots and congestion shockwaves in freeways. Tractable control synthesis remains elusive because hybrid systems violate many of the fundamental topological assumptions made by prior algorithms for purely continuous or discrete systems.This thesis exploits compositionality and system structure to provide a suite of algorithmic and theoretical techniques to tackle acute computational bottlenecks in hybrid control synthesis.The first half of this thesis provides a framework for engineers to model control systems and construct algorithmic pipelines for control synthesis.By explicitly capturing system structure, this framework gives users the flexibility to rapidly iterate over and leverage a library of optimizations for control synthesis.We demonstrate this framework in the context of abstraction-based control, a synthesis workflow that translates continuous systems into finite state machines by throwing away high precision information. Different optimization techniques such as multi-scale grids, lazy abstraction, and decomposed synthesis, can all be expressed as modifications to a computational pipeline. We demonstrate computational gains while synthesizing safe motion primitives for numerous robotic examples.The second half addresses distributed control synthesis where multiple controllers act as agents that seek to jointly satisfy a specification and are restricted by some communication topology. We introduce parametric assume-guarantee contracts as a formalism to derive guarantees about the closed loop behavior of a collection of interacting components. Dynamic contracts allow contract parameters to change at runtime and enable coordination of multiple interacting sub-systems.These results are demonstrated in the context of a freeway ramp meter and an adjacent arterial network
Review of graph-based hazardous event detection methods for autonomous driving systems
Automated and autonomous vehicles are often required to operate in complex road environments with potential hazards that may lead to hazardous events causing injury or even death. Therefore, a reliable autonomous hazardous event detection system is a key enabler for highly autonomous vehicles (e.g., Level 4 and 5 autonomous vehicles) to operate without human supervision for significant periods of time. One promising solution to the problem is the use of graph-based methods that are powerful tools for relational reasoning. Using graphs to organise heterogeneous knowledge about the operational environment, link scene entities (e.g., road users, static objects, traffic rules) and describe how they affect each other. Due to a growing interest and opportunity presented by graph-based methods for autonomous hazardous event detection, this paper provides a comprehensive review of the state-of-the-art graph-based methods that we categorise as rule-based, probabilistic, and machine learning-driven. Additionally, we present an in-depth overview of the available datasets to facilitate hazardous event training and evaluation metrics to assess model performance. In doing so, we aim to provide a thorough overview and insight into the key research opportunities and open challenges
Resource allocation and congestion control in vehicular ad-hoc networks through optimization algorithms and artificial intelligence
[SPA] Esta tesis doctoral se presenta bajo la modalidad de compendio de publicaciones. En los 煤ltimos a帽os la creciente demanda de la industria del transporte junto con requisitos de seguridad cada vez m谩s estrictos han promovido el r谩pido desarrollo de las comunicaciones vehiculares. Tales comunicaciones se basan en el intercambio de mensajes peri贸dicos (beacons) que contienen informaci贸n cr铆tica de los veh铆culos. Esta difusi贸n de informaci贸n da origen a lo que com煤nmente se denomina conciencia cooperativa, que permite ampliar las capacidades de numerosos sistemas de asistencia en carretera y las diferentes aplicaciones de seguridad. Ciertamente, la difusi贸n de informaci贸n entre veh铆culos es la base de la conducci贸n aut贸noma y reduce dr谩sticamente el riesgo de colisi贸n y otros eventos indeseados. Sin embargo, es importante tener en cuenta que la carga agregada de los beacons transmitidos puede congestionar r谩pidamente el canal, comprometiendo la recepci贸n de paquetes y, por lo tanto, poniendo en peligro las ventajas que ofrecen tales comunicaciones. Para garantizar la disponibilidad del canal tanto para la recepci贸n correcta de mensajes de emergencia y de las m铆nimas balizas necesarias para satisfacer los requisitos de las aplicaciones de seguridad, una determinada fracci贸n del canal debe de ser reservada. En la literatura relacionada, el control de la congesti贸n se ha abordado mediante el ajuste de varios par谩metros de transmisi贸n (tasa de mensaje, potencia y tasa de bit), pero todav铆a existen numerosos desaf铆os por abordar. Por ejemplo, aunque los par谩metros de transmisi贸n suelen ajustarse individualmente debido a la simplicidad del problema de optimizaci贸n, aqu铆 se muestran las ventajas de ajustar varios par谩metros de forma simult谩nea. En esta tesis, se propone el uso de diferentes algoritmos distribuidos que alcancen el nivel de congesti贸n deseado sin requerir infraestructura ninguna en carretera. La primera parte de esta tesis aborda la asignaci贸n de la tasa de balizamiento mediante la maximizaci贸n de la utilidad de red (NUM) y diferentes m茅tricas de riesgo como el tiempo de colisi贸n y la velocidad de la carretera de aviso. En la segunda parte, no solo se estudian diferentes combinaciones consistentes de par谩metros, sino que tambi茅n nos sumergimos en el paradigma de los algoritmos no cooperativos, en los que no se requiere informaci贸n de los veh铆culos vecinos. El problema de control de la congesti贸n es formulado como un Proceso de Decisi贸n de Markov (MDP) y resuelto mediante t茅cnicas de inteligencia artificial, m谩s concretamente, mediante aprendizaje por refuerzo (RL). Se proponen diferentes soluciones que van desde simples m茅todos tabulares, adecuados para entornos discretos, como Q-learning, hasta funciones de aproximaci贸n m谩s complejas adecuadas para espacios continuos, como SARSA basado en semi-gradiente o redes neuronales artificiales.[ENG] This doctoral dissertation has been presented in the form of thesis by publication. The ever-increasing growth of the transportation industry demands combined with new safety requirements has triggered the development of vehicular communications. These communications among vehicles are based on the exchange of periodical messages or beacons containing valuable information about vehicle state. This gives rise to the socalled cooperative awareness, which allows extending the capabilities of numerous driver assistance systems and safety applications. Disseminating information among vehicles certainly lessens the risk of collision and other undesired events. Nevertheless, the aggregated beaconing load can rapidly jam the channel, compromising packet reception, and therefore endangering the advantages offered by such communications. To guarantee the availability of the channel for emergency messages and the minimum beacons receptions that satisfy safety application requirements, a given fraction of the channel capacity should be available. This congestion control has been addressed by adjusting several transmission parameters but some challenges are still unresolved. Although these parameters are usually optimized individually because of the convexity of the optimization problem, we show the advantages of combining them. In this thesis, we propose the use of different distributed algorithms that reach the desired congestion level without explicitly requiring any costly infrastructure. The first part of this thesis addresses beaconing rate allocation. We propose several distributed solutions based on Network Utility Maximization (NUM) and different risk metrics such as time-to-collision and advisory road speed. In the second part, we not only study different combinations of well-coupled parameters but also dive into the paradigm of noncooperative algorithms, in which no information from neighboring vehicles or centralized infrastructure are required. We formulate the congestion control problem as a Markov Decision Process and solve it by means of different reinforcement learning techniques. In particular, we propose different solutions ranging from tabular methods suitable for simple and discrete environments, like Q-learning, to more complex functions approximations for continuous action-state spaces, such as Semi-gradient SARSA or artificial neural networks.Esta tesis doctoral se presenta bajo la modalidad de compendio de publicaciones. Est谩 formada por estos seis art铆culos: 1. (j1) Aznar-Poveda, J., Egea-Lopez, E., Garcia-Sanchez, A. J., and Pavon-Mari帽o, P. (2019, October). Time-to-Collision-Based Awareness and Congestion Control for Vehicular Communications. IEEE Access, 7, 154192-154208. DOI: 10.1109/ACCESS.2019.2949131. 2. (c1) Aznar-Poveda, J., Egea-Lopez, E., and Garcia-Sanchez, A. J. (2020, May). Cooperative Awareness Message Dissemination in EN 302 637-2: An Adaptation for Winding Roads. IEEE 91st Vehicular Technology Conference (VTC2020-Spring) (pp. 1-5). IEEE. DOI: 10.1109/VTC2020-Spring48590.2020.9128815. 3. (c2) Aznar-Poveda, J., Egea-Lopez, E., Garcia-Sanchez, A. J., and Garcia-Haro, J. (2020, July). Advisory Speed Estimation for an Improved V2X Communications Awareness in Winding Roads. In 2020 22nd International Conference on Transparent Optical Networks (ICTON) (pp. 1-4). IEEE. DOI: 10.1109/ICTON51198.2020.9203478 4. (j2) Aznar-Poveda, J., Garcia-Sanchez, A. J., Egea-Lopez, E., and Garcia-Haro, J. (2021, January). MDPRP: A Q-Learning Approach for the Joint Control of Beaconing Rate and Transmission Power in VANETs. IEEE Access, 9, 10166-10178. DOI: 10.1109/ACCESS.2021.3050625 5. (j3) Aznar-Poveda, J., Garcia-Sanchez, A. J., Egea-Lopez, E., and Garcia-Haro, J. (2021, August). Simultaneous Data Rate and Transmission Power Adaptation in V2V Communications: A Deep Reinforcement Learning Approach. IEEE Access 9, 122067-122081. DOI: 10.1109/ACCESS.2021.3109422 6. (j4) Aznar-Poveda, J., Garcia-Sanchez, A. J., Egea-Lopez, E., and Garcia-Haro, J. (2021, December). Approximate Reinforcement Learning to Control Beaconing Congestion in Distributed Networks. Scientific Reports, 12, 142. DOI: 10.1038/s41598-021-04123-9Escuela Internacional de Doctorado de la Universidad Polit茅cnica de CartagenaUniversidad Polit茅cnica de CartagenaPrograma de Doctorado en Tecnolog铆as de la Informaci贸n y las Comunicacione
Proceedings, MSVSCC 2012
Proceedings of the 6th Annual Modeling, Simulation & Visualization Student Capstone Conference held on April 19, 2012 at VMASC in Suffolk, Virginia