Predictive Maneuver Planning and Control of an Autonomous Vehicle in Multi-Vehicle Traffic with Observation Uncertainty

Autonomous vehicle technology is a promising development for improving the safety, efficiency and environmental impact of on-road transportation systems. However, the task of guiding an autonomous vehicle by rapidly and systematically accommodating the plethora of changing constraints, e.g. of avoiding multiple stationary and moving obstacles, obeying traffic rules, signals and so on as well as the uncertain state observation due to sensor imperfections, remains a major challenge. This dissertation attempts to address this challenge via designing a robust and efficient predictive motion planning framework that can generate the appropriate vehicle maneuvers (selecting and tracking specific lanes, and related speed references) as well as the constituent motion trajectories while considering the differential vehicle kinematics of the controlled vehicle and other constraints of operating in public traffic. The main framework combines a finite state machine (FSM)-based maneuver decision module with a model predictive control (MPC)-based trajectory planner. Based on the prediction of the traffic environment, reference speeds are assigned to each lane in accordance with the detection of objects during measurement update. The lane selection decisions themselves are then incorporated within the MPC optimization. The on-line maneuver/motion planning effort for autonomous vehicles in public traffic is a non-convex problem due to the multiple collision avoidance constraints with overlapping areas, lane boundaries, and nonlinear vehicle-road dynamics constraints. This dissertation proposes and derives some remedies for these challenges within the planning framework to improve the feasibility and optimality of the solution. Specifically, it introduces vehicle grouping notions and derives conservative and smooth algebraic models to describe the overlapped space of several individual infeasible spaces and help prevent the optimization from falling into undesired local minima. Furthermore, in certain situations, a forced objective selection strategy is needed and adopted to help the optimization jump out of local minima. Furthermore, the dissertation considers stochastic uncertainties prevalent in dynamic and complex traffic and incorporate them with in the predictive planning and control framework. To this end, Bayesian filters are implemented to estimate the uncertainties in object motions and then propagate them into the prediction horizon. Then, a pair-wise probabilistic collision condition is defined for objects with non-negligible geometrical shape/sizes and computationally efficient and conservative forms are derived to efficiently and analytically approximate the involved multi-variate integrals. The probabilistic collision evaluation is then applied within a vehicle grouping algorithms to cluster the object vehicles with closeness in positions and speeds and eventually within the stochastic predictive maneuver planner framework to tighten the chanced-constraints given a deterministic confidence margin. It is argued that these steps make the planning problem tractable for real-time implementation on autonomously controlled vehicles

Human Motion Trajectory Prediction: A Survey

With growing numbers of intelligent autonomous systems in human environments, the ability of such systems to perceive, understand and anticipate human behavior becomes increasingly important. Specifically, predicting future positions of dynamic agents and planning considering such predictions are key tasks for self-driving vehicles, service robots and advanced surveillance systems. This paper provides a survey of human motion trajectory prediction. We review, analyze and structure a large selection of work from different communities and propose a taxonomy that categorizes existing methods based on the motion modeling approach and level of contextual information used. We provide an overview of the existing datasets and performance metrics. We discuss limitations of the state of the art and outline directions for further research.Comment: Submitted to the International Journal of Robotics Research (IJRR), 37 page

Multi Sensor Multi Target Perception and Tracking for Informed Decisions in Public Road Scenarios

Multi-target tracking in public traffic calls for a tracking system with automated track initiation and termination facilities in a randomly evolving driving environment. Besides, the key problem of data association needs to be handled effectively considering the limitations in the computational resources on-board an autonomous car. The challenge of the tracking problem is further evident in the use of high-resolution automotive sensors which return multiple detections per object. Furthermore, it is customary to use multiple sensors that cover different and/or over-lapping Field of View and fuse sensor detections to provide robust and reliable tracking. As a consequence, in high-resolution multi-sensor settings, the data association uncertainty, and the corresponding tracking complexity increases pointing to a systematic approach to handle and process sensor detections. In this work, we present a multi-target tracking system that addresses target birth/initiation and death/termination processes with automatic track management features. These tracking functionalities can help facilitate perception during common events in public traffic as participants (suddenly) change lanes, navigate intersections, overtake and/or brake in emergencies, etc. Various tracking approaches including the ones based on joint integrated probability data association (JIPDA) filter, Linear Multi-target Integrated Probabilistic Data Association (LMIPDA) Filter, and their multi-detection variants are adapted to specifically include algorithms that handle track initiation and termination, clutter density estimation and track management. The utility of the filtering module is further elaborated by integrating it into a trajectory tracking problem based on model predictive control. To cope with tracking complexity in the case of multiple high-resolution sensors, we propose a hybrid scheme that combines the approaches of data clustering at the local sensor and multiple detections tracking schemes at the fusion layer. We implement a track-to-track fusion scheme that de-correlates local (sensor) tracks to avoid double counting and apply a measurement partitioning scheme to re-purpose the LMIPDA tracking algorithm to multi-detection cases. In addition to the measurement partitioning approach, a joint extent and kinematic state estimation scheme are integrated into the LMIPDA approach to facilitate perception and tracking of an individual as well as group targets as applied to multi-lane public traffic. We formulate the tracking problem as a two hierarchical layer. This arrangement enhances the multi-target tracking performance in situations including but not limited to target initialization(birth process), target occlusion, missed detections, unresolved measurement, target maneuver, etc. Also, target groups expose complex individual target interactions to help in situation assessment which is challenging to capture otherwise. The simulation studies are complemented by experimental studies performed on single and multiple (group) targets. Target detections are collected from a high-resolution radar at a frequency of 20Hz; whereas RTK-GPS data is made available as ground truth for one of the target vehicle\u27s trajectory

Intelligent and High-Performance Behavior Design of Autonomous Systems via Learning, Optimization and Control

Nowadays, great societal demands have rapidly boosted the development of autonomous systems that densely interact with humans in many application domains, from manufacturing to transportation and from workplaces to daily lives. The shift from isolated working environments to human-dominated space requires autonomous systems to be empowered to handle not only environmental uncertainties such as external vibrations but also interaction uncertainties arising from human behavior which is in nature probabilistic, causal but not strictly rational, internally hierarchical and socially compliant.This dissertation is concerned with the design of intelligent and high-performance behavior of such autonomous systems, leveraging the strength from control, optimization, learning, and cognitive science. The work consists of two parts. In Part I, the problem of high-level hybrid human-machine behavior design is addressed. The goal is to achieve safe, efficient and human-like interaction with people. A framework based on the theory of mind, utility theories and imitation learning is proposed to efficiently represent and learn the complicated behavior of humans. Built upon that, machine behaviors at three different levels - the perceptual level, the reasoning level, and the action level - are designed via imitation learning, optimization, and online adaptation, allowing the system to interpret, reason and behave as human, particularly when a variety of uncertainties exist. Applications to autonomous driving are considered throughout Part I. Part II is concerned with the design of high-performance low-level individual machine behavior in the presence of model uncertainties and external disturbances. Advanced control laws based on adaptation, iterative learning and the internal structures of uncertainties/disturbances are developed to assure that the high-level interactive behaviors can be reliably executed. Applications on robot manipulators and high-precision motion systems are discussed in this part

Cooperative Perception for Social Driving in Connected Vehicle Traffic

The development of autonomous vehicle technology has moved to the center of automotive research in recent decades. In the foreseeable future, road vehicles at all levels of automation and connectivity will be required to operate safely in a hybrid traffic where human operated vehicles (HOVs) and fully and semi-autonomous vehicles (AVs) coexist. Having an accurate and reliable perception of the road is an important requirement for achieving this objective. This dissertation addresses some of the associated challenges via developing a human-like social driver model and devising a decentralized cooperative perception framework. A human-like driver model can aid the development of AVs by building an understanding of interactions among human drivers and AVs in a hybrid traffic, therefore facilitating an efficient and safe integration. The presented social driver model categorizes and defines the driver\u27s psychological decision factors in mathematical representations (target force, object force, and lane force). A model predictive control (MPC) is then employed for the motion planning by evaluating the prevailing social forces and considering the kinematics of the controlled vehicle as well as other operating constraints to ensure a safe maneuver in a way that mimics the predictive nature of the human driver\u27s decision making process. A hierarchical model predictive control structure is also proposed, where an additional upper level controller aggregates the social forces over a longer prediction horizon upon the availability of an extended perception of the upcoming traffic via vehicular networking. Based on the prediction of the upper level controller, a sequence of reference lanes is passed to a lower level controller to track while avoiding local obstacles. This hierarchical scheme helps reduce unnecessary lane changes resulting in smoother maneuvers. The dynamic vehicular communication environment requires a robust framework that must consistently evaluate and exploit the set of communicated information for the purpose of improving the perception of a participating vehicle beyond the limitations. This dissertation presents a decentralized cooperative perception framework that considers uncertainties in traffic measurements and allows scalability (for various settings of traffic density, participation rate, etc.). The framework utilizes a Bhattacharyya distance filter (BDF) for data association and a fast covariance intersection fusion scheme (FCI) for the data fusion processes. The conservatism of the covariance intersection fusion scheme is investigated in comparison to the traditional Kalman filter (KF), and two different fusion architectures: sensor-to-sensor and sensor-to-system track fusion are evaluated. The performance of the overall proposed framework is demonstrated via Monte Carlo simulations with a set of empirical communications models and traffic microsimulations where each connected vehicle asynchronously broadcasts its local perception consisting of estimates of the motion states of self and neighboring vehicles along with the corresponding uncertainty measures of the estimates. The evaluated framework includes a vehicle-to-vehicle (V2V) communication model that considers intermittent communications as well as a model that takes into account dynamic changes in an individual vehicle’s sensors’ FoV in accordance with the prevailing traffic conditions. The results show the presence of optimality in participation rate, where increasing participation rate beyond a certain level adversely affects the delay in packet delivery and the computational complexity in data association and fusion processes increase without a significant improvement in the achieved accuracy via the cooperative perception. In a highly dense traffic environment, the vehicular network can often be congested leading to limited bandwidth availability at high participation rates of the connected vehicles in the cooperative perception scheme. To alleviate the bandwidth utilization issues, an information-value discriminating networking scheme is proposed, where each sender broadcasts selectively chosen perception data based on the novelty-value of information. The potential benefits of these approaches include, but are not limited to, the reduction of bandwidth bottle-necking and the minimization of the computational cost of data association and fusion post processing of the shared perception data at receiving nodes. It is argued that the proposed information-value discriminating communication scheme can alleviate these adverse effects without sacrificing the fidelity of the perception

Optimisation of Mobile Communication Networks - OMCO NET

The mini conference “Optimisation of Mobile Communication Networks” focuses on advanced methods for search and optimisation applied to wireless communication networks. It is sponsored by Research & Enterprise Fund Southampton Solent University. The conference strives to widen knowledge on advanced search methods capable of optimisation of wireless communications networks. The aim is to provide a forum for exchange of recent knowledge, new ideas and trends in this progressive and challenging area. The conference will popularise new successful approaches on resolving hard tasks such as minimisation of transmit power, cooperative and optimal routing

Cooperative multi-sensor tracking of vulnerable road users in the presence of missing detections

This paper presents a vulnerable road user (VRU) tracking algorithm capable of handling noisy and missing detections from heterogeneous sensors. We propose a cooperative fusion algorithm for matching and reinforcing of radar and camera detections using their proximity and positional uncertainty. The belief in the existence and position of objects is then maximized by temporal integration of fused detections by a multi-object tracker. By switching between observation models, the tracker adapts to the detection noise characteristics making it robust to individual sensor failures. The main novelty of this paper is an improved imputation sampling function for updating the state when detections are missing. The proposed function uses a likelihood without association that is conditioned on the sensor information instead of the sensor model. The benefits of the proposed solution are two-fold: firstly, particle updates become computationally tractable and secondly, the problem of imputing samples from a state which is predicted without an associated detection is bypassed. Experimental evaluation shows a significant improvement in both detection and tracking performance over multiple control algorithms. In low light situations, the cooperative fusion outperforms intermediate fusion by as much as 30%, while increases in tracking performance are most significant in complex traffic scenes

Probabilistic Framework for Behavior Characterization of Traffic Participants Enabling Long Term Prediction

This research aims at developing new methods that predict the behaviors of the human driven traffic participants to enable safe operation of autonomous vehicles in complex traffic environments. Autonomous vehicles are expected to operate amongst human driven conventional vehicles in the traffic at least for the next few decades. For safe navigation they will need to infer the intents as well as the behaviors of the human traffic participants using extrinsically observable information, so that their trajectories can be predicted for a time horizon long enough to do a predictive risk analysis and gracefully avert any risky situation. This research approaches this challenge by recognizing that any maneuver performed by a human driver can be divided into four stages that depend on the surrounding context: intent determination, maneuver preparation, gap acceptance and maneuver execution. It builds on the hypothesis that for a given driver, the behavior not only spans across these four maneuver stages, but across multiple maneuvers. As a result, identifying the driver behavior in any of these stages can help characterize the nature of all the subsequent maneuvers that the driver is likely to perform, thus resulting in a more accurate prediction for a longer time horizon. To enable this, a novel probabilistic framework is proposed that couples the different maneuver stages of the observed traffic participant together and associates them to a driving style. To realize this framework two candidate Multiple Model Adaptive Estimation approaches were compared: Autonomous Multiple Model (AMM) and Interacting Multiple Model(IMM) filtering approach. The IMM approach proved superior to the AMM approach and was eventually validated using a trajectory extracted from a real world dataset for efficacy. The proposed framework was then implemented by extending the validated IMM approach with contextual information of the observed traffic participant. The classification of the driving style of the traffic participant (behavior characterization) was then demonstrated for two use case scenarios. The proposed contextual IMM (CIMM) framework also showed improvements in the performance of the behavior classification of the traffic participants compared to the IMM for the identified use case scenarios. This outcome warrants further exploration of this framework for different traffic scenarios. Further, it contributes towards the ongoing endeavors for safe deployment of autonomous vehicles on public roads

Fusion of Data from Heterogeneous Sensors with Distributed Fields of View and Situation Evaluation for Advanced Driver Assistance Systems

In order to develop a driver assistance system for pedestrian protection, pedestrians in the environment of a truck are detected by radars and a camera and are tracked across distributed fields of view using a Joint Integrated Probabilistic Data Association filter. A robust approach for prediction of the system vehicles trajectory is presented. It serves the computation of a probabilistic collision risk based on reachable sets where different sources of uncertainty are taken into account