Intelligent and High-Performance Behavior Design of Autonomous Systems via Learning, Optimization and Control

Nowadays, great societal demands have rapidly boosted the development of autonomous systems that densely interact with humans in many application domains, from manufacturing to transportation and from workplaces to daily lives. The shift from isolated working environments to human-dominated space requires autonomous systems to be empowered to handle not only environmental uncertainties such as external vibrations but also interaction uncertainties arising from human behavior which is in nature probabilistic, causal but not strictly rational, internally hierarchical and socially compliant.This dissertation is concerned with the design of intelligent and high-performance behavior of such autonomous systems, leveraging the strength from control, optimization, learning, and cognitive science. The work consists of two parts. In Part I, the problem of high-level hybrid human-machine behavior design is addressed. The goal is to achieve safe, efficient and human-like interaction with people. A framework based on the theory of mind, utility theories and imitation learning is proposed to efficiently represent and learn the complicated behavior of humans. Built upon that, machine behaviors at three different levels - the perceptual level, the reasoning level, and the action level - are designed via imitation learning, optimization, and online adaptation, allowing the system to interpret, reason and behave as human, particularly when a variety of uncertainties exist. Applications to autonomous driving are considered throughout Part I. Part II is concerned with the design of high-performance low-level individual machine behavior in the presence of model uncertainties and external disturbances. Advanced control laws based on adaptation, iterative learning and the internal structures of uncertainties/disturbances are developed to assure that the high-level interactive behaviors can be reliably executed. Applications on robot manipulators and high-precision motion systems are discussed in this part

Reset controller design based on error minimization for a lane change maneuver

An intelligent vehicle must face a wide variety of situations ranging from safe and comfortable to more aggressive ones. Smooth maneuvers are adequately addressed by means of linear control, whereas more aggressive maneuvers are tackled by nonlinear techniques. Likewise, there exist intermediate scenarios where the required responses are smooth but constrained in some way (rise time, settling time, overshoot). Due to the existence of the fundamental linear limitations, which impose restrictions on the attainable time-domain and frequency-domain performance, linear systems cannot provide smoothness while operating in compliance with the previous restrictions. For this reason, this article aims to explore the effects of reset control on the alleviation of these limitations for a lane change maneuver under a set of demanding design conditions to guarantee a suitable ride quality and a swift response. To this end, several reset strategies are considered, determining the best reset condition to apply as well as the magnitude thereto. Concerning the reset condition that triggers the reset action, three strategies are considered: zero crossing of the controller input, fixed reset band and variable reset band. As far as the magnitude of the reset action is concerned, a full-reset technique is compared to a Lyapunov-based error minimization method to calculate the optimal reset percentage. The base linear controller subject to the reset action is searched via genetic algorithms. The proposed controllers are validated by means of CarSim.Agencia Estatal de Investigación | Ref. DPI2016-79278-C2-2-

Behavior Planning For Connected Autonomous Vehicles Using Feedback Deep Reinforcement Learning

With the development of communication technologies, connected autonomous vehicles (CAVs) can share information with each other. We propose a novel behavior planning method for CAVs to decide actions such as whether to change lane or keep lane based on the observation and shared information from neighbors, and to make sure that there exist corresponding control maneuvers such as acceleration and steering angle to guarantee the safety of each individual autonomous vehicle. We formulate this problem as a hybrid partially observable Markov decision process (HPOMDP) to consider objectives such as improving traffic flow efficiency and driving comfort and safety requirements. The discrete state transition is determined by the proposed feedback deep Q-learning algorithm using the feedback action from an underlying controller based on control barrier functions. The feedback deep Q-learning algorithm we design aims to solve the critical challenge of reinforcement learning (RL) in a physical system: guaranteeing the safety of the system while the RL is exploring the action space to increase the reward. We prove that our method renders a forward invariant safe set for the continuous state physical dynamic model of the system while the RL agent is learning. In experiments, our behavior planning method can increase traffic flow and driving comfort compared with the intelligent driving model (IDM). We also validate that our method maintains safety during the learning process.Comment: conferenc

An ensemble deep learning approach for driver lane change intention inference

With the rapid development of intelligent vehicles, drivers are increasingly likely to share their control authorities with the intelligent control unit. For building an efficient Advanced Driver Assistance Systems (ADAS) and shared-control systems, the vehicle needs to understand the drivers’ intent and their activities to generate assistant and collaborative control strategies. In this study, a driver intention inference system that focuses on the highway lane change maneuvers is proposed. First, a high-level driver intention mechanism and framework are introduced. Then, a vision-based intention inference system is proposed, which captures the multi-modal signals based on multiple low-cost cameras and the VBOX vehicle data acquisition system. A novel ensemble bi-directional recurrent neural network (RNN) model with Long Short-Term Memory (LSTM) units is proposed to deal with the time-series driving sequence and the temporal behavioral patterns. Naturalistic highway driving data that consists of lane-keeping, left and right lane change maneuvers are collected and used for model construction and evaluation. Furthermore, the driver's pre-maneuver activities are statistically analyzed. It is found that for situation-aware, drivers usually check the mirrors for more than six seconds before they initiate the lane change maneuver, and the time interval between steering the handwheel and crossing the lane is about 2 s on average. Finally, hypothesis testing is conducted to show the significant improvement of the proposed algorithm over existing ones. With five-fold cross-validation, the EBiLSTM model achieves an average accuracy of 96.1% for the intention that is inferred 0.5 s before the maneuver starts

Driver lane change intention inference for intelligent vehicles: framework, survey, and challenges

Intelligent vehicles and advanced driver assistance systems (ADAS) need to have proper awareness of the traffic context as well as the driver status since ADAS share the vehicle control authorities with the human driver. This study provides an overview of the ego-vehicle driver intention inference (DII), which mainly focus on the lane change intention on highways. First, a human intention mechanism is discussed in the beginning to gain an overall understanding of the driver intention. Next, the ego-vehicle driver intention is classified into different categories based on various criteria. A complete DII system can be separated into different modules, which consists of traffic context awareness, driver states monitoring, and the vehicle dynamic measurement module. The relationship between these modules and the corresponding impacts on the DII are analyzed. Then, the lane change intention inference (LCII) system is reviewed from the perspective of input signals, algorithms, and evaluation. Finally, future concerns and emerging trends in this area are highlighted

Driver lane change intention inference using machine learning methods.

Lane changing manoeuvre on highway is a highly interactive task for human drivers. The intelligent vehicles and the advanced driver assistance systems (ADAS) need to have proper awareness of the traffic context as well as the driver. The ADAS also need to understand the driver potential intent correctly since it shares the control authority with the human driver. This study provides a research on the driver intention inference, particular focus on the lane change manoeuvre on highways. This report is organised in a paper basis, where each chapter corresponding to a publication, which is submitted or to be submitted. Part Ⅰ introduce the motivation and general methodology framework for this thesis. Part Ⅱ includes the literature survey and the state-of-art of driver intention inference. Part Ⅲ contains the techniques for traffic context perception that focus on the lane detection. A literature review on lane detection techniques and its integration with parallel driving framework is proposed. Next, a novel integrated lane detection system is designed. Part Ⅳ contains two parts, which provides the driver behaviour monitoring system for normal driving and secondary tasks detection. The first part is based on the conventional feature selection methods while the second part introduces an end-to-end deep learning framework. The design and analysis of driver lane change intention inference system for the lane change manoeuvre is proposed in Part Ⅴ. Finally, discussions and conclusions are made in Part Ⅵ. A major contribution of this project is to propose novel algorithms which accurately model the driver intention inference process. Lane change intention will be recognised based on machine learning (ML) methods due to its good reasoning and generalizing characteristics. Sensors in the vehicle are used to capture context traffic information, vehicle dynamics, and driver behaviours information. Machine learning and image processing are the techniques to recognise human driver behaviour.PhD in Transpor

TMA (Truck Mounted Attenuators) alert system-development and testing

Truck Mounted Attenuators (TMAs) play a crucial role in safety of work zones as they decrease the impact of the crashes, reduce fatalities and injuries, and increase safety. However, there are almost no solid solutions to decrease the number of crashes with the TMA truck while maintaining the safety of the work zone workers. In this study, we aim to alarm the drivers following the TMA truck to avoid collisions and consequently, decrease the number and severity of the crashes. We used Unity 3D as the simulator to create scenarios with a smooth 45- degree and steeper 90-degree curves. Then we ran the simulator with different vehicles volumes and speed, including aggressive drivers in the simulator, and overall, creating different crash scenarios involving TMA trucks making some scenarios that some cars crash with the TMA truck which are rare in real life. Furthermore, computer vision has been used to define a safety zone on simulator videos to automate triggering the alarm when necessary to avoid crashes when vehicles cross the safety zone boundaries on the same lane as the TMA truck. After that, we used field videos from a TMA truck to evaluate our system. Results show that the proposed system achieved an average accuracy of 76.6 percent and 65 percent in simulator videos and TMA field video respectively. The only downside is having a fixed safety zone which causes problems when the geometry of the road changes or the TMA truck rotates to some degree and causes false alarms for the vehicles passing in the other lane. Overall, this system showed promising results and can be implemented in real-time for the TMAs to reduce collisions.Includes bibliographical references