Evaluation of model predictive control method for collision avoidance of automated vehicles

Indiana University-Purdue University Indianapolis (IUPUI)Collision avoidance design plays an essential role in autonomous vehicle technology. It's an attractive research area that will need much experimentation in the future. This research area is very important for providing the maximum safety to automated vehicles, which have to be tested several times under diFFerent circumstances for safety before use in real life. This thesis proposes a method for designing and presenting a collision avoidance maneuver by using a model predictive controller with a moving obstacle for automated vehicles. It consists of a plant model, an adaptive MPC controller, and a reference trajectory. The proposed strategy applies a dynamic bicycle model as the plant model, adaptive model predictive controller for the lateral control, and a custom reference trajectory for the scenario design. The model was developed using the Model Predictive Control Toolbox and Automated Driving Toolbox in Matlab. Builtin tools available in Matlab/Simulink were used to verify the modeling approach and analyze the performance of the system. The major contribution of this thesis work was implementing a novel dynamic obstacle avoidance control method for automated vehicles. The study used validated parameters obtained from previous research. The novelty of this research was performing the studies using a MPC based controller instead of a sliding mode controller, that was primarily used in other studies. The results obtained from the study are compared with the validated models. The comparisons consisted of the lateral overlap, lateral error, and steering angle simulation results between the models. Additionally, this study also included outcomes for the yaw angle. The comparisons and other outcomes obtained in this study indicated that the developed control model produced reasonably acceptable results and recommendations for future studies

A Learning-based Stochastic MPC Design for Cooperative Adaptive Cruise Control to Handle Interfering Vehicles

Vehicle to Vehicle (V2V) communication has a great potential to improve reaction accuracy of different driver assistance systems in critical driving situations. Cooperative Adaptive Cruise Control (CACC), which is an automated application, provides drivers with extra benefits such as traffic throughput maximization and collision avoidance. CACC systems must be designed in a way that are sufficiently robust against all special maneuvers such as cutting-into the CACC platoons by interfering vehicles or hard braking by leading cars. To address this problem, a Neural- Network (NN)-based cut-in detection and trajectory prediction scheme is proposed in the first part of this paper. Next, a probabilistic framework is developed in which the cut-in probability is calculated based on the output of the mentioned cut-in prediction block. Finally, a specific Stochastic Model Predictive Controller (SMPC) is designed which incorporates this cut-in probability to enhance its reaction against the detected dangerous cut-in maneuver. The overall system is implemented and its performance is evaluated using realistic driving scenarios from Safety Pilot Model Deployment (SPMD).Comment: 10 pages, Submitted as a journal paper at T-I

자율 주행 차량의 심층강화학습 기반 긴급 차선 변경 경로 최적화

학위논문(박사) -- 서울대학교대학원 : 공과대학 기계항공공학부, 2021.8. 최예림.The emergency lane change is a risk itself because it is made instantaneously in emergency such as a sudden stop of the vehicle in front in the driving lane. Therefore, the optimization of the lane change trajectory is an essential research area of autonomous vehicle. This research proposes a path optimization for emergency lane change of autonomous vehicles based on deep reinforcement learning. This algorithm is developed with a focus on fast and safe avoidance behavior and lane change in an emergency. As the first step of algorithm development, a simulation environment was established. IPG CARMAKER was selected for reliable vehicle dynamics simulation and construction of driving scenarios for reinforcement learning. This program is a highly reliable and can analyze the behavior of a vehicle similar to that of a real vehicle. In this research, a simulation was performed using the Hyundai I30-PDe full car model. And as a simulator for DRL and vehicle control, Matlab Simulink which can encompass all of control, measurement, and artificial intelligence was selected. By connecting two simulators, the emergency lane change trajectory is optimized based on DRL. The vehicle lane change trajectory is modeled as a 3rd order polynomial. The start and end point of the lane change is set and analyzed as a function of the lane change distance for the coefficient of the polynomial. In order to optimize the coefficients. A DRL architecture is constructed. 12 types of driving environment data are used for the observation space. And lane change distance which is a variable of polynomial is selected as the output of action space. Reward space is designed to maximize the learning ability. Dynamic & static reward and penalty are given at each time step of simulation, so that optimization proceeds in a direction in which the accumulated rewards could be maximized. Deep Deterministic Policy Gradient agent is used as an algorithm for optimization. An algorithm is developed for driving a vehicle in a dynamic simulation program. First, an algorithm is developed that can determine when, at what velocity, and in which direction to change the lane of a vehicle in an emergency situation. By estimating the maximum tire-road friction coefficient in real-time, the minimum distance for the driving vehicle to stop is calculated to determine the risk of longitudinal collision with the vehicle in front. Also, using Gipps’ safety distance formula, an algorithm is developed that detects the possibility of a collision with a vehicle coming from the lane to be changed, and determines whether to overtake the vehicle to pass forward or to go backward after as being overtaken. Based on this, the decision-making algorithm for the final lane change is developed by determine the collision risk and safety of the left and right lanes. With the developed algorithm that outputs the emergency lane change trajectory through the configured reinforcement learning structure and the general driving trajectory such as the lane keeping algorithm and the adaptive cruise control algorithm according to the situation, an integrated algorithm that drives the ego vehicle through the adaptive model predictive controller is developed. As the last step of the research, DRL was performed to optimize the developed emergency lane change path optimization algorithm. 60,000 trial-and-error learning is performed to develop the algorithm for each driving situation, and performance is evaluated through test driving.긴급 차선 변경은 주행 차선에서 선행차량 급정거와 같은 응급상황 발생시에 순간적으로 이루어지는 것이므로 그 자체에 위험성을 안고 있다. 지나치게 느리게 조향을 하는 경우, 주행 차량은 앞에 있는 장애물과의 충돌을 피할 수 없다. 이와 반대로 지나치게 빠르게 조향을 하는 경우, 차량과 지면 사이의 작용력은 타이어 마찰 한계를 넘게 된다. 이는 차량의 조종 안정성을 떨어트려 스핀이나 전복 등 다른 양상의 사고를 야기한다. 따라서 차선 변경 경로의 최적화는 자율 주행 차량의 응급 상황 대처에 필수적인 요소이다. 본 논문에서는 심층강화학습을 기반으로 자율 주행 차량의 긴급 차선 변경 경로를 최적화한다. 이 알고리즘은 선행차량의 급정거나 장애물 출현과 같은 응급상황 발생 시, 빠르고 안전한 회피 거동 및 차선 변경에 초점을 맞추어 개발되었다. 알고리즘 개발의 첫 번째 단계로서 시뮬레이션 환경이 구축되었다. 신뢰성 있는 차량 동역학 시뮬레이션과 강화학습을 위한 주행 시나리오 구축을 위하여 IPG CARMAKER가 선정되었다. 이 프로그램은 실제 산업 현장에서 사용되는 높은 신뢰성을 가진 프로그램으로 실제 차량과 유사한 차량의 거동을 분석할 수 있다. 본 연구에서는 현대자동차의 I30-PDe 모델을 사용하여 시뮬레이션을 수행하였다. 또한 강화학습과 차량제어를 위한 프로그램으로 제어, 계측, 인공지능을 모두 아우를 수 있는 Matlab Simulink를 선정하였다. 본 연구에서는 IPG CARMAKER와 Matlab Simulink를 연동하여 심층 강화 학습을 바탕으로 긴급 차선 변경 궤적을 최적화하였다. 차량의 차선 변경 궤적은 3차 다항식의 형상으로 모델링 되었다. 차선 변경 시작 지점과 종료 지점을 설정하여 다항식의 계수를 차선 변경 거리에 대한 함수로 해석하였다. 심층 강화 학습을 기반으로 계수들을 최적화하기 위하여, 강화 학습 아키텍처를 구성하였다. 관측 공간은 12가지의 주행 환경 데이터를 이용하였고, 강화 학습의 출력으로는 3차 함수의 변수인 차선 변경 거리를 선정하였다. 그리고 강화 학습의 학습 능력을 극대화할 수 있는 보상 공간을 설계하였다. 동적 보상, 정적 보상, 동적 벌칙, 정적 벌칙을 시뮬레이션의 매 단계마다 부여함으로써 보상 총 합이 최대화될 수 있는 방향으로 학습이 진행되었다. 최적화를 위한 알고리즘으로는 Deep Deterministic Policy Gradient agent가 사용되었다. 강화학습 아키텍처와 함께 동역학 시뮬레이션 프로그램에서의 차량 구동을 위한 알고리즘을 개발하였다. 먼저 응급상황시에 차량의 차선을 언제, 어떤 속도로, 어떤 방향으로 변경할 지 결정하는 의사결정 알고리즘을 개발하였다. 타이어와 도로 사이의 최대 마찰계수를 실시간으로 추정하여 주행 차량이 정지하기 위한 최소 거리를 산출함으로써 선행 차량과의 충돌 위험을 판단하였다. 또한 Gipps의 안전거리 공식을 사용하여 변경하고자 하는 차선에서 오는 차량과의 충돌 가능성을 감지하여 그 차량을 추월해서 앞으로 지나갈지, 추월을 당해서 뒤로 갈 것인지를 결정하는 알고리즘을 개발하였다. 이를 바탕으로 좌측 차선과 우측 차선의 충돌 위험성 및 안정성을 판단하여 최종적인 차선 변경을 위한 의사결정 알고리즘을 개발하였다. 구성된 강화 학습 구조를 통한 긴급 차선 변경 궤적과 차선 유지 장치, 적응형 순항 제어와 같은 일반 주행시의 궤적을 상황에 맞추어 출력하는 알고리즘을 개발하고 적응형 모델 예측 제어기를 통해 주행 차량을 구동하는 통합 알고리즘을 개발하였다. 본 연구의 마지막 단계로서, 개발된 긴급 차선 변경 경로 생성 알고리즘의 최적화를 위하여 심층 강화 학습이 수행되었다. 총 60,000회의 시행 착오 방식의 학습을 통해 각 주행 상황 별 최적의 차선 변경 제어 알고리즘을 개발하였고, 각 주행상황 별 최적의 차선 변경 궤적을 제시하였다.Chapter 1. Introduction 1 1.1. Research Background 1 1.2. Previous Research 5 1.3. Research Objective 9 1.4. Dissertation Overview 13 Chapter 2. Simulation Environment 19 2.1. Simulator 19 2.2. Scenario 26 Chapter 3. Methodology 28 3.1. Reinforcement learning 28 3.2. Deep reinforcement learning 30 3.3. Neural network 33 Chapter 4. DRL-enhanced Lane Change 36 4.1. Necessity of Evasive Steering Trajectory Optimization 36 4.2. Trajectory Planning 39 4.3. DRL Structure 42 4.3.1. Observation 43 4.3.2. Action 47 4.3.3. Reward 49 4.3.4. Neural Network Architecture 58 4.3.5. Deep Deterministic Policy Gradient (DDPG) Agent 60 Chapter 5. Autonomous Driving Algorithm Integration 64 5.1. Lane Change Decision Making 65 5.1.1. Longitudinal Collision Detection 66 5.1.2. Lateral Collision Detection 71 5.1.3. Lane Change Direction Decision 74 5.2. Path Planning 75 5.3. Vehicle Controller 76 5.4. Algorithm Integration 77 Chapter 6. Training & Results 79 Chapter 7. Conclusion 91 References 97 국문초록 104박

Optimal control of brakes and steering for autonomous collision avoidance using modified Hamiltonian algorithm

This paper considers the problem of collision avoidance for road vehicles, operating at the limits of friction. A two-level modelling and control methodology is proposed, with the upper level using a friction-limited particle model for motion planning, and the lower level using a nonlinear 3DOF model for optimal control allocation. Motion planning adopts a two-phase approach: the first phase is to avoid the obstacle, the second is to recover lane keeping with minimal additional lateral deviation. This methodology differs from the more standard approach of path-planning/path-following, as there is no explicit path reference used; the control reference is a target acceleration vector which simultaneously induces changes in direction and speed. The lower level control distributes vehicle targets to the brake and steer actuators via a new and efficient method, the Modified Hamiltonian Algorithm (MHA). MHA balances CG acceleration targets with yaw moment tracking to preserve lateral stability. A nonlinear 7DOF two-track vehicle model confirms the overall validity of this novel methodology for collision avoidance

Artificial potential functions for highway driving with collision avoidance

We present a set of potential function components to assist an automated or semi-automated vehicle in navigating a multi-lane, populated highway. The resulting potential field is constructed as a superposition of disparate functions for lane- keeping, road-staying, speed preference, and vehicle avoidance and passing. The construction of the vehicle avoidance potential is of primary importance, incorporating the structure and protocol of laned highway driving. Particularly, the shape and dimensions of the potential field behind each obstacle vehicle can appropriately encourage control vehicle slowing and/or passing, depending on the cars' velocities and surrounding traffic. Hard barriers on roadway edges and soft boundaries between navigable lanes keep the vehicle on the highway, with a preference to travel in a lane center

Efficient Automated Driving Strategies Leveraging Anticipation and Optimal Control

Automated vehicles and advanced driver assistance systems bring computation, sensing, and communication technologies that exceed human abilities in some ways. For example, automated vehicles may sense a panorama all at once, do not suffer from human impairments and distractions, and could wirelessly communicate precise data with neighboring vehicles. Prototype and commercial deployments have demonstrated the capability to relieve human operators of some driving tasks up to and including fully autonomous taxi rides in some areas. The ultimate impact of this technology’s large-scale market penetration on energy efficiency remains unclear, with potential negative factors like road use by empty vehicles competing with positive ones like automatic eco-driving. Fundamentally enabled by historic and look-ahead data, this dissertation addresses the use of automated driving and driver assistance to optimize vehicle motion for energy efficiency. Facets of this problem include car following, co-optimized acceleration and lane change planning, and collaborative multi-agent guidance. Optimal control, especially model predictive control, is used extensively to improve energy efficiency while maintaining safe and timely driving via constraints. Techniques including chance constraints and mixed integer programming help overcome uncertainty and non-convexity challenges. Extensions of these techniques to tractor trailers on sloping roads are provided by making use of linear parameter-varying models. To approach the wheel-input energy eco-driving problem over generally shaped sloping roads with the computational potential for closed-loop implementation, a linear programming formulation is constructed. Distributed and collaborative techniques that enable connected and automated vehicles to accommodate their neighbors in traffic are also explored and compared to centralized control. Using simulations and vehicle-in-the-loop car following experiments, the proposed algorithms are benchmarked against others that do not make use of look-ahead information

Hierarchical Distributed MPC for Longitudinal and Lateral Vehicle Platoon Control with Collision Avoidance

This paper proposes a hierarchical distributed model predictive control (MPC) method for vehicle platoon control in both longitudinal and lateral directions. In the upper layer, a novel path-planning module and a trajectory-fusion module are utilized to compute a smooth reference trajectory for each follower. In the lower layer, the longitudinal and lateral distributed model predictive controllers are decoupled to control the velocity and steering respectively. To ensure safety and reduce the computation burden, the constraints to avoid collision are reformulated by using the strong duality theory. A simulation is conducted to demonstrate the effectiveness of the proposed control algorithm in maintaining platoon formation and ensuring the safety of the platoon

Hierarchical Off-Road Path Planning and Its Validation Using a Scaled Autonomous Car\u27

In the last few years. while a lot of research effort has been spent on autonomous vehicle navigation, primarily focused on on-road vehicles, off-road path planning still presents new challenges. Path planning for an autonomous ground vehicle over a large horizon in an unstructured environment when high-resolution a-priori information is available, is still very much an open problem due to the computations involved. Localization and control of an autonomous vehicle and how the control algorithms interact with the path planner is a complex task. The first part of this research details the development of a path decision support tool for off-road application implementing a novel hierarchical path planning framework and verification in a simulation environment. To mimic real world issues, like communication delay, sensor noise, modeling error, etc., it was important that we validate the framework in a real environment. In the second part of the research, development of a scaled autonomous car as part of a real experimental environment is discussed which provides a compromise between cost as well as implementation complexities compared to a full-scale car. The third part of the research, explains the development of a vehicle-in-loop (VIL) environment with demo examples to illustrate the utility of such a platform. Our proposed path planning algorithm mitigates the challenge of high computational cost to find the optimal path over a large scale high-resolution map. A global path planner runs in a centralized server and uses Dynamic Programming (DP) with coarse information to create an optimal cost grid. A local path planner utilizes Model Predictive Control (MPC), running on-board, using the cost map along with high-resolution information (available via various sensors as well as V2V communication) to generate the local optimal path. Such an approach ensures the MPC follows a global optimal path while being locally optimal. A central server efficiently creates and updates route critical information available via vehicle-to-infrastructure(V2X) communication while using the same to update the prescribed global cost grid. For localization of the scaled car, a three-axis inertial measurement unit (IMU), wheel encoders, a global positioning system (GPS) unit and a mono-camera are mounted. Drift in IMU is one of the major issues which we addressed in this research besides developing a low-level controller which helped in implementing the MPC in a constrained computational environment. Using a camera and tire edge detection algorithm we have developed an online steering angle measurement package as well as a steering angle estimation algorithm to be utilized in case of low computational resources. We wanted to study the impact of connectivity on a fleet of vehicles running in off-road terrain. It is costly as well as time consuming to run all real vehicles. Also some scenarios are difficult to recreate in real but need a simulation environment. So we have developed a vehicle-in-loop (VIL) platform using a VIL simulator, a central server and the real scaled car to combine the advantages of both real and simulation environment. As a demo example to illustrate the utility of VIL platform, we have simulated an animal crossing scenario and analyze how our obstacle avoidance algorithms performs under different conditions. In the future it will help us to analyze the impact of connectivity on platoons moving in off-road terrain. For the vehicle-in-loop environment, we have used JavaScript Object Notation (JSON) data format for information exchange using User Datagram Protocol (UDP) for implementing Vehicle-to-Vehicle (V2V) and MySQL server for Vehicle-to-Infrastructure (V2I) communication