Interaction-aware Kalman Neural Networks for Trajectory Prediction

Forecasting the motion of surrounding obstacles (vehicles, bicycles, pedestrians and etc.) benefits the on-road motion planning for intelligent and autonomous vehicles. Complex scenes always yield great challenges in modeling the patterns of surrounding traffic. For example, one main challenge comes from the intractable interaction effects in a complex traffic system. In this paper, we propose a multi-layer architecture Interaction-aware Kalman Neural Networks (IaKNN) which involves an interaction layer for resolving high-dimensional traffic environmental observations as interaction-aware accelerations, a motion layer for transforming the accelerations to interaction aware trajectories, and a filter layer for estimating future trajectories with a Kalman filter network. Attributed to the multiple traffic data sources, our end-to-end trainable approach technically fuses dynamic and interaction-aware trajectories boosting the prediction performance. Experiments on the NGSIM dataset demonstrate that IaKNN outperforms the state-of-the-art methods in terms of effectiveness for traffic trajectory prediction.Comment: 8 pages, 4 figures, Accepted for IEEE Intelligent Vehicles Symposium (IV) 202

Egocentric Vision-based Future Vehicle Localization for Intelligent Driving Assistance Systems

Predicting the future location of vehicles is essential for safety-critical applications such as advanced driver assistance systems (ADAS) and autonomous driving. This paper introduces a novel approach to simultaneously predict both the location and scale of target vehicles in the first-person (egocentric) view of an ego-vehicle. We present a multi-stream recurrent neural network (RNN) encoder-decoder model that separately captures both object location and scale and pixel-level observations for future vehicle localization. We show that incorporating dense optical flow improves prediction results significantly since it captures information about motion as well as appearance change. We also find that explicitly modeling future motion of the ego-vehicle improves the prediction accuracy, which could be especially beneficial in intelligent and automated vehicles that have motion planning capability. To evaluate the performance of our approach, we present a new dataset of first-person videos collected from a variety of scenarios at road intersections, which are particularly challenging moments for prediction because vehicle trajectories are diverse and dynamic.Comment: To appear on ICRA 201

Autonomous Drifting Using Reinforcement Learning

Autonomous vehicles or self-driving cars are prevalent nowadays, many vehicle manufacturers, and other tech companies are trying to develop autonomous vehicles. One major goal of the self-driving algorithms is to perform manoeuvres safely, even when some anomaly arises. To solve these kinds of complex issues, Artificial Intelligence and Machine Learning methods are used. One of these motion planning problems is when the tires lose their grip on the road, an autonomous vehicle should handle this situation. Thus the paper provides an Autonomous Drifting algorithm using Reinforcement Learning. The algorithm is based on a model-free learning algorithm, Twin Delayed Deep Deterministic Policy Gradients (TD3). The model is trained on six different tracks in a simulator, which is developed specifically for autonomous driving systems; namely CARLA

Model Predictive Control of Highway Emergency Maneuvering and Collision Avoidance

Autonomous emergency maneuvering (AEM) is an active safety system that automates safe maneuvers to avoid imminent collision, particularly in highway driving situations. Uncertainty about the surrounding vehicles’ decisions and also about the road condition, which has significant effects on the vehicle’s maneuverability, makes it challenging to implement the AEM strategy in practice. With the rise of vehicular networks and connected vehicles, vehicles would be able to share their perception and also intentions with other cars. Therefore, cooperative AEM can incor- porate surrounding vehicles’ decisions and perceptions in order to improve vehicles’ predictions and estimations and thereby provide better decisions for emergency maneuvering. In this thesis, we develop an adaptive, cooperative motion planning scheme for emergency maneuvering, based on the model predictive control (MPC) approach, for vehicles within a ve- hicular network. The proposed emergency maneuver planning scheme finds the best combination of longitudinal and lateral maneuvers to avoid imminent collision with surrounding vehicles and obstacles. To implement real-time MPC for the non-convex problem of collision free motion planning, safety constraints are suggested to be convexified based on the road geometry. To take advantage of vehicular communication, the surrounding vehicles’ decisions are incorporated in the prediction model to improve the motion planning results. The MPC approach is prone to loss of feasibility due to the limited prediction horizon for decision-making. For the autonomous vehicle motion planning problem, many of detected ob- stacles, which are beyond the prediction horizon, cannot be considered in the instantaneous de- cisions, and late consideration of them may cause infeasibility. The conditions that guarantee persistent feasibility of a model predictive motion planning scheme are studied in this thesis. Maintaining the system’s states in a control invariant set of the system guarantees the persis- tent feasibility of the corresponding MPC scheme. Specifically, we present two approaches to compute control invariant sets of the motion planning problem; the linearized convexified ap- proach and the brute-force approach. The resulting computed control invariant sets of these two approaches are compared with each other to demonstrate the performance of the proposed algorithm. Time-variation of the road condition affects the vehicle dynamics and constraints. Therefore, it necessitates the on-line identification of the road friction parameter and implementation of an adaptive emergency maneuver motion planning scheme. In this thesis, we investigate coopera- tive road condition estimation in order to improve collision avoidance performance of the AEM system. Each vehicle estimates the road condition individually, and disseminates it through the vehicular network. Accordingly, a consensus estimation algorithm fuses the individual estimates to find the maximum likelihood estimate of the road condition parameter. The performance of the proposed cooperative road condition estimation has been validated through simulations

Contextual information aided target tracking and path planning for autonomous ground vehicles

Recently, autonomous vehicles have received worldwide attentions from academic research, automotive industry and the general public. In order to achieve a higher level of automation, one of the most fundamental requirements of autonomous vehicles is the capability to respond to internal and external changes in a safe, timely and appropriate manner. Situational awareness and decision making are two crucial enabling technologies for safe operation of autonomous vehicles. This thesis presents a solution for improving the automation level of autonomous vehicles in both situational awareness and decision making aspects by utilising additional domain knowledge such as constraints and influence on a moving object caused by environment and interaction between different moving objects. This includes two specific sub-systems, model based target tracking in environmental perception module and motion planning in path planning module. In the first part, a rigorous Bayesian framework is developed for pooling road constraint information and sensor measurement data of a ground vehicle to provide better situational awareness. Consequently, a new multiple targets tracking (MTT) strategy is proposed for solving target tracking problems with nonlinear dynamic systems and additional state constraints. Besides road constraint information, a vehicle movement is generally affected by its surrounding environment known as interaction information. A novel dynamic modelling approach is then proposed by considering the interaction information as virtual force which is constructed by involving the target state, desired dynamics and interaction information. The proposed modelling approach is then accommodated in the proposed MTT strategy for incorporating different types of domain knowledge in a comprehensive manner. In the second part, a new path planning strategy for autonomous vehicles operating in partially known dynamic environment is suggested. The proposed MTT technique is utilized to provide accurate on-board tracking information with associated level of uncertainty. Based on the tracking information, a path planning strategy is developed to generate collision free paths by not only predicting the future states of the moving objects but also taking into account the propagation of the associated estimation uncertainty within a given horizon. To cope with a dynamic and uncertain road environment, the strategy is implemented in a receding horizon fashion

Traffic Management System for the combined optimal routing, scheduling and motion planning of self-driving vehicles inside reserved smart road networks

The topic discussed in this thesis belongs to the field of automation of transport systems, which has grown in importance in the last decade, both in the innovation field (where different automation technologies have been gradually introduced in different sectors of road transport, in the promising view of making it more efficient, safer, and greener) and in the research field (where different research activities and publications have addressed the problem under different points of view). More in detail, this work addresses the problem of autonomous vehicles coordina tion inside reserved road networks by proposing a novel Traffic Management System (TMS) for the combined routing, scheduling and motion planning of the vehicles. To this aim, the network is assumed to have a modular structure, which results from a certain number of roads and intersections assembled together. The way in which roads and intersections are put together defines the network layout. Within such a system architecture, the main tasks addressed by the TMS are: (1) at the higher level, the optimal routing of the vehicles in the network, exploiting the available information coming from the vehicles and the various elements of the network; (2) at a lower level, the modeling and optimization of the vehicle trajectories and speeds for each road and for each intersection in the network; (3) the coordination between the vehicles and the elements of the network, to ensure a combined approach that considers, in a recursive way, the scheduling and motion planning of the vehicles in the various elements when solving the routing problem. In particular, the routing and the scheduling and motion planning problems are formulated as MILP optimization problems, aiming to maximize the performance of the entire network (routing model) and the performance of its single elements - roads and intersections (scheduling and motion planning model) while guaranteeing the requested level of safety and comfort for the passengers. Besides, one of the main features of the proposed approach consists of the integration of the scheduling decisions and the motion planning computation by means of constraints regarding the speed limit, the acceleration, and the so-called safety dynamic constraints on incompatible positions of conflicting vehicles. In particular, thanks to these last constraints, it is possible to consider the real space occupancy of the vehicles avoiding collisions. After the theoretical discussion of the proposed TMS and of its components and models, the thesis presents and discusses the results of different numerical experiments, aimed at testing the TMS in some specific scenarios. In particular, the routing model and the scheduling and motion planning model are tested on different scenarios, which demonstrate the effectiveness and the validity of such approach in performing the addressed tasks, also compared with more traditional methods. Finally, the computational effort needed for the problem solution, which is a key element to take into account, is discussed both for the road element and the intersection element