Model predictive path integral control: Theoretical foundations and applications to autonomous driving

This thesis presents a new approach for stochastic model predictive (optimal) control: model predictive path integral control, which is based on massive parallel sampling of control trajectories. We first show the theoretical foundations of model predictive path integral control, which are based on a combination of path integral control theory and an information theoretic interpretation of stochastic optimal control. We then apply the method to high speed autonomous driving on a 1/5 scale vehicle and analyze the performance and robustness of the method. Extensive experimental results are used to identify and solve key problems relating to robustness of the approach, which leads to a robust stochastic model predictive control algorithm capable of consistently pushing the limits of performance on the 1/5 scale vehicle.Ph.D

Bridging Active Exploration and Uncertainty-Aware Deployment Using Probabilistic Ensemble Neural Network Dynamics

In recent years, learning-based control in robotics has gained significant attention due to its capability to address complex tasks in real-world environments. With the advances in machine learning algorithms and computational capabilities, this approach is becoming increasingly important for solving challenging control problems in robotics by learning unknown or partially known robot dynamics. Active exploration, in which a robot directs itself to states that yield the highest information gain, is essential for efficient data collection and minimizing human supervision. Similarly, uncertainty-aware deployment has been a growing concern in robotic control, as uncertain actions informed by the learned model can lead to unstable motions or failure. However, active exploration and uncertainty-aware deployment have been studied independently, and there is limited literature that seamlessly integrates them. This paper presents a unified model-based reinforcement learning framework that bridges these two tasks in the robotics control domain. Our framework uses a probabilistic ensemble neural network for dynamics learning, allowing the quantification of epistemic uncertainty via Jensen-Renyi Divergence. The two opposing tasks of exploration and deployment are optimized through state-of-the-art sampling-based MPC, resulting in efficient collection of training data and successful avoidance of uncertain state-action spaces. We conduct experiments on both autonomous vehicles and wheeled robots, showing promising results for both exploration and deployment.Comment: 2023 Robotics: Science and Systems (RSS). Project page: https://taekyung.me/rss2023-bridgin

APPLYING COLLABORATIVE ONLINE ACTIVE LEARNING IN VEHICULAR NETWORKS FOR FUTURE CONNECTED AND AUTONOMOUS VEHICLES

The main objective of this thesis is to provide a framework for, and proof of concept of, collaborative online active learning in vehicular networks. Another objective is to advance the state of the art in simulation-based evaluation and validation of connected intelligent vehicle applications. With advancements in machine learning and artificial intelligence, connected autonomous vehicles (CAVs) have begun to migrate from laboratory development and testing conditions to driving on public roads. Their deployment in our environmental landscape offers potential for decreases in road accidents and traffic congestion, as well as improved mobility in overcrowded cities. Although common driving scenarios can be relatively easily solved with classic perception, path planning, and motion control methods, the remaining unsolved scenarios are corner cases in which traditional methods fail. These unsolved cases are the keys to deploying CAVs safely on the road, but they require an enormous amount of data collection and high-quality human annotation, which are very cost-ineffective considering the ever-changing real-world scenarios and highly diverse road/weather conditions. Additionally, evaluating and testing applications for CAVs in real testbeds are extremely expensive, as obvious failures like crashes tend to be rare events and can hardly be captured through predefined test scenarios. Therefore, realistic simulation tools with the benefit of lower cost as well as generating reproducible experiment results are needed to complement the real testbeds in validating applications for CAVs. Therefore, in this thesis, we address the challenges therein and establish the fundamentals of the collaborative online active learning framework in vehicular network for future connected and autonomous vehicles.Ph.D