Active Learning of Discrete-Time Dynamics for Uncertainty-Aware Model Predictive Control

Model-based control requires an accurate model of the system dynamics for precisely and safely controlling the robot in complex and dynamic environments. Moreover, in the presence of variations in the operating conditions, the model should be continuously refined to compensate for dynamics changes. In this paper, we present a self-supervised learning approach that actively models the dynamics of nonlinear robotic systems. We combine offline learning from past experience and online learning from current robot interaction with the unknown environment. These two ingredients enable a highly sample-efficient and adaptive learning process, capable of accurately inferring model dynamics in real-time even in operating regimes that greatly differ from the training distribution. Moreover, we design an uncertainty-aware model predictive controller that is heuristically conditioned to the aleatoric (data) uncertainty of the learned dynamics. This controller actively chooses the optimal control actions that (i) optimize the control performance and (ii) improve the efficiency of online learning sample collection. We demonstrate the effectiveness of our method through a series of challenging real-world experiments using a quadrotor system. Our approach showcases high resilience and generalization capabilities by consistently adapting to unseen flight conditions, while it significantly outperforms classical and adaptive control baselines

Efficient Deep Learning of Robust Policies from MPC using Imitation and Tube-Guided Data Augmentation

Imitation Learning (IL) has been increasingly employed to generate computationally efficient policies from task-relevant demonstrations provided by Model Predictive Control (MPC). However, commonly employed IL methods are often data- and computationally-inefficient, as they require a large number of MPC demonstrations, resulting in long training times, and they produce policies with limited robustness to disturbances not experienced during training. In this work, we propose an IL strategy to efficiently compress a computationally expensive MPC into a Deep Neural Network (DNN) policy that is robust to previously unseen disturbances. By using a robust variant of the MPC, called Robust Tube MPC (RTMPC), and leveraging properties from the controller, we introduce a computationally-efficient Data Aggregation (DA) method that enables a significant reduction of the number of MPC demonstrations and training time required to generate a robust policy. Our approach opens the possibility of zero-shot transfer of a policy trained from a single MPC demonstration collected in a nominal domain, such as a simulation or a robot in a lab/controlled environment, to a new domain with previously-unseen bounded model errors/perturbations. Numerical and experimental evaluations performed using linear and nonlinear MPC for agile flight on a multirotor show that our method outperforms strategies commonly employed in IL (such as DAgger and DR) in terms of demonstration-efficiency, training time, and robustness to perturbations unseen during training.Comment: Under review. arXiv admin note: text overlap with arXiv:2109.0991

Data-Driven MPC for Quadrotors

Aerodynamic forces render accurate high-speed trajectory tracking with quadrotors extremely challenging. These complex aerodynamic effects become a significant disturbance at high speeds, introducing large positional tracking errors, and are extremely difficult to model. To fly at high speeds, feedback control must be able to account for these aerodynamic effects in real-time. This necessitates a modelling procedure that is both accurate and efficient to evaluate. Therefore, we present an approach to model aerodynamic effects using Gaussian Processes, which we incorporate into a Model Predictive Controller to achieve efficient and precise real-time feedback control, leading to up to 70% reduction in trajectory tracking error at high speeds. We verify our method by extensive comparison to a state-of-the-art linear drag model in synthetic and real-world experiments at speeds of up to 14m/s and accelerations beyond 4g.Comment: 8 page

Real-Time Collision Imminent Steering Using One-Level Nonlinear Model Predictive Control

Automotive active safety features are designed to complement or intervene a human driver's actions in safety critical situations. Existing active safety features, such as adaptive cruise control and lane keep assist, are able to exploit the ever growing sensor and computing capabilities of modern automobiles. An emerging feature, collision imminent steering, is designed to perform an evasive lane change to avoid collision if the vehicle believes collision cannot be avoided by braking alone. This is a challenging maneuver, as the expected highway setting is characterized by high speeds, narrow lane restrictions, and hard safety constraints. To perform such a maneuver, the vehicle may be required to operate at the nonlinear dynamics limits, necessitating advanced control strategies to enforce safety and drivability constraints. This dissertation presents a one-level nonlinear model predictive controller formulation to perform a collision imminent steering maneuver in a highway setting at high speeds, with direct consideration of safety criteria in the highway environment and the nonlinearities characteristic of such a potentially aggressive maneuver. The controller is cognizant of highway sizing constraints, vehicle handling capability and stability limits, and time latency when calculating the control action. In simulated testing, it is shown the controller can avoid collision by conducting a lane change in roughly half the distance required to avoid collision by braking alone. In preliminary vehicle testing, it is shown the control formulation is compatible with the existing perception pipeline, and prescribed control action can safely perform a lane change at low speed. Further, the controller must be suitable for real-time implementation and compatible with expected automotive control architecture. Collision imminent steering, and more broadly collision avoidance, control is a computationally challenging problem. At highway speeds, the required time for action is on the order of hundreds of milliseconds, requiring a control formulation capable of operating at tens of Hertz. To this extent, this dissertation investigates the computational expense of such a controller, and presents a framework for designing real-time compatible nonlinear model predictive controllers. Specifically, methods for numerically simulating the predicted vehicle response and response sensitivities are compared, their cross interaction with trajectory optimization strategy are considered, and the resulting mapping to a parallel computing hardware architecture is investigated. The framework systematically evaluates the underlying numerical optimization problem for bottlenecks, from which it provides alternative solutions strategies to achieve real-time performance. As applied to the baseline collision imminent steering controller, the procedure results in an approximate three order of magnitude reduction in compute wall time, supporting real-time performance and enabling preliminary testing on automotive grade hardware.PHDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/163063/1/jbwurts_1.pd

Online monitoring and control of voltage stability margin via machine learning-based adaptive approaches

Voltage instability or voltage collapse, observed in many blackout events, poses a significant threat to power system reliability. To prevent voltage collapse, the countermeasures suggested by the post analyses of the blackouts usually include the adoption of better online voltage stability monitoring and control tools. Recently, the variability and uncertainty imposed by the increasing penetration of renewable energy further magnifies this need. This work investigates the methodologies for online voltage stability margin (VSM) monitoring and control in the new era of smart grid and big data. It unleashes the value of online measurements and leverages the fruitful results in machine learning and demand response. An online VSM monitoring approach based on local regression and adaptive database is proposed. Considering the increasing variability and uncertainty of power system operation, this approach utilizes the locality of underlying pattern between VSM and reactive power reserve (RPR), and can adapt to the changing condition of system. LASSO (Least Absolute Shrinkage and Selection Operator) is tailored to solve the local regression problem so as to mitigate the curse of dimensionality for large-scale system. Along with the VSM prediction, its prediction interval is also estimated simultaneously in a simple but effective way, and utilized as an evidence to trigger the database updating. IEEE 30-bus system and a 60,000-bus large system are used to test and demonstrate the proposed approach. The results show that the proposed approach can be successfully employed in online voltage stability monitoring for real size systems, and the adaptivity of model and data endows the proposed approach with the advantage in the circumstances where large and unforeseen changes of system condition are inevitable. In case degenerative system conditions are identified, a control strategy is needed to steer the system back to security. A model predictive control (MPC) based framework is proposed to maintain VSM in near-real-time while minimizing the control cost. VSM is locally modeled as a linear function of RPRs based on the VSM monitoring tool, which convexifies the intricate VSM-constrained optimization problem. Thermostatically controlled loads (TCLs) are utilized through a demand response (DR) aggregator as the efficient measure to enhance voltage stability. For such an advanced application of the energy management system (EMS), plug-and-play is a necessary feature that makes the new controller really applicable in a cooperative operating environment. In this work, the cooperation is realized by a predictive interface strategy, which predicts the behaviors of relevant controllers using the simple models declared and updated by those controllers. In particular, the customer dissatisfaction, defined as the cumulative discomfort caused by DR, is explicitly constrained in respect of customers\u27 interests. This constraint maintains the applicability of the control. IEEE 30-bus system is used to demonstrate the proposed control strategy. Adaptivity and proactivity lie at the heart of the proposed approach. By making full use of real-time information, the proposed approach is competent at the task of VSM monitoring and control in a non-stationary and uncertain operating environment

Holistic Vehicle Control Using Learning MPC

In recent years, learning MPC schemes have been introduced to address these challenges of traditional MPC. They typically leverage different machine learning techniques to learn the system dynamics directly from data, allowing it to handle model uncertainty more effectively. Besides, they can adapt to changes by continuously updating the learned model using real-time data, ensuring that the controller remains effective even as the system evolves. However, there are some challenges for the existing learning MPC techniques. Firstly, learning-based control approaches often lack interpretability. Understanding and interpreting the learned models and their learning and prediction processes are crucial for safety critical systems such as vehicle stability systems. Secondly, existing learning MPC techniques rely solely on learned models, which might result in poor performance or instability if the model encounters scenarios that differ significantly from the training data. Thirdly, existing learning MPC techniques typically require large amounts of high-quality data for training accurate models, which can be expensive or impractical in the vehicle stability control domain. To address these challenges, this thesis proposes a novel hybrid learning MPC approach for HVC. The main objective is to leverage the capabilities of machine learning algorithms to learn accurate and adaptive models of vehicle dynamics from data, enabling enhanced control strategies for improved stability and maneuverability. The hybrid learning MPC scheme maintains a traditional physics-based vehicle model and a data-based learning model. In the learned model, a variety of machine-learning techniques can be used to predict vehicle dynamics based on learning from collected vehicle data. The performance of the developed hybrid learning MPC controller using torque vectoring (TV) as the actuator is evaluated through the Matlab/Simulink and CarSim co-simulation with a high-fidelity Chevy Equinox vehicle model under a series of harsh maneuvers. Extensive real-world experiments using a Chevy Equinox electric testing vehicle are conducted. Both simulation results and experimental results show that the developed hybrid learning MPC approach consistently outperforms existing MPC methods with better yaw rate tracking performance and smaller vehicle sideslip under various driving conditions

Safe Robot Planning and Control Using Uncertainty-Aware Deep Learning

In order for robots to autonomously operate in novel environments over extended periods of time, they must learn and adapt to changes in the dynamics of their motion and the environment. Neural networks have been shown to be a versatile and powerful tool for learning dynamics and semantic information. However, there is reluctance to deploy these methods on safety-critical or high-risk applications, since neural networks tend to be black-box function approximators. Therefore, there is a need for investigation into how these machine learning methods can be safely leveraged for learning-based controls, planning, and traversability. The aim of this thesis is to explore methods for both establishing safety guarantees as well as accurately quantifying risks when using deep neural networks for robot planning, especially in high-risk environments. First, we consider uncertainty-aware Bayesian Neural Networks for adaptive control, and introduce a method for guaranteeing safety under certain assumptions. Second, we investigate deep quantile regression learning methods for learning time-and-state varying uncertainties, which we use to perform trajectory optimization with Model Predictive Control. Third, we introduce a complete framework for risk-aware traversability and planning, which we use to enable safe exploration of extreme environments. Fourth, we again leverage deep quantile regression and establish a method for accurately learning the distribution of traversability risks in these environments, which can be used to create safety constraints for planning and control.Ph.D