Contraction Theory for Robust Learning-Based Control: Toward Aerospace and Robotic Autonomy

Machine learning and AI have been used for achieving autonomy in various aerospace and robotic systems. In next-generation research tasks, which could involve highly nonlinear, complicated, and large-scale decision-making problems in safety-critical situations, however, the existing performance guarantees of black-box AI approaches may not be sufficiently powerful. This thesis gives a mathematical overview of contraction theory, with some practical examples drawn from joint projects with NASA JPL, for enjoying formal guarantees of nonlinear control theory even with the use of machine learning-based and data-driven methods. This is not to argue that these methods are always better than conventional approaches, but to provide formal tools to investigate their performance for further discussion, so we can design and operate truly autonomous aerospace and robotic systems safely, robustly, adaptively, and intelligently in real-time. Contraction theory is an analytical tool to study differential dynamics of a non-autonomous (i.e., time-varying) nonlinear system under a contraction metric defined with a uniformly positive definite matrix, the existence of which results in a necessary and sufficient characterization of incremental exponential stability of multiple solution trajectories with respect to each other. Its nonlinear stability analysis boils down to finding a suitable contraction metric that satisfies a stability condition expressed as a linear matrix inequality, resulting in many parallels drawn between linear systems theory and contraction theory for nonlinear systems. This yields much-needed safety and stability guarantees for neural network-based control and estimation schemes, without resorting to a more involved method of using uniform asymptotic stability for input-to-state stability. Such distinctive features permit the systematic construction of a contraction metric via convex optimization, thereby obtaining an explicit exponential bound on the distance between a time-varying target trajectory and solution trajectories perturbed externally due to disturbances and learning errors. The first two parts of this thesis are about a theoretical overview of contraction theory and its advantages, with an emphasis on deriving formal robustness and stability guarantees for deep learning-based 1) feedback control, 2) state estimation, 3) motion planning, 4) multi-agent collision avoidance and robust tracking augmentation, 5) adaptive control, 6) neural net-based system identification and control, for nonlinear systems perturbed externally by deterministic and stochastic disturbances. In particular, we provide a detailed review of techniques for finding contraction metrics and associated control and estimation laws using deep neural networks. In the third part of the thesis, we present several numerical simulations and empirical validation of our proposed approaches to assess the impact of our findings on realizing aerospace and robotic autonomy. We mainly focus on the two joint projects with NASA JPL: 1) Science-Infused Spacecraft Autonomy for Interstellar Object Exploration and 2) Constellation Autonomous Space Technology Demonstration of Orbital Reconfiguration (CASTOR), where we also perform hardware demonstrations of our methods using our thruster-based spacecraft simulators (M-STAR) and in high-conflict, distributed, intelligent UAV swarm reconfiguration with up to 20 UAVs (crazyflies).</p

Interactive Planning and Sensing for Aircraft in Uncertain Environments with Spatiotemporally Evolving Threats

Autonomous aerial, terrestrial, and marine vehicles provide a platform for several applications including cargo transport, information gathering, surveillance, reconnaissance, and search-and-rescue. To enable such applications, two main technical problems are commonly addressed.On the one hand, the motion-planning problem addresses optimal motion to a destination: an application example is the delivery of a package in the shortest time with least fuel. Solutions to this problem often assume that all relevant information about the environment is available, possibly with some uncertainty. On the other hand, the information gathering problem addresses the maximization of some metric of information about the environment: application examples include such as surveillance and environmental monitoring. Solutions to the motion-planning problem in vehicular autonomy assume that information about the environment is available from three sources: (1) the vehicle’s own onboard sensors, (2) stationary sensor installations (e.g. ground radar stations), and (3) other information gathering vehicles, i.e., mobile sensors, especially with the recent emphasis on collaborative teams of autonomous vehicles with heterogeneous capabilities. Each source typically processes the raw sensor data via estimation algorithms. These estimates are then available to a decision making system such as a motion- planning algorithm. The motion-planner may use some or all of the estimates provided. There is an underlying assumption of “separation� between the motion-planning algorithm and the information about environment. This separation is common in linear feedback control systems, where estimation algorithms are designed independent of control laws, and control laws are designed with the assumption that the estimated state is the true state. In the case of motion-planning, there is no reason to believe that such a separation between the motion-planning algorithm and the sources of estimated environment information will lead to optimal motion plans, even if the motion planner and the estimators are themselves optimal. The goal of this dissertation is to investigate whether the removal of this separation, via interactive motion-planning and sensing, can significantly improve the optimality of motion- planning. The major contribution of this work is interactive planning and sensing. We consider the problem of planning the path of a vehicle, which we refer to as the actor, to traverse a threat field with minimum threat exposure. The threat field is an unknown, time- variant, and strictly positive scalar field defined on a compact 2D spatial domain – the actor’s workspace. The threat field is estimated by a network of mobile sensors that can measure the threat field pointwise. All measurements are noisy. The objective is to determine a path for the actor to reach a desired goal with minimum risk, which is a measure sensitive not only to the threat exposure itself, but also to the uncertainty therein. A novelty of this problem setup is that the actor can communicate with the sensor network and request that the sensors position themselves in a procedure we call sensor reconfiguration such that the actor’s risk is minimized. This work continues with a foundation in motion planning in time-varying fields where waiting is a control input. Waiting is examined in the context of finding an optimal path with considerations for the cost of exposure to a threat field, the cost of movement, and the cost of waiting. For example, an application where waiting may be beneficial in motion-planning is the delivery of a package where adverse weather may pose a risk to the safety of a UAV and its cargo. In such scenarios, an optimal plan may include “waiting until the storm passes.� Results on computational efficiency and optimality of considering waiting in path- planning algorithms are presented. In addition, the relationship of waiting in a time- varying field represented with varying levels of resolution, or multiresolution is studied. Interactive planning and sensing is further developed for the case of time-varying environments. This proposed extension allows for the evaluation of different mission windows, finite sensor network reconfiguration durations, finite planning durations, and varying number of available sensors. Finally, the proposed method considers the effect of waiting in the path planner under the interactive planning and sensing for time-varying fields framework. Future work considers various extensions of the proposed interactive planning and sensing framework including: generalizing the environment using Gaussian processes, sensor reconfiguration costs, multiresolution implementations, nonlinear parameters, decentralized sensor networks and an application to aerial payload delivery by parafoil

Optimal control problems solved via swarm intelligence

Questa tesi descrive come risolvere problemi di controllo ottimo tramite swarm in telligence. Grande enfasi viene posta circa la formulazione del problema di controllo ottimo, in particolare riguardo a punti fondamentali come l’identificazione delle incognite, la trascrizione numerica e la scelta del risolutore per la programmazione non lineare. L’algoritmo Particle Swarm Optimization viene preso in considerazione e la maggior parte dei problemi proposti sono risolti utilizzando una formulazione differential flatness. Quando viene usato l’approccio di dinamica inversa, il problema di ottimo relativo ai parametri di trascrizione è risolto assumendo che le traiettorie da identificare siano approssimate con curve B-splines. La tecnica Inverse-dynamics Particle Swarm Optimization, che viene impiegata nella maggior parte delle applicazioni numeriche di questa tesi, è una combinazione del Particle Swarm e della formulazione differential flatness. La tesi investiga anche altre opportunità di risolvere problemi di controllo ottimo tramite swarm intelligence, per esempio usando un approccio di dinamica diretta e imponendo a priori le condizioni necessarie di ottimalitá alla legge di controllo. Per tutti i problemi proposti, i risultati sono analizzati e confrontati con altri lavori in letteratura. Questa tesi mostra quindi the algoritmi metaeuristici possono essere usati per risolvere problemi di controllo ottimo, ma soluzioni ottime o quasi-ottime possono essere ottenute al variare della formulazione del problema.This thesis deals with solving optimal control problems via swarm intelligence. Great emphasis is given to the formulation of the optimal control problem regarding fundamental issues such as unknowns identification, numerical transcription and choice of the nonlinear programming solver. The Particle Swarm Optimization is taken into account, and most of the proposed problems are solved using a differential flatness formulation. When the inverse-dynamics approach is used, the transcribed parameter optimization problem is solved assuming that the unknown trajectories are approximated with B-spline curves. The Inverse-dynamics Particle Swarm Optimization technique, which is employed in the majority of the numerical applications in this work, is a combination of Particle Swarm and differential flatness formulation. This thesis also investigates other opportunities to solve optimal control problems with swarm intelligence, for instance using a direct dynamics approach and imposing a-priori the necessary optimality conditions to the control policy. For all the proposed problems, results are analyzed and compared with other works in the literature. This thesis shows that metaheuristic algorithms can be used to solve optimal control problems, but near-optimal or optimal solutions can be attained depending on the problem formulation

A Model for Optimal Human Navigation with Stochastic Effects

We present a method for optimal path planning of human walking paths in mountainous terrain, using a control theoretic formulation and a Hamilton-Jacobi-Bellman equation. Previous models for human navigation were entirely deterministic, assuming perfect knowledge of the ambient elevation data and human walking velocity as a function of local slope of the terrain. Our model includes a stochastic component which can account for uncertainty in the problem, and thus includes a Hamilton-Jacobi-Bellman equation with viscosity. We discuss the model in the presence and absence of stochastic effects, and suggest numerical methods for simulating the model. We discuss two different notions of an optimal path when there is uncertainty in the problem. Finally, we compare the optimal paths suggested by the model at different levels of uncertainty, and observe that as the size of the uncertainty tends to zero (and thus the viscosity in the equation tends to zero), the optimal path tends toward the deterministic optimal path

Multi-objective Robust Strategy Synthesis for Interval Markov Decision Processes

Interval Markov decision processes (IMDPs) generalise classical MDPs by having interval-valued transition probabilities. They provide a powerful modelling tool for probabilistic systems with an additional variation or uncertainty that prevents the knowledge of the exact transition probabilities. In this paper, we consider the problem of multi-objective robust strategy synthesis for interval MDPs, where the aim is to find a robust strategy that guarantees the satisfaction of multiple properties at the same time in face of the transition probability uncertainty. We first show that this problem is PSPACE-hard. Then, we provide a value iteration-based decision algorithm to approximate the Pareto set of achievable points. We finally demonstrate the practical effectiveness of our proposed approaches by applying them on several case studies using a prototypical tool.Comment: This article is a full version of a paper accepted to the Conference on Quantitative Evaluation of SysTems (QEST) 201

A risk-aware architecture for resilient spacecraft operations

In this paper we discuss a resilient, risk-aware software architecture for onboard, real-time autonomous operations that is intended to robustly handle uncertainty in space-craft behavior within hazardous and unconstrained environments, without unnecessarily increasing complexity. This architecture, the Resilient Spacecraft Executive (RSE), serves three main functions: (1) adapting to component failures to allow graceful degradation, (2) accommodating environments, science observations, and spacecraft capabilities that are not fully known in advance, and (3) making risk-aware decisions without waiting for slow ground-based reactions. This RSE is made up of four main parts: deliberative, habitual, and reflexive layers, and a state estimator that interfaces with all three. We use a risk-aware goal-directed executive within the deliberative layer to perform risk-informed planning, to satisfy the mission goals (specified by mission control) within the specified priorities and constraints. Other state-of-the-art algorithms to be integrated into the RSE include correct-by-construction control synthesis and model-based estimation and diagnosis. We demonstrate the feasibility of the architecture in a simple implementation of the RSE for a simulated Mars rover scenario

Reliable Decision-Making with Imprecise Models

The rapid growth in the deployment of autonomous systems across various sectors has generated considerable interest in how these systems can operate reliably in large, stochastic, and unstructured environments. Despite recent advances in artificial intelligence and machine learning, it is challenging to assure that autonomous systems will operate reliably in the open world. One of the causes of unreliable behavior is the impreciseness of the model used for decision-making. Due to the practical challenges in data collection and precise model specification, autonomous systems often operate based on models that do not represent all the details in the environment. Even if the system has access to a comprehensive decision-making model that accounts for all the details in the environment and all possible scenarios the agent may encounter, it may be intractable to solve this complex model optimally. Consequently, this complex, high fidelity model may be simplified to accelerate planning, introducing imprecision. Reasoning with such imprecise models affects the reliability of autonomous systems. A system\u27s actions may sometimes produce unexpected, undesirable consequences, which are often identified after deployment. How can we design autonomous systems that can operate reliably in the presence of uncertainty and model imprecision? This dissertation presents solutions to address three classes of model imprecision in a Markov decision process, along with an analysis of the conditions under which bounded-performance can be guaranteed. First, an adaptive outcome selection approach is introduced to devise risk-aware reduced models of the environment that efficiently balance the trade-off between model simplicity and fidelity, to accelerate planning in resource-constrained settings. Second, a framework that extends stochastic shortest path framework to problems with imperfect information about the goal state during planning is introduced, along with two solution approaches to solve this problem. Finally, two complementary solution approaches are presented to minimize the negative side effects of agent actions. The techniques presented in this dissertation enable an autonomous system to detect and mitigate undesirable behavior, without redesigning the model entirely

A Reactive Planning Framework for Dexterous Robotic Manipulation

This thesis investigates a reactive motion planning and controller framework that enables robots to manipulate objects dexterously. We develop a robotic platform that can quickly and reliably replan actions based on sensed information. Robotic manipulation is subject to noise due to uncertainty in frictional contact information, and reactivity is key for robustness. The planning framework has been designed with generality in mind and naturally extends to a variety of robotic tasks, manipulators and sensors. This design is validated experimentally on an ABB IRB 14000 dual-arm industrial collaborative robot. In this research, we are interested in dexterous robot manipulation, where the key technology is to move an object from an initial location to a desired configuration. The robot makes use of a high resolution tactile sensor to monitor the progress of the task and drive the reactive behavior of the robot to counter mistakes or unaccounted environment conditions. The motion planning framework is integrated with a task planner that dictates the high-level manipulation behavior of the robot, as well as a low-level controller, that adapts robot motions based on measured tactile signaOutgoin