Error estimates for a tree structure algorithm solving finite horizon control problems

In the Dynamic Programming approach to optimal control problems a crucial role is played by the value function that is characterized as the unique viscosity solution of a Hamilton-Jacobi-Bellman (HJB) equation. It is well known that this approach suffers of the "curse of dimensionality" and this limitation has reduced its practical in real world applications. Here we analyze a dynamic programming algorithm based on a tree structure. The tree is built by the time discrete dynamics avoiding in this way the use of a fixed space grid which is the bottleneck for high-dimensional problems, this also drops the projection on the grid in the approximation of the value function. We present some error estimates for a first order approximation based on the tree-structure algorithm. Moreover, we analyze a pruning technique for the tree to reduce the complexity and minimize the computational effort. Finally, we present some numerical tests

Neural networks for first order HJB equations and application to front propagation with obstacle terms

We consider a deterministic optimal control problem with a maximum running cost functional, in a finite horizon context, and propose deep neural network approximations for Bellman's dynamic programming principle, corresponding also to some first-order Hamilton-Jacobi-Bellman equations. This work follows the lines of Hur\'e et al. (SIAM J. Numer. Anal., vol. 59 (1), 2021, pp. 525-557) where algorithms are proposed in a stochastic context. However, we need to develop a completely new approach in order to deal with the propagation of errors in the deterministic setting, where no diffusion is present in the dynamics. Our analysis gives precise error estimates in an average norm. The study is then illustrated on several academic numerical examples related to front propagations models in the presence of obstacle constraints, showing the relevance of the approach for average dimensions (e.g. from $2$ to $8$), even for non-smooth value functions

COMPUTATION OF AVOIDANCE REGIONS FOR DRIVER ASSISTANCE SYSTEMS BY USING A HAMILTON-JACOBI APPROACH

International audienceWe consider the problem of computing safety regions, modeled as nonconvex backward reachable sets, for a nonlinear car collision avoidance model with time-dependent obstacles. The Hamilton-Jacobi-Bellman framework is used. A new formulation of level set functions for obstacle avoidance is given and sufficient conditions for granting the obstacle avoidance on the whole time interval are obtained, even though the conditions are checked only at discrete times. Different scenarios including various road configurations, different geometry of vehicle and obstacles, as well as fixed or moving obstacles, are then studied and computed. Computations involve solving nonlinear partial differential equations of up to five space dimensions plus time with nonsmooth obstacle representations, and an efficient solver is used to this end. A comparison with a direct optimal control approach is also done for one of the examples

Neural networks for differential games

We study deterministic optimal control problems for differential games with finite horizon. We propose new approximations of the strategies in feedback form, and show error estimates and a convergence result of the value in some weak sense for one of the formulations. This result applies in particular to neural networks approximations. This work follows some ideas introduced in Bokanowski, Prost and Warin (PDEA, 2023) for deterministic optimal control problems, yet with a simplified approach for the error estimates, which allows to consider a global optimization scheme instead of a time-marching scheme. We also give a new approximation result between the continuous and the semi-discrete optimal control value in the game setting, improving the classical convergence order under some assumptions on the dynamical system. Numerical examples are performed on elementary academic problems related to backward reachability, with exact analytic solutions given, as well as a two-player game in presence of state constraints. We use stochastic gradient type algorithms in order to deal with the min-max problem.Comment: 43 page

Optimal control for safety-critical systems

Enforcing safety plays a crucial role within the optimisation and control literature. Despite notable advances in recent years, optimal control for safety-critical and high-dimensional systems remains a challenging problem. Developing a general theoretical framework for integrating safety within optimal control is not tractable as the numerical inaccuracy and computational cost often grow exponentially with the number of states. Furthermore, different notions of safety require different methodologies and present unique theoretical and computational challenges. This thesis focuses on the challenges that arise when addressing scalability and safety considerations simultaneously. Safety is a multi-facet problem that involves hard constraint satisfaction, avoiding sharing information considered as private, as well as robustifying towards uncertainty that could otherwise compromise safety. The initial chapters of the thesis focus on Hamilton-Jacobi reachability, which has become a well-established method of computing reachable sets for complex nonlinear systems. In addition to enforcing that the system remains within a safe part of the state-space, we consider application-specific abstractions to deal with scalability, the interplay between competing performance objectives and safety objectives, and the challenges arising from multi-objective optimal control problems. We then investigate safety considerations due to the amount of information that needs to be shared between agents in a multi-agent networked control setting. Extending classical state-aggregation in approximate dynamic programming, we introduce a method of solving a large-scale Markov Decision Process in a fully distributed manner. The final chapter considers stochastic safety constraints under a statistical learning theoretic lens. Utilising randomised algorithms, we establish probably approximately correct (PAC) bounds on predicting a future label in a binary classification problem whereby the classifier changes in an unknown structured manner

Optimal control of normalized simr models with vaccination and treatment

We study a model based on the so called SIR model to control the spreading of a disease in a varying population via vaccination and treatment. Since we assume that medical treatment is not immediate we add a new compartment, M, to the SIR model. We work with the normalized version of the proposed model. For such model we consider the problem of steering the system to a specified target. We consider both a fixed time optimal control problem with L-1 cost and the minimum time problem to drive the system to the target. In contrast to the literature, we apply different techniques of optimal control to our problems of interest. Using the direct method, we first solve the fixed time problem and then proceed to validate the computed solutions using both necessary conditions and second order sufficient conditions. Noteworthy, we perform a sensitivity analysis of the solutions with respect to some parameters in the model. We also use the Hamiltonian Jacobi approach to study how the minimum time function varies with respect to perturbations of the initial conditions. Additionally, we consider a multi-objective approach to study the trade off between the minimum time and the social costs of the control of diseases. Finally, we propose the application of Model Predictive Control to deal with uncertainties of the model

Game-Theoretic Safety Assurance for Human-Centered Robotic Systems

In order for autonomous systems like robots, drones, and self-driving cars to be reliably introduced into our society, they must have the ability to actively account for safety during their operation. While safety analysis has traditionally been conducted offline for controlled environments like cages on factory floors, the much higher complexity of open, human-populated spaces like our homes, cities, and roads makes it unviable to rely on common design-time assumptions, since these may be violated once the system is deployed. Instead, the next generation of robotic technologies will need to reason about safety online, constructing high-confidence assurances informed by ongoing observations of the environment and other agents, in spite of models of them being necessarily fallible.This dissertation aims to lay down the necessary foundations to enable autonomous systems to ensure their own safety in complex, changing, and uncertain environments, by explicitly reasoning about the gap between their models and the real world. It first introduces a suite of novel robust optimal control formulations and algorithmic tools that permit tractable safety analysis in time-varying, multi-agent systems, as well as safe real-time robotic navigation in partially unknown environments; these approaches are demonstrated on large-scale unmanned air traffic simulation and physical quadrotor platforms. After this, it draws on Bayesian machine learning methods to translate model-based guarantees into high-confidence assurances, monitoring the reliability of predictive models in light of changing evidence about the physical system and surrounding agents. This principle is first applied to a general safety framework allowing the use of learning-based control (e.g. reinforcement learning) for safety-critical robotic systems such as drones, and then combined with insights from cognitive science and dynamic game theory to enable safe human-centered navigation and interaction; these techniques are showcased on physical quadrotors—flying in unmodeled wind and among human pedestrians—and simulated highway driving. The dissertation ends with a discussion of challenges and opportunities ahead, including the bridging of safety analysis and reinforcement learning and the need to ``close the loop'' around learning and adaptation in order to deploy increasingly advanced autonomous systems with confidence

Multiresolution strategies for the numerical solution of optimal control problems

Optimal control problems are often characterized by discontinuities or switchings in the control variables. One way of accurately capturing the irregularities in the solution is to use a high resolution (dense) uniform grid. This requires a large amount of computational resources both in terms of CPU time and memory. Hence, in order to accurately capture any irregularities in the solution using a few computational resources, one can refine the mesh locally in the region close to an irregularity instead of refining the mesh uniformly over the whole domain. Therefore, a novel multiresolution scheme for data compression has been designed which is shown to outperform similar data compression schemes. Specifically, we have shown that the proposed approach results in fewer grid points in the grid compared to a common multiresolution data compression scheme. The validity of the proposed mesh refinement algorithm has been verified by solving several challenging initial-boundary value problems for evolution equations in 1D. The examples have demonstrated the stability and robustness of the proposed algorithm. Next, a direct multiresolution-based approach for solving trajectory optimization problems is developed. The original optimal control problem is transcribed into a nonlinear programming (NLP) problem that is solved using standard NLP codes. The novelty of the proposed approach hinges on the automatic calculation of a suitable, nonuniform grid over which the NLP problem is solved, which tends to increase numerical efficiency and robustness. Control and/or state constraints are handled with ease, and without any additional computational complexity. The proposed algorithm is based on a simple and intuitive method to balance several conflicting objectives, such as accuracy of the solution, convergence, and speed of the computations. The benefits of the proposed algorithm over uniform grid implementations are demonstrated with the help of several nontrivial examples. Furthermore, two sequential multiresolution trajectory optimization algorithms for solving problems with moving targets and/or dynamically changing environments have been developed.Ph.D.Committee Chair: Tsiotras, Panagiotis; Committee Member: Calise, Anthony J.; Committee Member: Egerstedt, Magnus; Committee Member: Prasad, J. V. R.; Committee Member: Russell, Ryan P.; Committee Member: Zhou, Hao-Mi

Multi-objective optimal control problems with application to energy management systems

Orientadora: Dra. Elizabeth Wegner KarasCoorientadoras: Dra. Claudia Sagastizabal, Dra. Hasnaa ZidaniDissertação (mestrado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Matemática. Defesa : Curitiba, 22/02/2019Inclui referências: p. 122-130Resumo: Este trabalho investiga problemas de controle ótimo em tem po contínuo. Em horizonte de tem po finito, apresentamos um a nova abordagem ao analisar problemas com objetivos de diferentes naturezas, que precisam ser minimizados simultaneamente. Um objetivo esta na forma clássica de Bolza e o outro e definido como um a funcão de maximo. Baseados na teoria de Hamilton-Jacobi-Bellman caracterizamos a fronteira de Pareto fraca e a fronteira de Pareto para tais problemas. Prim eiram ente, definimos um problema de controle átim o auxiliar sem restricoes de estado e mostramos que a fronteira de Pareto fraca íe um subconjunto do conjunto de nível zero da funcao valor correspondente. Em seguida, com um a abordagem geometrica estabelecemos a caracterizaçao da fronteira de Pareto. Alguns resultados numericos sao considerados para m ostrar a relevancia do nosso metodo. Os bons resultados obtidos para horizonte de tem po finito nos m otivaram a investigar problemas de controle íotimo multiobjetivo com horizonte de tem po infinito. Com um a abordagem similar, caracterizamos a fronteira de Pareto para essa classe de problemas. Introduzimos um metodo, baseado no princípio da programacão dinamica, para reconstrucão de trajetorias de problemas de controle otimo com restricçoães de estado e horizonte de tem po infinito. A teoria íe aplicada em sistemas de gestãao de energia. Para problemas de energia simples, mas ainda representativos, que minimizam custo de geracao e emissão de CO2, comparamos a habilidade de diferentes baterias como substituto para o mecanismo de deslocamento de dem anda de ponta (load shaving). Com a resoluçcãao do problema multiobjetivo íe possível obter um a relaçcaão entre a minimizacao dos custos de geraçao de energia e de emissao de gás carbônico das usinas term oeletricas consideradas no modelo. P a la v ra s-c h a v e : Controle otimo multi-objetivo; caracterizaçao da fronteira de Pareto; abordagem de Hamilton-Jacobi-Bellman; baterias para armazenamento de energia; resposta a demanda.Abstract: In this work we investigate optim al control problems in continuous time. A novel theory is developed for finite horizon problems w ith two objectives of different nature th a t need to be minimized simultaneously. One objective is in the classical Bolza form and the other one is defined as a maximum function. Based on the Hamilton-Jacobi-Bellman framework we characterize the weak Pareto front and the Pareto front for such problems. First we define an auxiliary optim al control problem w ithout state constraints and show th a t the weak Pareto front is a subset of the zero level set of the corresponding value function. Then w ith a geometrical approach we establish a characterization of the Pareto front. Some numerical examples are considered to show the interest of our proposal. The encouraging results obtained with the finite horizon m otivated us to investigate infinite horizon multi-objective optim al control problems and characterize the corresponding Pareto front. Additionally, we introduce a m ethod, based on the dynamical programming principle, to reconstruct optim al trajectories for infinite horizon control problems w ith state constraints. The theory is applied to energy management systems. We compare the ability of different batteries as a substitute of the load shaving mechanism in smoothing the load peaks, for simple, yet representative, power mix systems w ith two different objectives. The multi-objective approach makes it possible to obtain a compromise between the minimization of generation costs and the carbon emissions of the therm al power plants in the mix. K e y w o rd s: M ulti-Objective optim al control problems; Pareto front characterization; Hamilton-Jacobi-Bellman approach; energy management systems; battery energy storage systems; dem and response