Sufficient Conditions for Optimal Control Problems with Terminal Constraints and Free Terminal Times with Applications to Aerospace

Motivated by the flight control problem of designing control laws for a Ground Collision Avoidance System (GCAS), this thesis formulates sufficient conditions for a strong local minimum for a terminally constrained optimal control problem with a free-terminal time. The conditions develop within the framework of a construction of a field of extremals by means of the method of characteristics, a procedure for the solution of first-order linear partial differential equations, but modified to apply to the Hamilton-Jacobi-Bellman equation of optimal control. Additionally, the thesis constructs these sufficient conditions for optimality with a mathematically rigorous development. The proof uses an approach which generalizes and differs significantly from procedures outlined in the classical literature on control engineering, where similar formulas are derived, but only in a cursory, formal and sometimes incomplete way. Additionally, the thesis gives new arrangements of the relevant expressions arising in the formulation of sufficient conditions for optimality that lead to more concise formulas for the resulting perturbation feedback control schemes. These results are applied to an emergency perturbation-feedback guidance scheme which recovers an aircraft from a dangerous flight-path angle to a safe one. Discussion of required background material contrasts nonlinear and linear optimal control theory are contrasted in the context of aerospace applications. A simplified version of the classical model for an F-16 fighter aircraft is used in numerical computation to very, by example, that the sufficient conditions for optimality developed in this thesis can be used off-line to detect possible failures in perturbation feedback control schemes, which arise if such methods are applied along extremal controlled trajectories and which only satisfy the necessary conditions for optimality without being locally optimal. The sufficient conditions for optimality developed in this thesis, on the other hand, guarantee the local validity of such perturbation feedback control schemes. This thesis presents various graphs that compare the neighboring extremals which were derived from the perturbation feedback control scheme with optimal ones that start from the same initial condition. Future directions for this work include extending the perturbation feedback control schemes to optimization problems that are further constrained, possibly through control constraints, state-space constraints or mixed state-control constraints

A derivative-free approach for a simulation-based optimization problem in healthcare

Hospitals have been challenged in recent years to deliver high quality care with limited resources. Given the pressure to contain costs,developing procedures for optimal resource allocation becomes more and more critical in this context. Indeed, under/overutilization of emergency room and ward resources can either compromise a hospital's ability to provide the best possible care, or result in precious funding going toward underutilized resources. Simulation--based optimization tools then help facilitating the planning and management of hospital services, by maximizing/minimizing some specific indices (e.g. net profit) subject to given clinical and economical constraints. In this work, we develop a simulation--based optimization approach for the resource planning of a specific hospital ward. At each step, we first consider a suitably chosen resource setting and evaluate both efficiency and satisfaction of the restrictions by means of a discrete--event simulation model. Then, taking into account the information obtained by the simulation process, we use a derivative--free optimization algorithm to modify the given setting. We report results for a real--world problem coming from the obstetrics ward of an Italian hospital showing both the effectiveness and the efficiency of the proposed approach

Optimal Control of the Thermistor Problem in Three Spatial Dimensions

This paper is concerned with the state-constrained optimal control of the three-dimensional thermistor problem, a fully quasilinear coupled system of a parabolic and elliptic PDE with mixed boundary conditions. This system models the heating of a conducting material by means of direct current. Local existence, uniqueness and continuity for the state system are derived by employing maximal parabolic regularity in the fundamental theorem of Pr\"uss. Global solutions are addressed, which includes analysis of the linearized state system via maximal parabolic regularity, and existence of optimal controls is shown if the temperature gradient is under control. The adjoint system involving measures is investigated using a duality argument. These results allow to derive first-order necessary conditions for the optimal control problem in form of a qualified optimality system. The theoretical findings are illustrated by numerical results

Control Regularization for Reduced Variance Reinforcement Learning

Dealing with high variance is a significant challenge in model-free reinforcement learning (RL). Existing methods are unreliable, exhibiting high variance in performance from run to run using different initializations/seeds. Focusing on problems arising in continuous control, we propose a functional regularization approach to augmenting model-free RL. In particular, we regularize the behavior of the deep policy to be similar to a policy prior, i.e., we regularize in function space. We show that functional regularization yields a bias-variance trade-off, and propose an adaptive tuning strategy to optimize this trade-off. When the policy prior has control-theoretic stability guarantees, we further show that this regularization approximately preserves those stability guarantees throughout learning. We validate our approach empirically on a range of settings, and demonstrate significantly reduced variance, guaranteed dynamic stability, and more efficient learning than deep RL alone.Comment: Appearing in ICML 201