Optimal Flood Control

A mathematical model for optimal control of the water levels in a chain of reservoirs is studied. Some remarks regarding sensitivity with respect to the time horizon, terminal cost and forecast of inflow are made

Mean-Field Optimal Control

We introduce the concept of {\it mean-field optimal control} which is the rigorous limit process connecting finite dimensional optimal control problems with ODE constraints modeling multi-agent interactions to an infinite dimensional optimal control problem with a constraint given by a PDE of Vlasov-type, governing the dynamics of the probability distribution of interacting agents. While in the classical mean-field theory one studies the behavior of a large number of small individuals {\it freely interacting} with each other, by simplifying the effect of all the other individuals on any given individual by a single averaged effect, we address the situation where the individuals are actually influenced also by an external {\it policy maker}, and we propagate its effect for the number $N$ of individuals going to infinity. On the one hand, from a modeling point of view, we take into account also that the policy maker is constrained to act according to optimal strategies promoting its most parsimonious interaction with the group of individuals. This will be realized by considering cost functionals including $L^1$-norm terms penalizing a broadly distributed control of the group, while promoting its sparsity. On the other hand, from the analysis point of view, and for the sake of generality, we consider broader classes of convex control penalizations. In order to develop this new concept of limit rigorously, we need to carefully combine the classical concept of mean-field limit, connecting the finite dimensional system of ODE describing the dynamics of each individual of the group to the PDE describing the dynamics of the respective probability distribution, with the well-known concept of $\Gamma$-convergence to show that optimal strategies for the finite dimensional problems converge to optimal strategies of the infinite dimensional problem.Comment: 31 page

Discrete Variational Optimal Control

This paper develops numerical methods for optimal control of mechanical systems in the Lagrangian setting. It extends the theory of discrete mechanics to enable the solutions of optimal control problems through the discretization of variational principles. The key point is to solve the optimal control problem as a variational integrator of a specially constructed higher-dimensional system. The developed framework applies to systems on tangent bundles, Lie groups, underactuated and nonholonomic systems with symmetries, and can approximate either smooth or discontinuous control inputs. The resulting methods inherit the preservation properties of variational integrators and result in numerically robust and easily implementable algorithms. Several theoretical and a practical examples, e.g. the control of an underwater vehicle, will illustrate the application of the proposed approach.Comment: 30 pages, 6 figure

Deception in Optimal Control

In this paper, we consider an adversarial scenario where one agent seeks to achieve an objective and its adversary seeks to learn the agent's intentions and prevent the agent from achieving its objective. The agent has an incentive to try to deceive the adversary about its intentions, while at the same time working to achieve its objective. The primary contribution of this paper is to introduce a mathematically rigorous framework for the notion of deception within the context of optimal control. The central notion introduced in the paper is that of a belief-induced reward: a reward dependent not only on the agent's state and action, but also adversary's beliefs. Design of an optimal deceptive strategy then becomes a question of optimal control design on the product of the agent's state space and the adversary's belief space. The proposed framework allows for deception to be defined in an arbitrary control system endowed with a reward function, as well as with additional specifications limiting the agent's control policy. In addition to defining deception, we discuss design of optimally deceptive strategies under uncertainties in agent's knowledge about the adversary's learning process. In the latter part of the paper, we focus on a setting where the agent's behavior is governed by a Markov decision process, and show that the design of optimally deceptive strategies under lack of knowledge about the adversary naturally reduces to previously discussed problems in control design on partially observable or uncertain Markov decision processes. Finally, we present two examples of deceptive strategies: a "cops and robbers" scenario and an example where an agent may use camouflage while moving. We show that optimally deceptive strategies in such examples follow the intuitive idea of how to deceive an adversary in the above settings

Localized LQR Optimal Control

This paper introduces a receding horizon like control scheme for localizable distributed systems, in which the effect of each local disturbance is limited spatially and temporally. We characterize such systems by a set of linear equality constraints, and show that the resulting feasibility test can be solved in a localized and distributed way. We also show that the solution of the local feasibility tests can be used to synthesize a receding horizon like controller that achieves the desired closed loop response in a localized manner as well. Finally, we formulate the Localized LQR (LLQR) optimal control problem and derive an analytic solution for the optimal controller. Through a numerical example, we show that the LLQR optimal controller, with its constraints on locality, settling time, and communication delay, can achieve similar performance as an unconstrained H2 optimal controller, but can be designed and implemented in a localized and distributed way.Comment: Extended version for 2014 CDC submissio

Infinite horizon sparse optimal control

A class of infinite horizon optimal control problems involving $L^p$-type cost functionals with $0<p\leq 1$ is discussed. The existence of optimal controls is studied for both the convex case with $p=1$ and the nonconvex case with $0<p<1$, and the sparsity structure of the optimal controls promoted by the $L^p$-type penalties is analyzed. A dynamic programming approach is proposed to numerically approximate the corresponding sparse optimal controllers

Numerical fractional optimal control of respiratory syncytial virus infection in Octave/MATLAB

In this article, we develop a simple mathematical GNU Octave/MATLAB code that is easy to modify for the simulation of mathematical models governed by fractional-order differential equations, and for the resolution of fractional-order optimal control problems through Pontryagin's maximum principle (indirect approach to optimal control). For this purpose, a fractional-order model for the respiratory syncytial virus (RSV) infection is considered. The model is an improvement of one first proposed by the authors in [Chaos Solitons Fractals 117 (2018), 142--149]. The initial value problem associated with the RSV infection fractional model is numerically solved using Garrapa's fde12 solver and two simple methods coded here in Octave/MATLAB: the fractional forward {Euler's} method and the predict-evaluate-correct-evaluate (PECE) method of Adams--Bashforth--Moulton. A fractional optimal control problem is then formulated having treatment as the control. The fractional Pontryagin maximum principle is used to characterize the fractional optimal control and the extremals of the problem are determined numerically through the implementation of the forward-backward PECE method. The implemented algorithms are available on GitHub and, at the end of the paper, in appendixes, both for the uncontrolled initial value problem as well as for the fractional optimal control problem, using the free GNU Octave computing software and assuring compatibility with MATLAB. The developed Octave/Matlab code is available at [https://github.com/SilverioRosa/numres-focp]This research was funded by The Portuguese Foundation for Science and Technology (FCT—Fundação para a Ciência e a Tecnologia), grants number UIDB/50008/2020 (S.R.) and UIDB/04106/2020 (D.F.M.T.).publishe