Infinite horizon sparse optimal control

A class of infinite horizon optimal control problems involving $L^p$-type cost functionals with $0<p\leq 1$ is discussed. The existence of optimal controls is studied for both the convex case with $p=1$ and the nonconvex case with $0<p<1$, and the sparsity structure of the optimal controls promoted by the $L^p$-type penalties is analyzed. A dynamic programming approach is proposed to numerically approximate the corresponding sparse optimal controllers

Approximate Dynamic Programming via Sum of Squares Programming

We describe an approximate dynamic programming method for stochastic control problems on infinite state and input spaces. The optimal value function is approximated by a linear combination of basis functions with coefficients as decision variables. By relaxing the Bellman equation to an inequality, one obtains a linear program in the basis coefficients with an infinite set of constraints. We show that a recently introduced method, which obtains convex quadratic value function approximations, can be extended to higher order polynomial approximations via sum of squares programming techniques. An approximate value function can then be computed offline by solving a semidefinite program, without having to sample the infinite constraint. The policy is evaluated online by solving a polynomial optimization problem, which also turns out to be convex in some cases. We experimentally validate the method on an autonomous helicopter testbed using a 10-dimensional helicopter model.Comment: 7 pages, 5 figures. Submitted to the 2013 European Control Conference, Zurich, Switzerlan

Stackelberg Games of Water Extraction

International audienceWe consider a discrete time, infinite horizon dynamic game of groundwater extraction. A Water Agency charges an extraction cost to water users, and controls the marginal extraction cost so that it depends linearly on total water extraction (through a parameter n) and on rainfall (through parameter m). The water users are selfish and myopic, and the goal of the agency is to give them incentives them so as to, at the same time, improve their total welfare and improve the long-term level of the resource. We look at this problem in several situations for a linear-quadratic model. In the first situation, the parameters n and m are considered to be fixed over time, and the Agency selects the value that maximizes the total discounted welfare of agents. We analyze this solution, from the economic and environmental point of view, as a function of model parameters,including the discount factor that is used. A first result shows that when Water Agency is patient (discount factor tends to 1) optimal marginal extraction cost asks for strategic interactions between agents. In a second situation, we look at the dynamic Stackelberg game where the Agency decides at each time what cost parameter they must announce in order to maximize the welfare function. We present the sensitivity analysis of the solution for a small time horizon, and present a numerical scheme for the infinite-horizon problem

Stochastic Model Predictive Control with Discounted Probabilistic Constraints

This paper considers linear discrete-time systems with additive disturbances, and designs a Model Predictive Control (MPC) law to minimise a quadratic cost function subject to a chance constraint. The chance constraint is defined as a discounted sum of violation probabilities on an infinite horizon. By penalising violation probabilities close to the initial time and ignoring violation probabilities in the far future, this form of constraint enables the feasibility of the online optimisation to be guaranteed without an assumption of boundedness of the disturbance. A computationally convenient MPC optimisation problem is formulated using Chebyshev's inequality and we introduce an online constraint-tightening technique to ensure recursive feasibility based on knowledge of a suboptimal solution. The closed loop system is guaranteed to satisfy the chance constraint and a quadratic stability condition.Comment: 6 pages, Conference Proceeding