On average control generating families for singularly perturbed optimal control problems with long run average optimality criteria

The paper aims at the development of tools for analysis and construction of near optimal solutions of singularly perturbed (SP) optimal controls problems with long run average optimality criteria. The idea that we exploit is to first asymptotically approximate a given problem of optimal control of the SP system by a certain averaged optimal control problem, then reformulate this averaged problem as an infinite-dimensional (ID) linear programming (LP) problem, and then approximate the latter by semi-infinite LP problems. We show that the optimal solution of these semi-infinite LP problems and their duals (that can be found with the help of a modification of an available LP software) allow one to construct near optimal controls of the SP system. We demonstrate the construction with a numerical example.Comment: 36 pages, 4 figures. arXiv admin note: substantial text overlap with arXiv:1309.373

Use of approximations of Hamilton-Jacobi-Bellman inequality for solving periodic optimization problems

We show that necessary and sufficient conditions of optimality in periodic optimization problems can be stated in terms of a solution of the corresponding HJB inequality, the latter being equivalent to a max-min type variational problem considered on the space of continuously differentiable functions. We approximate the latter with a maximin problem on a finite dimensional subspace of the space of continuously differentiable functions and show that a solution of this problem (existing under natural controllability conditions) can be used for construction of near optimal controls. We illustrate the construction with a numerical example.Comment: 29 pages, 2 figure

From Infinite to Finite Programs: Explicit Error Bounds with Applications to Approximate Dynamic Programming

We consider linear programming (LP) problems in infinite dimensional spaces that are in general computationally intractable. Under suitable assumptions, we develop an approximation bridge from the infinite-dimensional LP to tractable finite convex programs in which the performance of the approximation is quantified explicitly. To this end, we adopt the recent developments in two areas of randomized optimization and first order methods, leading to a priori as well as a posterior performance guarantees. We illustrate the generality and implications of our theoretical results in the special case of the long-run average cost and discounted cost optimal control problems for Markov decision processes on Borel spaces. The applicability of the theoretical results is demonstrated through a constrained linear quadratic optimal control problem and a fisheries management problem.Comment: 30 pages, 5 figure

Restless bandit marginal productivity indices I: singleproject case and optimal control of a make-to-stock M/G/1 queue

This paper develops a framework based on convex optimization and economic ideas to formulate and solve by an index policy the problem of optimal dynamic effort allocation to a generic discrete-state restless bandit (i.e. binary-action: work/rest) project, elucidating a host of issues raised by Whittle (1988)Žs seminal work on the topic. Our contributions include: (i) a unifying definition of a projectŽs marginal productivity index (MPI), characterizing optimal policies; (ii) a complete characterization of indexability (existence of the MPI) as satisfaction by the project of the law of diminishing returns (to effort); (iii) sufficient indexability conditions based on partial conservation laws (PCLs), extending previous results of the author from the finite to the countable state case; (iv) application to a semi-Markov project, including a new MPI for a mixed longrun-average (LRA)/ bias criterion, which exists in relevant queueing control models where the index proposed by Whittle (1988) does not; and (v) optimal MPI policies for service-controlled make-to-order (MTO) and make-to-stock (MTS) M/G/1 queues with convex back order and stock holding cost rates, under discounted and LRA criteria

Active network management for electrical distribution systems: problem formulation, benchmark, and approximate solution

With the increasing share of renewable and distributed generation in electrical distribution systems, Active Network Management (ANM) becomes a valuable option for a distribution system operator to operate his system in a secure and cost-effective way without relying solely on network reinforcement. ANM strategies are short-term policies that control the power injected by generators and/or taken off by loads in order to avoid congestion or voltage issues. Advanced ANM strategies imply that the system operator has to solve large-scale optimal sequential decision-making problems under uncertainty. For example, decisions taken at a given moment constrain the future decisions that can be taken and uncertainty must be explicitly accounted for because neither demand nor generation can be accurately forecasted. We first formulate the ANM problem, which in addition to be sequential and uncertain, has a nonlinear nature stemming from the power flow equations and a discrete nature arising from the activation of power modulation signals. This ANM problem is then cast as a stochastic mixed-integer nonlinear program, as well as second-order cone and linear counterparts, for which we provide quantitative results using state of the art solvers and perform a sensitivity analysis over the size of the system, the amount of available flexibility, and the number of scenarios considered in the deterministic equivalent of the stochastic program. To foster further research on this problem, we make available at http://www.montefiore.ulg.ac.be/~anm/ three test beds based on distribution networks of 5, 33, and 77 buses. These test beds contain a simulator of the distribution system, with stochastic models for the generation and consumption devices, and callbacks to implement and test various ANM strategies