Scalable First-Order Methods for Robust MDPs

Robust Markov Decision Processes (MDPs) are a powerful framework for modeling sequential decision-making problems with model uncertainty. This paper proposes the first first-order framework for solving robust MDPs. Our algorithm interleaves primal-dual first-order updates with approximate Value Iteration updates. By carefully controlling the tradeoff between the accuracy and cost of Value Iteration updates, we achieve an ergodic convergence rate of $O \left( A^{2} S^{3}\log(S)\log(\epsilon^{-1}) \epsilon^{-1} \right)$ for the best choice of parameters on ellipsoidal and Kullback-Leibler $s$-rectangular uncertainty sets, where $S$ and $A$ is the number of states and actions, respectively. Our dependence on the number of states and actions is significantly better (by a factor of $O(A^{1.5}S^{1.5})$) than that of pure Value Iteration algorithms. In numerical experiments on ellipsoidal uncertainty sets we show that our algorithm is significantly more scalable than state-of-the-art approaches. Our framework is also the first one to solve robust MDPs with $s$-rectangular KL uncertainty sets

Network Target Coordination for Design Optimization of Decomposed Systems

A complex engineered system is often decomposed into a number of different subsystems that interact on one another and together produce results not obtainable by the subsystems alone. Effective coordination of the interdependencies shared among these subsystems is critical to fulfill the stakeholder expectations and technical requirements of the original system. The past research has shown that various coordination methods obtain different solution accuracies and exhibit different computational efficiencies when solving a decomposed system. Addressing these coordination decisions may lead to improved complex system design. This dissertation studies coordination methods through two types of decomposition structures, hierarchical, and nonhierarchical. For coordinating hierarchically decomposed systems, linear and proximal cutting plane methods are applied based on augmented Lagrangian relaxation and analytical target cascading (ATC). Three nonconvex, nonlinear design problems are used to verify the numerical performance of the proposed coordination method and the obtained results are compared to traditional update schemes of subgradient-based algorithm. The results suggest that the cutting plane methods can significantly improve the solution accuracy and computational efficiency of the hierarchically decomposed systems. In addition, a biobjective optimization method is also used to capture optimality and feasibility. The numerical performance of the biobjective algorithm is verified by solving an analytical mass allocation problem. For coordinating nonhierarchically decomposed complex systems, network target coordination (NTC) is developed by modeling the distributed subsystems as different agents in a network. To realize parallel computing of the subsystems, NTC via a consensus alternating direction method of multipliers is applied to eliminate the use of the master problem, which is required by most distributed coordination methods. In NTC, the consensus is computed using a locally update scheme, providing the potential to realize an asynchronous solution process. The numerical performance of NTC is verified using a geometrical programming problem and two engineering problems

Multiplier methods for engineering optimization

International audienceMultiplier methods used to solve the constrained engineering optimization problem are described. These methods solve the problem by minimizing a sequence of unconstrained problems defined using the cost and constraint functions. The methods, proposed in 1969, have been determined to be quite robust, although not as efficient as other algorithms. They can be more effective for some engineering applications, such as optimum design and control oflarge scale dynamic systems. Since 1969 several modifications and extensions of the methods have been developed. Therefore, it is important to review the theory and computational procedures of these methods so that more efficient and effective ones can be developed for engineering applications. Recent methods that are similar to the multiplier methods are also discussed. These are continuous multiplier update, exact penalty and exponential penalty methods

Decentralised Optimisation and Control in Electrical Power Systems

Emerging smart-grid-enabling technologies will allow an unprecedented degree of observability and control at all levels in a power system. Combined with flexible demand devices (e.g. electric vehicles or various household appliances), increased distributed generation, and the potential development of small scale distributed storage, they could allow procuring energy at minimum cost and environmental impact. That however presupposes real-time coordination of demand of individual households and industries down at the distribution level, with generation and renewables at the transmission level. In turn this implies the need to solve energy management problems of a much larger scale compared to the one we currently solve today. This of course raises significant computational and communications challenges. The need for an answer to these problems is reflected in today’s power systems literature where a significant number of papers cover subjects such as generation and/or demand management at both transmission and/or distribution, electric vehicle charging, voltage control devices setting, etc. The methods used are centralized or decentralized, handling continuous and/or discrete controls, approximate or exact, and incorporate a wide range of problem formulations. All these papers tackle aspects of the same problem, i.e. the close to real-time determination of operating set-points for all controllable devices available in a power system. Yet, a consensus regarding the associated formulation and time-scale of application has not been reached. Of course, given the large scale of the problem, decentralization is unavoidably part of the solution. In this work we explore the existing and developing trends in energy management and place them into perspective through a complete framework that allows optimizing energy usage at all levels in a power system

A fast algorithm for quadratic resource allocation problems with nested constraints

We study the quadratic resource allocation problem and its variant with lower and upper constraints on nested sums of variables. This problem occurs in many applications, in particular battery scheduling within decentralized energy management (DEM) for smart grids. We present an algorithm for this problem that runs in $O(n \log n)$ time and, in contrast to existing algorithms for this problem, achieves this time complexity using relatively simple and easy-to-implement subroutines and data structures. This makes our algorithm very attractive for real-life adaptation and implementation. Numerical comparisons of our algorithm with a subroutine for battery scheduling within an existing tool for DEM research indicates that our algorithm significantly reduces the overall execution time of the DEM system, especially when the battery is expected to be completely full or empty multiple times in the optimal schedule. Moreover, computational experiments with synthetic data show that our algorithm outperforms the currently most efficient algorithm by more than one order of magnitude. In particular, our algorithm is able to solves all considered instances with up to one million variables in less than 17 seconds on a personal computer