A value iteration method for the average cost dynamic programming problem

Cover title.Includes bibliographical references (p. 13-14).Supported by NSF. 9300494-DMIby Dimitri P. Bertsekas

Підвищення ефективності інформаційного забезпечення процесів оборонного планування за рахунок використання розпаралелювання обчислень

Визначено підходи для підвищення ефективності інформаційного забезпечення процесів управління та планування в органах військового управління. Розвинено підхід, який базується на використанні системи збалансованих показників та методології чисельної оптимізації дискретних технологічних та інформаційних процесів. Запропоновано модифікований алгоритм методу Швейцера для основної схеми оптимізації дискретних технологічних та інформаційних процесів оборонного планування з розпаралелюванням обчислень

Spectral Theorem for Convex Monotone Homogeneous Maps, and Ergodic Control

We consider convex maps f:R^n -> R^n that are monotone (i.e., that preserve the product ordering of R^n), and nonexpansive for the sup-norm. This includes convex monotone maps that are additively homogeneous (i.e., that commute with the addition of constants). We show that the fixed point set of f, when it is non-empty, is isomorphic to a convex inf-subsemilattice of R^n, whose dimension is at most equal to the number of strongly connected components of a critical graph defined from the tangent affine maps of f. This yields in particular an uniqueness result for the bias vector of ergodic control problems. This generalizes results obtained previously by Lanery, Romanovsky, and Schweitzer and Federgruen, for ergodic control problems with finite state and action spaces, which correspond to the special case of piecewise affine maps f. We also show that the length of periodic orbits of f is bounded by the cyclicity of its critical graph, which implies that the possible orbit lengths of f are exactly the orders of elements of the symmetric group on n letters.Comment: 38 pages, 13 Postscript figure

Solution methods for controlled queueing networks

In this dissertation we look at a controlled queueing network where a controller routes the incoming arrivals to parallel queues using state-dependent rules. Besides this general arrival there are dedicated arrivals to each queue. The dedicated arrivals can only be served by their designated server, hence there is no routing decision involved. The goal of the controller is to find a stationary policy that will minimize the average number of customers in the system;The problem is modeled as a semi-Markov decision process and solved using techniques from the theory of Markov decision processes. We develop an efficient policy iteration based methodology which performs better than the value iteration method which is widely thought of as the best method to use for large-scale problems. The novelty in our approach is to use iterative methods in solving the system of linear equations, and also take advantage of the sparsity of matrices. The methodology could be used for other problems that are similar in nature. Using this methodology we solve much larger problems than reported in the literature. We also look at how several heuristic methods perform on our problem. No heuristic method is suitable to use for all instances. In general, however, these heuristic methods offer quick and reasonable solutions to very large problems

Optimal regulation of finite Markov Chains

Imperial Users onl

Optimization of stochastic-dynamic decision problems with applications in energy and production systems

Die vorliegende Arbeit beschäftigt sich mit der mathematischen Optimierung von stochastisch-dynamischen Entscheidungsproblemen. Diese Problemklasse stellt eine besondere Herausforderung für die mathematische Optimierung dar, da bislang kein Lösungsverfahren bekannt ist, das in polynomieller Zeit zu einer exakten Lösung konvergiert. Alle generischen Verfahren der dynamischen Optimierung unterliegen dem sogenannten "Fluch der Dimensionen", der dazu führt, dass die Problemkomplexität exponentiell in der Anzahl der Zustandsvariablen zunimmt. Da Entscheidungsprobleme von realistischer Größenordnung meist über eine Vielzahl von Zustandsvariablen verfügen, stoßen exakte Lösungsverfahren schnell an ihre Grenzen. Einen vielversprechenden Ausweg, um dem Fluch der Dimensionen zu entgehen, stellen Verfahren der "approximativ-dynamischen Optimierung" dar (engl.: "approximate dynamic programming"), welche versuchen eine Nährungslösung des stochastisch-dynamischen Problems zu berechnen. Diese Verfahren erzeugen eine künstliche Stichprobe des Entscheidungsprozesses mittels Monte-Carlo-Simulation und konstruieren basierend auf dieser Stichprobe eine Approximation der Wertfunktion des dynamischen Problems. Dabei wird die Stichprobe so gewählt, dass lediglich diejenigen Zustände in die Stichprobe aufgenommen werden, welche für den Entscheidungsprozess von Bedeutung sind, wodurch eine vollständige Enumeration des Zustandsraums vermieden wird. In dieser Arbeit werden Verfahren der approximativ-dynamischen Optimierung auf verschiedene Probleme der Produktions- und Energiewirtschaft angewendet und daraufhin überprüft, ob sie in der Lage sind, das zugrundeliegende mathematische Optimierungproblem nährungsweise zu lösen. Die Arbeit kommt zu dem Ergebnis, dass sich komplexe stochastisch-dynamische Bewirtschaftungsprobleme effizient lösen lassen, sofern das Optimierungsproblem konvex und der Zufallsprozess unabhängig vom Entscheidungsprozess ist. Handelt es sich hingegen um ein diskretes Optimierungsproblem, so stoßen auch Verfahren der approximativ-dynamischen Optimierung an ihre Grenzen. In diesem Fall sind gut kalibrierte, einfache Entscheidungsregeln möglicherweise die bessere Alternative.This thesis studies mathematical optimization methods for stochastic-dynamic decision problems. This problem class is particularly challenging, as there still exists no algorithm that converges to an exact solution in polynomial time. Existing generic solution methods are all subject to the "curse of dimensionality", which means that problem complexity increases exponentially in the number of state variables. Since problems of realistic size typically come with a large number of state variables, applying exact solution methods is impractical. A promising methodology to break the curse of dimensionality is "approximate dynamic programming". To avoid a complete enumeration of the state space, solution techniques based on this methodology use Monte Carlo simulation to sample states that are relevant to the decision process and then approximate the value function of the dynamic program by a function of much lower complexity. This thesis applies approximate dynamic programming techniques to different resource management problems that arise in production and energy settings and studies whether these techniques are capable of solving the underlying optimization problems. The thesis concludes that stochastic-dynamic resource management problems can be solved efficiently if the underlying optimization problem is convex and randomness independent of the resource states. If the optimization problem is discrete, however, the problem remains hard to solve, even for approximate dynamic programming techniques. In this case, simple but well-adjusted decision policies may be the better choice

Source coding for communication concentrators

Originally presented as the author's thesis, (Ph.D.) in the M.I.T. Dept. of Electrical Engineering and Computer Science, 1978.Prepared under Advanced Research Projects Agency Grant ONR-N00014-75-C-1183.Bibliography: p. 194-198.by Pierre Am?e Humblet

Finite memory estimation and control of finite probabilistic systems.

Bibliography : leaves 196-200.Thesis (Ph. D.)--Massachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science, 1977.Microfiche copy available in the Institute Archives and Barker Engineering Library.by Loren Kerry Platzman.Ph.D