A Hierarchical Structure For Finite Horizon Dynamic Programming Problems

In dynamic programming (Markov decision) problems, hierarchicalstructure (aggregation) is usually used to simplify computation. Most research on aggregation ofMarkov decision problems is limited to the infinite horizon case, which has good tracking ability. However, in reallife, finite horizon stochastic shortest path problems are oftenencountered. In this paper, we propose a hierarchical structure to solve finite horizon stochastic shortest pathproblems in parallel. In general, the approach reducesthe time complexity of the original problem to a logarithm level, which hassignificant practical meaning

Adaptive aggregation methods for discounted dynamic programming

"Proceedings of the 25th IEEE Conferecne on Decision and Control, Athens, Greece, December 1986."Bibliography: p. 17.This work was sponsored by the Office of Naval Research under contract no. N00014-84-C-0577by Dimitri P. Bertsekas, David A. Castañon

Aggregation — Disaggregation Algorithms for Discrete Stochastic Systems

In this paper an aggregation — disaggregation method is formulated for a finite horizon Markov decision process with two-dimensional state and action spaces. This second dimension of the state and the action contains a similar type of information in which aggregation is both natural and simple. The quality of the approach is illustrated by an example

Adaptive aggregation methods for infinite horizon dynamic programming

Bibliography: p. 31-32.This work was sponsored by the Office of Naval Research under contract no. N00014-84-K-0577by Dimitri P. Bertsekas, David A. Castañon

Adaptive aggregation methods for infinite horizon dynamic programming

"July 1988."Includes bibliographical references.Work supported by the Office of Naval Research under contract N00014-84-C-0577by Dimitri P. Bertsekas, David A. Castañon

A New Adaptive Aggregation Algorithm for Infinite Horizon Dynamic Programming

Dynamic programming suffers the "curse of dimensionality" when it isemployed for complex control systems. State aggregation is used to solvethe problem and acceleratecomputation by looking for a sub-optimal policy. In this paper, a new method, which converges much faster thanconventional aggregated value iteration based on TD(0), is proposed for computing the valuefunctions of theaggregated system. Preliminary results show that the new method increases thespeed of convergence impressively. Aggregation introduces errorsinevitably. An adaptive aggregation scheme employing the newcomputation method isalso proposed to reduce the aggregation errors