Time Blocks Decomposition of Multistage Stochastic Optimization Problems

Multistage stochastic optimization problems are, by essence, complex because their solutions are indexed both by stages (time) and by uncertainties (scenarios). Their large scale nature makes decomposition methods appealing.The most common approaches are time decomposition --- and state-based resolution methods, like stochastic dynamic programming, in stochastic optimal control --- and scenario decomposition --- like progressive hedging in stochastic programming. We present a method to decompose multistage stochastic optimization problems by time blocks, which covers both stochastic programming and stochastic dynamic programming. Once established a dynamic programming equation with value functions defined on the history space (a history is a sequence of uncertainties and controls), we provide conditions to reduce the history using a compressed "state" variable. This reduction is done by time blocks, that is, at stages that are not necessarily all the original unit stages, and we prove areduced dynamic programming equation. Then, we apply the reduction method by time blocks to \emph{two time-scales} stochastic optimization problems and to a novel class of so-called \emph{decision-hazard-decision} problems, arising in many practical situations, like in stock management. The \emph{time blocks decomposition} scheme is as follows: we use dynamic programming at slow time scale where the slow time scale noises are supposed to be stagewise independent, and we produce slow time scale Bellman functions; then, we use stochastic programming at short time scale, within two consecutive slow time steps, with the final short time scale cost given by the slow time scale Bellman functions, and without assuming stagewise independence for the short time scale noises

The Capacity of Channels with Feedback

We introduce a general framework for treating channels with memory and feedback. First, we generalize Massey's concept of directed information and use it to characterize the feedback capacity of general channels. Second, we present coding results for Markov channels. This requires determining appropriate sufficient statistics at the encoder and decoder. Third, a dynamic programming framework for computing the capacity of Markov channels is presented. Fourth, it is shown that the average cost optimality equation (ACOE) can be viewed as an implicit single-letter characterization of the capacity. Fifth, scenarios with simple sufficient statistics are described