Optimization problems involving sequential decisions in a stochastic
environment were studied in Stochastic Programming (SP), Stochastic Optimal
Control (SOC) and Markov Decision Processes (MDP). In this paper we mainly
concentrate on SP and SOC modelling approaches. In these frameworks there are
natural situations when the considered problems are convex. Classical approach
to sequential optimization is based on dynamic programming. It has the problem
of the so-called ``Curse of Dimensionality", in that its computational
complexity increases exponentially with increase of dimension of state
variables. Recent progress in solving convex multistage stochastic problems is
based on cutting planes approximations of the cost-to-go (value) functions of
dynamic programming equations. Cutting planes type algorithms in dynamical
settings is one of the main topics of this paper. We also discuss Stochastic
Approximation type methods applied to multistage stochastic optimization
problems. From the computational complexity point of view, these two types of
methods seem to be complimentary to each other. Cutting plane type methods can
handle multistage problems with a large number of stages, but a relatively
smaller number of state (decision) variables. On the other hand, stochastic
approximation type methods can only deal with a small number of stages, but a
large number of decision variables