IIn this paper we show that some HJB equations arising from both finite and
infinite horizon stochastic optimal control problems have a regular singular
point at the origin. This makes them amenable to solution by power series
techniques. This extends the work of Al'brecht who showed that the HJB
equations of an infinite horizon deterministic optimal control problem can have
a regular singular point at the origin, Al'brekht solved the HJB equations by
power series, degree by degree. In particular, we show that the infinite
horizon stochastic optimal control problem with linear dynamics, quadratic cost
and bilinear noise leads to a new type of algebraic Riccati equation which we
call the Stochastic Algebraic Riccati Equation (SARE). If SARE can be solved
then one has a complete solution to this infinite horizon stochastic optimal
control problem. We also show that a finite horizon stochastic optimal control
problem with linear dynamics, quadratic cost and bilinear noise leads to a
Stochastic Differential Riccati Equation (SDRE) that is well known. If these
problems are the linear-quadratic-bilinear part of a nonlinear finite horizon
stochastic optimal control problem then we show how the higher degree terms of
the solutions can be computed degree by degree. To our knowledge this
computation is new