15,604 research outputs found
Semidefinite Relaxations for Stochastic Optimal Control Policies
Recent results in the study of the Hamilton Jacobi Bellman (HJB) equation
have led to the discovery of a formulation of the value function as a linear
Partial Differential Equation (PDE) for stochastic nonlinear systems with a
mild constraint on their disturbances. This has yielded promising directions
for research in the planning and control of nonlinear systems. This work
proposes a new method obtaining approximate solutions to these linear
stochastic optimal control (SOC) problems. A candidate polynomial with variable
coefficients is proposed as the solution to the SOC problem. A Sum of Squares
(SOS) relaxation is then taken to the partial differential constraints, leading
to a hierarchy of semidefinite relaxations with improving sub-optimality gap.
The resulting approximate solutions are shown to be guaranteed over- and
under-approximations for the optimal value function.Comment: Preprint. Accepted to American Controls Conference (ACC) 2014 in
Portland, Oregon. 7 pages, colo
Domain Decomposition for Stochastic Optimal Control
This work proposes a method for solving linear stochastic optimal control
(SOC) problems using sum of squares and semidefinite programming. Previous work
had used polynomial optimization to approximate the value function, requiring a
high polynomial degree to capture local phenomena. To improve the scalability
of the method to problems of interest, a domain decomposition scheme is
presented. By using local approximations, lower degree polynomials become
sufficient, and both local and global properties of the value function are
captured. The domain of the problem is split into a non-overlapping partition,
with added constraints ensuring continuity. The Alternating Direction
Method of Multipliers (ADMM) is used to optimize over each domain in parallel
and ensure convergence on the boundaries of the partitions. This results in
improved conditioning of the problem and allows for much larger and more
complex problems to be addressed with improved performance.Comment: 8 pages. Accepted to CDC 201
A decomposition procedure based on approximate newton directions
The efficient solution of large-scale linear and nonlinear optimization problems may require exploiting any special structure in them in an efficient manner. We describe and analyze some cases in which this special structure can be used with very little cost to obtain search directions from decomposed subproblems. We also study how to correct these directions using (decomposable) preconditioned conjugate gradient methods to ensure local convergence in all cases. The choice of appropriate preconditioners results in a natural manner from the structure in the problem. Finally, we conduct computational experiments to compare the resulting procedures with direct methods, as well as to study the impact of different preconditioner choices
Bounds for deterministic and stochastic dynamical systems using sum-of-squares optimization
We describe methods for proving upper and lower bounds on infinite-time
averages in deterministic dynamical systems and on stationary expectations in
stochastic systems. The dynamics and the quantities to be bounded are assumed
to be polynomial functions of the state variables. The methods are
computer-assisted, using sum-of-squares polynomials to formulate sufficient
conditions that can be checked by semidefinite programming. In the
deterministic case, we seek tight bounds that apply to particular local
attractors. An obstacle to proving such bounds is that they do not hold
globally; they are generally violated by trajectories starting outside the
local basin of attraction. We describe two closely related ways past this
obstacle: one that requires knowing a subset of the basin of attraction, and
another that considers the zero-noise limit of the corresponding stochastic
system. The bounding methods are illustrated using the van der Pol oscillator.
We bound deterministic averages on the attracting limit cycle above and below
to within 1%, which requires a lower bound that does not hold for the unstable
fixed point at the origin. We obtain similarly tight upper and lower bounds on
stochastic expectations for a range of noise amplitudes. Limitations of our
methods for certain types of deterministic systems are discussed, along with
prospects for improvement.Comment: 25 pages; Added new Section 7.2; Added references; Corrected typos;
Submitted to SIAD
- …