36,268 research outputs found
Optimization Methods for Inverse Problems
Optimization plays an important role in solving many inverse problems.
Indeed, the task of inversion often either involves or is fully cast as a
solution of an optimization problem. In this light, the mere non-linear,
non-convex, and large-scale nature of many of these inversions gives rise to
some very challenging optimization problems. The inverse problem community has
long been developing various techniques for solving such optimization tasks.
However, other, seemingly disjoint communities, such as that of machine
learning, have developed, almost in parallel, interesting alternative methods
which might have stayed under the radar of the inverse problem community. In
this survey, we aim to change that. In doing so, we first discuss current
state-of-the-art optimization methods widely used in inverse problems. We then
survey recent related advances in addressing similar challenges in problems
faced by the machine learning community, and discuss their potential advantages
for solving inverse problems. By highlighting the similarities among the
optimization challenges faced by the inverse problem and the machine learning
communities, we hope that this survey can serve as a bridge in bringing
together these two communities and encourage cross fertilization of ideas.Comment: 13 page
Policy Gradients for CVaR-Constrained MDPs
We study a risk-constrained version of the stochastic shortest path (SSP)
problem, where the risk measure considered is Conditional Value-at-Risk (CVaR).
We propose two algorithms that obtain a locally risk-optimal policy by
employing four tools: stochastic approximation, mini batches, policy gradients
and importance sampling. Both the algorithms incorporate a CVaR estimation
procedure, along the lines of Bardou et al. [2009], which in turn is based on
Rockafellar-Uryasev's representation for CVaR and utilize the likelihood ratio
principle for estimating the gradient of the sum of one cost function
(objective of the SSP) and the gradient of the CVaR of the sum of another cost
function (in the constraint of SSP). The algorithms differ in the manner in
which they approximate the CVaR estimates/necessary gradients - the first
algorithm uses stochastic approximation, while the second employ mini-batches
in the spirit of Monte Carlo methods. We establish asymptotic convergence of
both the algorithms. Further, since estimating CVaR is related to rare-event
simulation, we incorporate an importance sampling based variance reduction
scheme into our proposed algorithms
Relative Entropy Relaxations for Signomial Optimization
Signomial programs (SPs) are optimization problems specified in terms of
signomials, which are weighted sums of exponentials composed with linear
functionals of a decision variable. SPs are non-convex optimization problems in
general, and families of NP-hard problems can be reduced to SPs. In this paper
we describe a hierarchy of convex relaxations to obtain successively tighter
lower bounds of the optimal value of SPs. This sequence of lower bounds is
computed by solving increasingly larger-sized relative entropy optimization
problems, which are convex programs specified in terms of linear and relative
entropy functions. Our approach relies crucially on the observation that the
relative entropy function -- by virtue of its joint convexity with respect to
both arguments -- provides a convex parametrization of certain sets of globally
nonnegative signomials with efficiently computable nonnegativity certificates
via the arithmetic-geometric-mean inequality. By appealing to representation
theorems from real algebraic geometry, we show that our sequences of lower
bounds converge to the global optima for broad classes of SPs. Finally, we also
demonstrate the effectiveness of our methods via numerical experiments
- …