1,798 research outputs found
A Variational Perspective on Accelerated Methods in Optimization
Accelerated gradient methods play a central role in optimization, achieving
optimal rates in many settings. While many generalizations and extensions of
Nesterov's original acceleration method have been proposed, it is not yet clear
what is the natural scope of the acceleration concept. In this paper, we study
accelerated methods from a continuous-time perspective. We show that there is a
Lagrangian functional that we call the \emph{Bregman Lagrangian} which
generates a large class of accelerated methods in continuous time, including
(but not limited to) accelerated gradient descent, its non-Euclidean extension,
and accelerated higher-order gradient methods. We show that the continuous-time
limit of all of these methods correspond to traveling the same curve in
spacetime at different speeds. From this perspective, Nesterov's technique and
many of its generalizations can be viewed as a systematic way to go from the
continuous-time curves generated by the Bregman Lagrangian to a family of
discrete-time accelerated algorithms.Comment: 38 pages. Subsumes an earlier working draft arXiv:1509.0361
Accelerated Linearized Bregman Method
In this paper, we propose and analyze an accelerated linearized Bregman (ALB)
method for solving the basis pursuit and related sparse optimization problems.
This accelerated algorithm is based on the fact that the linearized Bregman
(LB) algorithm is equivalent to a gradient descent method applied to a certain
dual formulation. We show that the LB method requires
iterations to obtain an -optimal solution and the ALB algorithm
reduces this iteration complexity to while requiring
almost the same computational effort on each iteration. Numerical results on
compressed sensing and matrix completion problems are presented that
demonstrate that the ALB method can be significantly faster than the LB method
Gravity Duals of Lifshitz-like Fixed Points
We find candidate macroscopic gravity duals for scale-invariant but
non-Lorentz invariant fixed points, which do not have particle number as a
conserved quantity. We compute two-point correlation functions which exhibit
novel behavior relative to their AdS counterparts, and find holographic
renormalization group flows to conformal field theories. Our theories are
characterized by a dynamical critical exponent , which governs the
anisotropy between spatial and temporal scaling , ; we focus on the case with . Such theories describe
multicritical points in certain magnetic materials and liquid crystals, and
have been shown to arise at quantum critical points in toy models of the
cuprate superconductors. This work can be considered a small step towards
making useful dual descriptions of such critical points.Comment: 17 pages, harvmac; v2 comments about behavior of metric near r=0
added (thanks to S. Hartnoll and G. Horowitz
Fast Image Recovery Using Variable Splitting and Constrained Optimization
We propose a new fast algorithm for solving one of the standard formulations
of image restoration and reconstruction which consists of an unconstrained
optimization problem where the objective includes an data-fidelity
term and a non-smooth regularizer. This formulation allows both wavelet-based
(with orthogonal or frame-based representations) regularization or
total-variation regularization. Our approach is based on a variable splitting
to obtain an equivalent constrained optimization formulation, which is then
addressed with an augmented Lagrangian method. The proposed algorithm is an
instance of the so-called "alternating direction method of multipliers", for
which convergence has been proved. Experiments on a set of image restoration
and reconstruction benchmark problems show that the proposed algorithm is
faster than the current state of the art methods.Comment: Submitted; 11 pages, 7 figures, 6 table
Efficient Linear Programming for Dense CRFs
The fully connected conditional random field (CRF) with Gaussian pairwise
potentials has proven popular and effective for multi-class semantic
segmentation. While the energy of a dense CRF can be minimized accurately using
a linear programming (LP) relaxation, the state-of-the-art algorithm is too
slow to be useful in practice. To alleviate this deficiency, we introduce an
efficient LP minimization algorithm for dense CRFs. To this end, we develop a
proximal minimization framework, where the dual of each proximal problem is
optimized via block coordinate descent. We show that each block of variables
can be efficiently optimized. Specifically, for one block, the problem
decomposes into significantly smaller subproblems, each of which is defined
over a single pixel. For the other block, the problem is optimized via
conditional gradient descent. This has two advantages: 1) the conditional
gradient can be computed in a time linear in the number of pixels and labels;
and 2) the optimal step size can be computed analytically. Our experiments on
standard datasets provide compelling evidence that our approach outperforms all
existing baselines including the previous LP based approach for dense CRFs.Comment: 24 pages, 10 figures and 4 table
- …