2,459 research outputs found
Shape-Changing Trust-Region Methods Using Multipoint Symmetric Secant Matrices
In this work, we consider methods for large-scale and nonconvex unconstrained
optimization. We propose a new trust-region method whose subproblem is defined
using a so-called "shape-changing" norm together with densely-initialized
multipoint symmetric secant (MSS) matrices to approximate the Hessian.
Shape-changing norms and dense initializations have been successfully used in
the context of traditional quasi-Newton methods, but have yet to be explored in
the case of MSS methods. Numerical results suggest that trust-region methods
that use densely-initialized MSS matrices together with shape-changing norms
outperform MSS with other trust-region methods
Efficient SDP Inference for Fully-connected CRFs Based on Low-rank Decomposition
Conditional Random Fields (CRF) have been widely used in a variety of
computer vision tasks. Conventional CRFs typically define edges on neighboring
image pixels, resulting in a sparse graph such that efficient inference can be
performed. However, these CRFs fail to model long-range contextual
relationships. Fully-connected CRFs have thus been proposed. While there are
efficient approximate inference methods for such CRFs, usually they are
sensitive to initialization and make strong assumptions. In this work, we
develop an efficient, yet general algorithm for inference on fully-connected
CRFs. The algorithm is based on a scalable SDP algorithm and the low- rank
approximation of the similarity/kernel matrix. The core of the proposed
algorithm is a tailored quasi-Newton method that takes advantage of the
low-rank matrix approximation when solving the specialized SDP dual problem.
Experiments demonstrate that our method can be applied on fully-connected CRFs
that cannot be solved previously, such as pixel-level image co-segmentation.Comment: 15 pages. A conference version of this work appears in Proc. IEEE
Conference on Computer Vision and Pattern Recognition, 201
Fast B-spline Curve Fitting by L-BFGS
We propose a novel method for fitting planar B-spline curves to unorganized
data points. In traditional methods, optimization of control points and foot
points are performed in two very time-consuming steps in each iteration: 1)
control points are updated by setting up and solving a linear system of
equations; and 2) foot points are computed by projecting each data point onto a
B-spline curve. Our method uses the L-BFGS optimization method to optimize
control points and foot points simultaneously and therefore it does not need to
perform either matrix computation or foot point projection in every iteration.
As a result, our method is much faster than existing methods
The Relationships Between CG, BFGS, and Two Limited-memory Algorithms
For the solution of linear systems, the conjugate gradient (CG) and BFGS are among the most popular and successful algorithms with their respective advantages. The limited-memory methods have been developed to combine the best of the two. We describe and examine CG, BFGS, and two limited-memory methods (L-BFGS and VSCG) in the context of linear systems. We focus on the relationships between each of the four algorithms, and we present numerical results to illustrate those relationships
Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields
We apply stochastic average gradient (SAG) algorithms for training
conditional random fields (CRFs). We describe a practical implementation that
uses structure in the CRF gradient to reduce the memory requirement of this
linearly-convergent stochastic gradient method, propose a non-uniform sampling
scheme that substantially improves practical performance, and analyze the rate
of convergence of the SAGA variant under non-uniform sampling. Our experimental
results reveal that our method often significantly outperforms existing methods
in terms of the training objective, and performs as well or better than
optimally-tuned stochastic gradient methods in terms of test error.Comment: AI/Stats 2015, 24 page
- …