1,142 research outputs found
Limited-memory BFGS Systems with Diagonal Updates
In this paper, we investigate a formula to solve systems of the form (B +
{\sigma}I)x = y, where B is a limited-memory BFGS quasi-Newton matrix and
{\sigma} is a positive constant. These types of systems arise naturally in
large-scale optimization such as trust-region methods as well as
doubly-augmented Lagrangian methods. We show that provided a simple condition
holds on B_0 and \sigma, the system (B + \sigma I)x = y can be solved via a
recursion formula that requies only vector inner products. This formula has
complexity M^2n, where M is the number of L-BFGS updates and n >> M is the
dimension of x
On Reduced Input-Output Dynamic Mode Decomposition
The identification of reduced-order models from high-dimensional data is a
challenging task, and even more so if the identified system should not only be
suitable for a certain data set, but generally approximate the input-output
behavior of the data source. In this work, we consider the input-output dynamic
mode decomposition method for system identification. We compare excitation
approaches for the data-driven identification process and describe an
optimization-based stabilization strategy for the identified systems
Scaled Projected-Directions Methods with Application to Transmission Tomography
Statistical image reconstruction in X-Ray computed tomography yields
large-scale regularized linear least-squares problems with nonnegativity
bounds, where the memory footprint of the operator is a concern. Discretizing
images in cylindrical coordinates results in significant memory savings, and
allows parallel operator-vector products without on-the-fly computation of the
operator, without necessarily decreasing image quality. However, it
deteriorates the conditioning of the operator. We improve the Hessian
conditioning by way of a block-circulant scaling operator and we propose a
strategy to handle nondiagonal scaling in the context of projected-directions
methods for bound-constrained problems. We describe our implementation of the
scaling strategy using two algorithms: TRON, a trust-region method with exact
second derivatives, and L-BFGS-B, a linesearch method with a limited-memory
quasi-Newton Hessian approximation. We compare our approach with one where a
change of variable is made in the problem. On two reconstruction problems, our
approach converges faster than the change of variable approach, and achieves
much tighter accuracy in terms of optimality residual than a first-order
method.Comment: 19 pages, 7 figure
A quasi-Newton proximal splitting method
A new result in convex analysis on the calculation of proximity operators in
certain scaled norms is derived. We describe efficient implementations of the
proximity calculation for a useful class of functions; the implementations
exploit the piece-wise linear nature of the dual problem. The second part of
the paper applies the previous result to acceleration of convex minimization
problems, and leads to an elegant quasi-Newton method. The optimization method
compares favorably against state-of-the-art alternatives. The algorithm has
extensive applications including signal processing, sparse recovery and machine
learning and classification
On Quasi-Newton Forward--Backward Splitting: Proximal Calculus and Convergence
We introduce a framework for quasi-Newton forward--backward splitting
algorithms (proximal quasi-Newton methods) with a metric induced by diagonal
rank- symmetric positive definite matrices. This special type of
metric allows for a highly efficient evaluation of the proximal mapping. The
key to this efficiency is a general proximal calculus in the new metric. By
using duality, formulas are derived that relate the proximal mapping in a
rank- modified metric to the original metric. We also describe efficient
implementations of the proximity calculation for a large class of functions;
the implementations exploit the piece-wise linear nature of the dual problem.
Then, we apply these results to acceleration of composite convex minimization
problems, which leads to elegant quasi-Newton methods for which we prove
convergence. The algorithm is tested on several numerical examples and compared
to a comprehensive list of alternatives in the literature. Our quasi-Newton
splitting algorithm with the prescribed metric compares favorably against
state-of-the-art. The algorithm has extensive applications including signal
processing, sparse recovery, machine learning and classification to name a few.Comment: arXiv admin note: text overlap with arXiv:1206.115
Efficient Semidefinite Spectral Clustering via Lagrange Duality
We propose an efficient approach to semidefinite spectral clustering (SSC),
which addresses the Frobenius normalization with the positive semidefinite
(p.s.d.) constraint for spectral clustering. Compared with the original
Frobenius norm approximation based algorithm, the proposed algorithm can more
accurately find the closest doubly stochastic approximation to the affinity
matrix by considering the p.s.d. constraint. In this paper, SSC is formulated
as a semidefinite programming (SDP) problem. In order to solve the high
computational complexity of SDP, we present a dual algorithm based on the
Lagrange dual formalization. Two versions of the proposed algorithm are
proffered: one with less memory usage and the other with faster convergence
rate. The proposed algorithm has much lower time complexity than that of the
standard interior-point based SDP solvers. Experimental results on both UCI
data sets and real-world image data sets demonstrate that 1) compared with the
state-of-the-art spectral clustering methods, the proposed algorithm achieves
better clustering performance; and 2) our algorithm is much more efficient and
can solve larger-scale SSC problems than those standard interior-point SDP
solvers.Comment: 13 page
- …