36 research outputs found
Limited-memory BFGS Systems with Diagonal Updates
In this paper, we investigate a formula to solve systems of the form (B +
{\sigma}I)x = y, where B is a limited-memory BFGS quasi-Newton matrix and
{\sigma} is a positive constant. These types of systems arise naturally in
large-scale optimization such as trust-region methods as well as
doubly-augmented Lagrangian methods. We show that provided a simple condition
holds on B_0 and \sigma, the system (B + \sigma I)x = y can be solved via a
recursion formula that requies only vector inner products. This formula has
complexity M^2n, where M is the number of L-BFGS updates and n >> M is the
dimension of x
On Solving L-SR1 Trust-Region Subproblems
In this article, we consider solvers for large-scale trust-region subproblems
when the quadratic model is defined by a limited-memory symmetric rank-one
(L-SR1) quasi-Newton matrix. We propose a solver that exploits the compact
representation of L-SR1 matrices. Our approach makes use of both an orthonormal
basis for the eigenspace of the L-SR1 matrix and the Sherman-Morrison-Woodbury
formula to compute global solutions to trust-region subproblems. To compute the
optimal Lagrange multiplier for the trust-region constraint, we use Newton's
method with a judicious initial guess that does not require safeguarding. A
crucial property of this solver is that it is able to compute high-accuracy
solutions even in the so-called hard case. Additionally, the optimal solution
is determined directly by formula, not iteratively. Numerical experiments
demonstrate the effectiveness of this solver.Comment: 2015-0
Sequential anomaly detection in the presence of noise and limited feedback
This paper describes a methodology for detecting anomalies from sequentially
observed and potentially noisy data. The proposed approach consists of two main
elements: (1) {\em filtering}, or assigning a belief or likelihood to each
successive measurement based upon our ability to predict it from previous noisy
observations, and (2) {\em hedging}, or flagging potential anomalies by
comparing the current belief against a time-varying and data-adaptive
threshold. The threshold is adjusted based on the available feedback from an
end user. Our algorithms, which combine universal prediction with recent work
on online convex programming, do not require computing posterior distributions
given all current observations and involve simple primal-dual parameter
updates. At the heart of the proposed approach lie exponential-family models
which can be used in a wide variety of contexts and applications, and which
yield methods that achieve sublinear per-round regret against both static and
slowly varying product distributions with marginals drawn from the same
exponential family. Moreover, the regret against static distributions coincides
with the minimax value of the corresponding online strongly convex game. We
also prove bounds on the number of mistakes made during the hedging step
relative to the best offline choice of the threshold with access to all
estimated beliefs and feedback signals. We validate the theory on synthetic
data drawn from a time-varying distribution over binary vectors of high
dimensionality, as well as on the Enron email dataset.Comment: 19 pages, 12 pdf figures; final version to be published in IEEE
Transactions on Information Theor