2,067 research outputs found
Sparse Inverse Covariance Selection via Alternating Linearization Methods
Gaussian graphical models are of great interest in statistical learning.
Because the conditional independencies between different nodes correspond to
zero entries in the inverse covariance matrix of the Gaussian distribution, one
can learn the structure of the graph by estimating a sparse inverse covariance
matrix from sample data, by solving a convex maximum likelihood problem with an
-regularization term. In this paper, we propose a first-order method
based on an alternating linearization technique that exploits the problem's
special structure; in particular, the subproblems solved in each iteration have
closed-form solutions. Moreover, our algorithm obtains an -optimal
solution in iterations. Numerical experiments on both synthetic
and real data from gene association networks show that a practical version of
this algorithm outperforms other competitive algorithms
PRISMA: PRoximal Iterative SMoothing Algorithm
Motivated by learning problems including max-norm regularized matrix
completion and clustering, robust PCA and sparse inverse covariance selection,
we propose a novel optimization algorithm for minimizing a convex objective
which decomposes into three parts: a smooth part, a simple non-smooth Lipschitz
part, and a simple non-smooth non-Lipschitz part. We use a time variant
smoothing strategy that allows us to obtain a guarantee that does not depend on
knowing in advance the total number of iterations nor a bound on the domain
Inference for Generalized Linear Models via Alternating Directions and Bethe Free Energy Minimization
Generalized Linear Models (GLMs), where a random vector is
observed through a noisy, possibly nonlinear, function of a linear transform
arise in a range of applications in nonlinear
filtering and regression. Approximate Message Passing (AMP) methods, based on
loopy belief propagation, are a promising class of approaches for approximate
inference in these models. AMP methods are computationally simple, general, and
admit precise analyses with testable conditions for optimality for large i.i.d.
transforms . However, the algorithms can easily diverge for general
. This paper presents a convergent approach to the generalized AMP
(GAMP) algorithm based on direct minimization of a large-system limit
approximation of the Bethe Free Energy (LSL-BFE). The proposed method uses a
double-loop procedure, where the outer loop successively linearizes the LSL-BFE
and the inner loop minimizes the linearized LSL-BFE using the Alternating
Direction Method of Multipliers (ADMM). The proposed method, called ADMM-GAMP,
is similar in structure to the original GAMP method, but with an additional
least-squares minimization. It is shown that for strictly convex, smooth
penalties, ADMM-GAMP is guaranteed to converge to a local minima of the
LSL-BFE, thus providing a convergent alternative to GAMP that is stable under
arbitrary transforms. Simulations are also presented that demonstrate the
robustness of the method for non-convex penalties as well
- …