19,929 research outputs found
Vector Approximate Message Passing for the Generalized Linear Model
The generalized linear model (GLM), where a random vector is
observed through a noisy, possibly nonlinear, function of a linear transform
output , arises in a range of applications such
as robust regression, binary classification, quantized compressed sensing,
phase retrieval, photon-limited imaging, and inference from neural spike
trains. When is large and i.i.d. Gaussian, the generalized
approximate message passing (GAMP) algorithm is an efficient means of MAP or
marginal inference, and its performance can be rigorously characterized by a
scalar state evolution. For general , though, GAMP can
misbehave. Damping and sequential-updating help to robustify GAMP, but their
effects are limited. Recently, a "vector AMP" (VAMP) algorithm was proposed for
additive white Gaussian noise channels. VAMP extends AMP's guarantees from
i.i.d. Gaussian to the larger class of rotationally invariant
. In this paper, we show how VAMP can be extended to the GLM.
Numerical experiments show that the proposed GLM-VAMP is much more robust to
ill-conditioning in than damped GAMP
Approximate Message Passing with Consistent Parameter Estimation and Applications to Sparse Learning
We consider the estimation of an i.i.d. (possibly non-Gaussian) vector \xbf
\in \R^n from measurements \ybf \in \R^m obtained by a general cascade model
consisting of a known linear transform followed by a probabilistic
componentwise (possibly nonlinear) measurement channel. A novel method, called
adaptive generalized approximate message passing (Adaptive GAMP), that enables
joint learning of the statistics of the prior and measurement channel along
with estimation of the unknown vector \xbf is presented. The proposed
algorithm is a generalization of a recently-developed EM-GAMP that uses
expectation-maximization (EM) iterations where the posteriors in the E-steps
are computed via approximate message passing. The methodology can be applied to
a large class of learning problems including the learning of sparse priors in
compressed sensing or identification of linear-nonlinear cascade models in
dynamical systems and neural spiking processes. We prove that for large i.i.d.
Gaussian transform matrices the asymptotic componentwise behavior of the
adaptive GAMP algorithm is predicted by a simple set of scalar state evolution
equations. In addition, we show that when a certain maximum-likelihood
estimation can be performed in each step, the adaptive GAMP method can yield
asymptotically consistent parameter estimates, which implies that the algorithm
achieves a reconstruction quality equivalent to the oracle algorithm that knows
the correct parameter values. Remarkably, this result applies to essentially
arbitrary parametrizations of the unknown distributions, including ones that
are nonlinear and non-Gaussian. The adaptive GAMP methodology thus provides a
systematic, general and computationally efficient method applicable to a large
range of complex linear-nonlinear models with provable guarantees.Comment: 14 pages, 3 figure
Estimation in Rotationally Invariant Generalized Linear Models via Approximate Message Passing
We consider the problem of signal estimation in generalized linear models
defined via rotationally invariant design matrices. Since these matrices can
have an arbitrary spectral distribution, this model is well suited for
capturing complex correlation structures which often arise in applications. We
propose a novel family of approximate message passing (AMP) algorithms for
signal estimation, and rigorously characterize their performance in the
high-dimensional limit via a state evolution recursion. Our rotationally
invariant AMP has complexity of the same order as the existing AMP derived
under the restrictive assumption of a Gaussian design; our algorithm also
recovers this existing AMP as a special case. Numerical results showcase a
performance close to Vector AMP (which is conjectured to be Bayes-optimal in
some settings), but obtained with a much lower complexity, as the proposed
algorithm does not require a computationally expensive singular value
decomposition.Comment: 35 pages, 8 figures, to appear in International Conference on Machine
Learning (ICML), 202
Hybrid approximate message passing
Gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable recent attention due to their computational simplicity, analytic tractability, and wide applicability in optimization and statistical inference problems. This paper presents a systematic framework for incorporating such approximate message passing (AMP) methods in general graphical models. The key concept is a partition of dependencies of a general graphical model into strong and weak edges, with the weak edges representing interactions through aggregates of small, linearizable couplings of variables. AMP approximations based on the Central Limit Theorem can be readily applied to aggregates of many weak edges and integrated with standard message passing updates on the strong edges. The resulting algorithm, which we call hybrid generalized approximate message passing (HyGAMP), can yield significantly simpler implementations of sum-product and max-sum loopy belief propagation. By varying the partition of strong and weak edges, a performance--complexity trade-off can be achieved. Group sparsity and multinomial logistic regression problems are studied as examples of the proposed methodology.The work of S. Rangan was supported in part by the National Science Foundation under Grants 1116589, 1302336, and 1547332, and in part by the industrial affiliates of NYU WIRELESS. The work of A. K. Fletcher was supported in part by the National Science Foundation under Grants 1254204 and 1738286 and in part by the Office of Naval Research under Grant N00014-15-1-2677. The work of V. K. Goyal was supported in part by the National Science Foundation under Grant 1422034. The work of E. Byrne and P. Schniter was supported in part by the National Science Foundation under Grant CCF-1527162. (1116589 - National Science Foundation; 1302336 - National Science Foundation; 1547332 - National Science Foundation; 1254204 - National Science Foundation; 1738286 - National Science Foundation; 1422034 - National Science Foundation; CCF-1527162 - National Science Foundation; NYU WIRELESS; N00014-15-1-2677 - Office of Naval Research
An Overview of Multi-Processor Approximate Message Passing
Approximate message passing (AMP) is an algorithmic framework for solving
linear inverse problems from noisy measurements, with exciting applications
such as reconstructing images, audio, hyper spectral images, and various other
signals, including those acquired in compressive signal acquisiton systems. The
growing prevalence of big data systems has increased interest in large-scale
problems, which may involve huge measurement matrices that are unsuitable for
conventional computing systems. To address the challenge of large-scale
processing, multiprocessor (MP) versions of AMP have been developed. We provide
an overview of two such MP-AMP variants. In row-MP-AMP, each computing node
stores a subset of the rows of the matrix and processes corresponding
measurements. In column- MP-AMP, each node stores a subset of columns, and is
solely responsible for reconstructing a portion of the signal. We will discuss
pros and cons of both approaches, summarize recent research results for each,
and explain when each one may be a viable approach. Aspects that are
highlighted include some recent results on state evolution for both MP-AMP
algorithms, and the use of data compression to reduce communication in the MP
network
- …