180 research outputs found
Properties of spatial coupling in compressed sensing
In this paper we address a series of open questions about the construction of
spatially coupled measurement matrices in compressed sensing. For hardware
implementations one is forced to depart from the limiting regime of parameters
in which the proofs of the so-called threshold saturation work. We investigate
quantitatively the behavior under finite coupling range, the dependence on the
shape of the coupling interaction, and optimization of the so-called seed to
minimize distance from optimality. Our analysis explains some of the properties
observed empirically in previous works and provides new insight on spatially
coupled compressed sensing.Comment: 5 pages, 6 figure
Blind Sensor Calibration using Approximate Message Passing
The ubiquity of approximately sparse data has led a variety of com- munities
to great interest in compressed sensing algorithms. Although these are very
successful and well understood for linear measurements with additive noise,
applying them on real data can be problematic if imperfect sensing devices
introduce deviations from this ideal signal ac- quisition process, caused by
sensor decalibration or failure. We propose a message passing algorithm called
calibration approximate message passing (Cal-AMP) that can treat a variety of
such sensor-induced imperfections. In addition to deriving the general form of
the algorithm, we numerically investigate two particular settings. In the
first, a fraction of the sensors is faulty, giving readings unrelated to the
signal. In the second, sensors are decalibrated and each one introduces a
different multiplicative gain to the measures. Cal-AMP shares the scalability
of approximate message passing, allowing to treat big sized instances of these
problems, and ex- perimentally exhibits a phase transition between domains of
success and failure.Comment: 27 pages, 9 figure
Critical Off-Equilibrium Dynamics in Glassy Systems
We consider off-equilibrium dynamics at the critical temperature in a class
of glassy system. The off-equilibrium correlation and response functions obey a
precise scaling form in the aging regime. The structure of the {\it
equilibrium} replicated Gibbs free energy fixes the corresponding {\it
off-equilibrium} scaling functions implicitly through two functional equations.
The details of the model enter these equations only through the ratio
of the cubic coefficients (proper vertexes) of the replicated Gibbs free
energy. Therefore the off-equilibrium dynamical exponents are controlled by the
very same parameter exponent that determines equilibrium
dynamics. We find approximate solutions to the equations and validate the
theory by means of analytical computations and numerical simulations.Comment: 19 pages, 7 figure
On Convergence of Approximate Message Passing
Approximate message passing is an iterative algorithm for compressed sensing
and related applications. A solid theory about the performance and convergence
of the algorithm exists for measurement matrices having iid entries of zero
mean. However, it was observed by several authors that for more general
matrices the algorithm often encounters convergence problems. In this paper we
identify the reason of the non-convergence for measurement matrices with iid
entries and non-zero mean in the context of Bayes optimal inference. Finally we
demonstrate numerically that when the iterative update is changed from parallel
to sequential the convergence is restored.Comment: 5 pages, 3 figure
Dynamics and termination cost of spatially coupled mean-field models
This work is motivated by recent progress in information theory and signal
processing where the so-called `spatially coupled' design of systems leads to
considerably better performance. We address relevant open questions about
spatially coupled systems through the study of a simple Ising model. In
particular, we consider a chain of Curie-Weiss models that are coupled by
interactions up to a certain range. Indeed, it is well known that the pure
(uncoupled) Curie-Weiss model undergoes a first order phase transition driven
by the magnetic field, and furthermore, in the spinodal region such systems are
unable to reach equilibrium in sub-exponential time if initialized in the
metastable state. By contrast, the spatially coupled system is, instead, able
to reach the equilibrium even when initialized to the metastable state. The
equilibrium phase propagates along the chain in the form of a travelling wave.
Here we study the speed of the wave-front and the so-called `termination
cost'--- \textit{i.e.}, the conditions necessary for the propagation to occur.
We reach several interesting conclusions about optimization of the speed and
the cost.Comment: 12 pages, 11 figure
Spectral Detection on Sparse Hypergraphs
We consider the problem of the assignment of nodes into communities from a
set of hyperedges, where every hyperedge is a noisy observation of the
community assignment of the adjacent nodes. We focus in particular on the
sparse regime where the number of edges is of the same order as the number of
vertices. We propose a spectral method based on a generalization of the
non-backtracking Hashimoto matrix into hypergraphs. We analyze its performance
on a planted generative model and compare it with other spectral methods and
with Bayesian belief propagation (which was conjectured to be asymptotically
optimal for this model). We conclude that the proposed spectral method detects
communities whenever belief propagation does, while having the important
advantages to be simpler, entirely nonparametric, and to be able to learn the
rule according to which the hyperedges were generated without prior
information.Comment: 8 pages, 5 figure
A Deterministic and Generalized Framework for Unsupervised Learning with Restricted Boltzmann Machines
Restricted Boltzmann machines (RBMs) are energy-based neural-networks which
are commonly used as the building blocks for deep architectures neural
architectures. In this work, we derive a deterministic framework for the
training, evaluation, and use of RBMs based upon the Thouless-Anderson-Palmer
(TAP) mean-field approximation of widely-connected systems with weak
interactions coming from spin-glass theory. While the TAP approach has been
extensively studied for fully-visible binary spin systems, our construction is
generalized to latent-variable models, as well as to arbitrarily distributed
real-valued spin systems with bounded support. In our numerical experiments, we
demonstrate the effective deterministic training of our proposed models and are
able to show interesting features of unsupervised learning which could not be
directly observed with sampling. Additionally, we demonstrate how to utilize
our TAP-based framework for leveraging trained RBMs as joint priors in
denoising problems
Inferring Sparsity: Compressed Sensing using Generalized Restricted Boltzmann Machines
In this work, we consider compressed sensing reconstruction from
measurements of -sparse structured signals which do not possess a writable
correlation model. Assuming that a generative statistical model, such as a
Boltzmann machine, can be trained in an unsupervised manner on example signals,
we demonstrate how this signal model can be used within a Bayesian framework of
signal reconstruction. By deriving a message-passing inference for general
distribution restricted Boltzmann machines, we are able to integrate these
inferred signal models into approximate message passing for compressed sensing
reconstruction. Finally, we show for the MNIST dataset that this approach can
be very effective, even for .Comment: IEEE Information Theory Workshop, 201
- …