19 research outputs found
The Mutual Information in Random Linear Estimation Beyond i.i.d. Matrices
There has been definite progress recently in proving the variational
single-letter formula given by the heuristic replica method for various
estimation problems. In particular, the replica formula for the mutual
information in the case of noisy linear estimation with random i.i.d. matrices,
a problem with applications ranging from compressed sensing to statistics, has
been proven rigorously. In this contribution we go beyond the restrictive
i.i.d. matrix assumption and discuss the formula proposed by Takeda, Uda,
Kabashima and later by Tulino, Verdu, Caire and Shamai who used the replica
method. Using the recently introduced adaptive interpolation method and random
matrix theory, we prove this formula for a relevant large sub-class of
rotationally invariant matrices.Comment: Presented at the 2018 IEEE International Symposium on Information
Theory (ISIT
Deep Boltzmann Machines: Rigorous Results at Arbitrary Depth
A class of deep Boltzmann machines is considered in the simplified
framework of a quenched system with Gaussian noise and independent entries. The quenched pressure of a K-layers spin glass model is
studied allowing interactions only among consecutive layers. A lower
bound for the pressure is found in terms of a convex combination of
K Sherrington-Kirkpatrick models and used to study the annealed
and replica symmetric regimes of the system. A map with a one dimensional monomer-dimer system is identified and used to rigorously
control the annealed region at arbitrary depth K with the methods
introduced by Heilmann and Lieb. The compression of this high noise
region displays a remarkable phenomenon of localisation of the processing layers. Furthermore a replica symmetric lower bound for the
limiting quenched pressure of the model is obtained in a suitable region
of the parameters and the replica symmetric pressure is proved to have
a unique stationary point
Concentration of multi-overlaps for random ferromagnetic spin models
We consider ferromagnetic spin models on dilute random graphs and prove that,
with suitable one-body infinitesimal perturbations added to the Hamiltonian,
the multi-overlaps concentrate for all temperatures, both with respect to the
thermal Gibbs average and the quenched randomness. Results of this nature have
been known only for the lowest order overlaps, at high temperature or on the
Nishimori line. Here we treat all multi-overlaps by a non-trivial application
of Griffiths-Kelly-Sherman correlation inequalities. Our results apply in
particular to the pure and mixed p-spin ferromagnets on random dilute
Erdoes-R\'enyi hypergraphs. On physical grounds one expects that multi-overlap
concentration directly implies the correctness of the cavity (or replica
symmetric) formula for the pressure. The proof of this formula for the general
p-spin ferromagnet on a random dilute hypergraph remains an open problem