845 research outputs found
Mutual Information in Rank-One Matrix Estimation
We consider the estimation of a n-dimensional vector x from the knowledge of
noisy and possibility non-linear element-wise measurements of xxT , a very
generic problem that contains, e.g. stochastic 2-block model, submatrix
localization or the spike perturbation of random matrices. We use an
interpolation method proposed by Guerra and later refined by Korada and Macris.
We prove that the Bethe mutual information (related to the Bethe free energy
and conjectured to be exact by Lesieur et al. on the basis of the non-rigorous
cavity method) always yields an upper bound to the exact mutual information. We
also provide a lower bound using a similar technique. For concreteness, we
illustrate our findings on the sparse PCA problem, and observe that (a) our
bounds match for a large region of parameters and (b) that it exists a phase
transition in a region where the spectum remains uninformative. While we
present only the case of rank-one symmetric matrix estimation, our proof
technique is readily extendable to low-rank symmetric matrix or low-rank
symmetric tensor estimationComment: 8 pages, 1 figure
A Deterministic Equivalent for the Analysis of Non-Gaussian Correlated MIMO Multiple Access Channels
Large dimensional random matrix theory (RMT) has provided an efficient
analytical tool to understand multiple-input multiple-output (MIMO) channels
and to aid the design of MIMO wireless communication systems. However, previous
studies based on large dimensional RMT rely on the assumption that the transmit
correlation matrix is diagonal or the propagation channel matrix is Gaussian.
There is an increasing interest in the channels where the transmit correlation
matrices are generally nonnegative definite and the channel entries are
non-Gaussian. This class of channel models appears in several applications in
MIMO multiple access systems, such as small cell networks (SCNs). To address
these problems, we use the generalized Lindeberg principle to show that the
Stieltjes transforms of this class of random matrices with Gaussian or
non-Gaussian independent entries coincide in the large dimensional regime. This
result permits to derive the deterministic equivalents (e.g., the Stieltjes
transform and the ergodic mutual information) for non-Gaussian MIMO channels
from the known results developed for Gaussian MIMO channels, and is of great
importance in characterizing the spectral efficiency of SCNs.Comment: This paper is the revision of the original manuscript titled "A
Deterministic Equivalent for the Analysis of Small Cell Networks". We have
revised the original manuscript and reworked on the organization to improve
the presentation as well as readabilit
Invariance of visual operations at the level of receptive fields
Receptive field profiles registered by cell recordings have shown that
mammalian vision has developed receptive fields tuned to different sizes and
orientations in the image domain as well as to different image velocities in
space-time. This article presents a theoretical model by which families of
idealized receptive field profiles can be derived mathematically from a small
set of basic assumptions that correspond to structural properties of the
environment. The article also presents a theory for how basic invariance
properties to variations in scale, viewing direction and relative motion can be
obtained from the output of such receptive fields, using complementary
selection mechanisms that operate over the output of families of receptive
fields tuned to different parameters. Thereby, the theory shows how basic
invariance properties of a visual system can be obtained already at the level
of receptive fields, and we can explain the different shapes of receptive field
profiles found in biological vision from a requirement that the visual system
should be invariant to the natural types of image transformations that occur in
its environment.Comment: 40 pages, 17 figure
A Universal Analysis of Large-Scale Regularized Least Squares Solutions
A problem that has been of recent interest in statistical inference, machine learning and signal processing is that of understanding the asymptotic behavior of regularized least squares solutions under random measurement matrices (or dictionaries). The Least Absolute Shrinkage and Selection Operator (LASSO or least-squares with â„“_1 regularization) is perhaps one of the most interesting examples. Precise expressions for the asymptotic performance of LASSO have been obtained for a number of different cases, in particular when the elements of the dictionary matrix are sampled independently from a Gaussian distribution. It has also been empirically observed that the resulting expressions remain valid when the entries of the dictionary matrix are independently sampled from certain non-Gaussian distributions. In this paper, we confirm these observations theoretically when the distribution is sub-Gaussian. We further generalize the previous expressions for a broader family of regularization functions and under milder conditions on the underlying random, possibly non-Gaussian, dictionary matrix. In particular, we establish the universality of the asymptotic statistics (e.g., the average quadratic risk) of LASSO with non-Gaussian dictionaries
Asymptotic Mutual Information for the Two-Groups Stochastic Block Model
We develop an information-theoretic view of the stochastic block model, a
popular statistical model for the large-scale structure of complex networks. A
graph from such a model is generated by first assigning vertex labels at
random from a finite alphabet, and then connecting vertices with edge
probabilities depending on the labels of the endpoints. In the case of the
symmetric two-group model, we establish an explicit `single-letter'
characterization of the per-vertex mutual information between the vertex labels
and the graph.
The explicit expression of the mutual information is intimately related to
estimation-theoretic quantities, and --in particular-- reveals a phase
transition at the critical point for community detection. Below the critical
point the per-vertex mutual information is asymptotically the same as if edges
were independent. Correspondingly, no algorithm can estimate the partition
better than random guessing. Conversely, above the threshold, the per-vertex
mutual information is strictly smaller than the independent-edges upper bound.
In this regime there exists a procedure that estimates the vertex labels better
than random guessing.Comment: 41 pages, 3 pdf figure
- …