2,506 research outputs found
Strong Converse and Second-Order Asymptotics of Channel Resolvability
We study the problem of channel resolvability for fixed i.i.d. input
distributions and discrete memoryless channels (DMCs), and derive the strong
converse theorem for any DMCs that are not necessarily full rank. We also
derive the optimal second-order rate under a condition. Furthermore, under the
condition that a DMC has the unique capacity achieving input distribution, we
derive the optimal second-order rate of channel resolvability for the worst
input distribution.Comment: 7 pages, a shorter version will appear in ISIT 2014, this version
includes the proofs of technical lemmas in appendice
Asymptotics of Fingerprinting and Group Testing: Capacity-Achieving Log-Likelihood Decoders
We study the large-coalition asymptotics of fingerprinting and group testing,
and derive explicit decoders that provably achieve capacity for many of the
considered models. We do this both for simple decoders (fast but suboptimal)
and for joint decoders (slow but optimal), and both for informed and uninformed
settings.
For fingerprinting, we show that if the pirate strategy is known, the
Neyman-Pearson-based log-likelihood decoders provably achieve capacity,
regardless of the strategy. The decoder built against the interleaving attack
is further shown to be a universal decoder, able to deal with arbitrary attacks
and achieving the uninformed capacity. This universal decoder is shown to be
closely related to the Lagrange-optimized decoder of Oosterwijk et al. and the
empirical mutual information decoder of Moulin. Joint decoders are also
proposed, and we conjecture that these also achieve the corresponding joint
capacities.
For group testing, the simple decoder for the classical model is shown to be
more efficient than the one of Chan et al. and it provably achieves the simple
group testing capacity. For generalizations of this model such as noisy group
testing, the resulting simple decoders also achieve the corresponding simple
capacities.Comment: 14 pages, 2 figure
Statistical mechanics of the vertex-cover problem
We review recent progress in the study of the vertex-cover problem (VC). VC
belongs to the class of NP-complete graph theoretical problems, which plays a
central role in theoretical computer science. On ensembles of random graphs, VC
exhibits an coverable-uncoverable phase transition. Very close to this
transition, depending on the solution algorithm, easy-hard transitions in the
typical running time of the algorithms occur.
We explain a statistical mechanics approach, which works by mapping VC to a
hard-core lattice gas, and then applying techniques like the replica trick or
the cavity approach. Using these methods, the phase diagram of VC could be
obtained exactly for connectivities , where VC is replica symmetric.
Recently, this result could be confirmed using traditional mathematical
techniques. For , the solution of VC exhibits full replica symmetry
breaking.
The statistical mechanics approach can also be used to study analytically the
typical running time of simple complete and incomplete algorithms for VC.
Finally, we describe recent results for VC when studied on other ensembles of
finite- and infinite-dimensional graphs.Comment: review article, 26 pages, 9 figures, to appear in J. Phys. A: Math.
Ge
Non-negative Principal Component Analysis: Message Passing Algorithms and Sharp Asymptotics
Principal component analysis (PCA) aims at estimating the direction of
maximal variability of a high-dimensional dataset. A natural question is: does
this task become easier, and estimation more accurate, when we exploit
additional knowledge on the principal vector? We study the case in which the
principal vector is known to lie in the positive orthant. Similar constraints
arise in a number of applications, ranging from analysis of gene expression
data to spike sorting in neural signal processing.
In the unconstrained case, the estimation performances of PCA has been
precisely characterized using random matrix theory, under a statistical model
known as the `spiked model.' It is known that the estimation error undergoes a
phase transition as the signal-to-noise ratio crosses a certain threshold.
Unfortunately, tools from random matrix theory have no bearing on the
constrained problem. Despite this challenge, we develop an analogous
characterization in the constrained case, within a one-spike model.
In particular: ~We prove that the estimation error undergoes a similar
phase transition, albeit at a different threshold in signal-to-noise ratio that
we determine exactly; ~We prove that --unlike in the unconstrained case--
estimation error depends on the spike vector, and characterize the least
favorable vectors; ~We show that a non-negative principal component can
be approximately computed --under the spiked model-- in nearly linear time.
This despite the fact that the problem is non-convex and, in general, NP-hard
to solve exactly.Comment: 51 pages, 7 pdf figure
Optimal Discrete Riesz Energy and Discrepancy
The Riesz -energy of an -point configuration in the Euclidean space
is defined as the sum of reciprocal -powers of all mutual
distances in this system. In the limit the Riesz -potential
( the Euclidean distance) governing the point interaction is replaced with
the logarithmic potential . In particular, we present a conjecture
for the leading term of the asymptotic expansion of the optimal
\IL_2-discrepancy with respect to spherical caps on the unit sphere in
which follows from Stolarsky's invariance principle [Proc.
Amer. Math. Soc. 41 (1973)] and the fundamental conjecture for the first two
terms of the asymptotic expansion of the optimal Riesz -energy of points
as .Comment: 8 page
Exact Recovery for a Family of Community-Detection Generative Models
Generative models for networks with communities have been studied extensively
for being a fertile ground to establish information-theoretic and computational
thresholds. In this paper we propose a new toy model for planted generative
models called planted Random Energy Model (REM), inspired by Derrida's REM. For
this model we provide the asymptotic behaviour of the probability of error for
the maximum likelihood estimator and hence the exact recovery threshold. As an
application, we further consider the 2 non-equally sized community Weighted
Stochastic Block Model (2-WSBM) on -uniform hypergraphs, that is equivalent
to the P-REM on both sides of the spectrum, for high and low edge cardinality
. We provide upper and lower bounds for the exact recoverability for any
, mapping these problems to the aforementioned P-REM. To the best of our
knowledge these are the first consistency results for the 2-WSBM on graphs and
on hypergraphs with non-equally sized community
- …