25,152 research outputs found
Quantum query complexity of entropy estimation
Estimation of Shannon and R\'enyi entropies of unknown discrete distributions
is a fundamental problem in statistical property testing and an active research
topic in both theoretical computer science and information theory. Tight bounds
on the number of samples to estimate these entropies have been established in
the classical setting, while little is known about their quantum counterparts.
In this paper, we give the first quantum algorithms for estimating
-R\'enyi entropies (Shannon entropy being 1-Renyi entropy). In
particular, we demonstrate a quadratic quantum speedup for Shannon entropy
estimation and a generic quantum speedup for -R\'enyi entropy
estimation for all , including a tight bound for the
collision-entropy (2-R\'enyi entropy). We also provide quantum upper bounds for
extreme cases such as the Hartley entropy (i.e., the logarithm of the support
size of a distribution, corresponding to ) and the min-entropy case
(i.e., ), as well as the Kullback-Leibler divergence between
two distributions. Moreover, we complement our results with quantum lower
bounds on -R\'enyi entropy estimation for all .Comment: 43 pages, 1 figur
How Many Queries Will Resolve Common Randomness?
A set of m terminals, observing correlated signals, communicate interactively
to generate common randomness for a given subset of them. Knowing only the
communication, how many direct queries of the value of the common randomness
will resolve it? A general upper bound, valid for arbitrary signal alphabets,
is developed for the number of such queries by using a query strategy that
applies to all common randomness and associated communication. When the
underlying signals are independent and identically distributed repetitions of m
correlated random variables, the number of queries can be exponential in signal
length. For this case, the mentioned upper bound is tight and leads to a
single-letter formula for the largest query exponent, which coincides with the
secret key capacity of a corresponding multiterminal source model. In fact, the
upper bound constitutes a strong converse for the optimum query exponent, and
implies also a new strong converse for secret key capacity. A key tool,
estimating the size of a large probability set in terms of Renyi entropy, is
interpreted separately, too, as a lossless block coding result for general
sources. As a particularization, it yields the classic result for a discrete
memoryless source.Comment: Accepted for publication in IEEE Transactions on Information Theor
Rate-Exponent Region for a Class of Distributed Hypothesis Testing Against Conditional Independence Problems
We study a class of -encoder hypothesis testing against conditional
independence problems. Under the criterion that stipulates minimization of the
Type II error subject to a (constant) upper bound on the Type I
error, we characterize the set of encoding rates and exponent for both discrete
memoryless and memoryless vector Gaussian settings. For the DM setting, we
provide a converse proof and show that it is achieved using the
Quantize-Bin-Test scheme of Rahman and Wagner. For the memoryless vector
Gaussian setting, we develop a tight outer bound by means of a technique that
relies on the de Bruijn identity and the properties of Fisher information. In
particular, the result shows that for memoryless vector Gaussian sources the
rate-exponent region is exhausted using the Quantize-Bin-Test scheme with
\textit{Gaussian} test channels; and there is \textit{no} loss in performance
caused by restricting the sensors' encoders not to employ time sharing.
Furthermore, we also study a variant of the problem in which the source, not
necessarily Gaussian, has finite differential entropy and the sensors'
observations noises under the null hypothesis are Gaussian. For this model, our
main result is an upper bound on the exponent-rate function. The bound is shown
to mirror a corresponding explicit lower bound, except that the lower bound
involves the source power (variance) whereas the upper bound has the source
entropy power. Part of the utility of the established bound is for
investigating asymptotic exponent/rates and losses incurred by distributed
detection as function of the number of sensors.Comment: Submitted for publication to the IEEE Transactions of Information
Theory. arXiv admin note: substantial text overlap with arXiv:1904.03028,
arXiv:1811.0393
A Tight Convex Upper Bound on the Likelihood of a Finite Mixture
The likelihood function of a finite mixture model is a non-convex function
with multiple local maxima and commonly used iterative algorithms such as EM
will converge to different solutions depending on initial conditions. In this
paper we ask: is it possible to assess how far we are from the global maximum
of the likelihood? Since the likelihood of a finite mixture model can grow
unboundedly by centering a Gaussian on a single datapoint and shrinking the
covariance, we constrain the problem by assuming that the parameters of the
individual models are members of a large discrete set (e.g. estimating a
mixture of two Gaussians where the means and variances of both Gaussians are
members of a set of a million possible means and variances). For this setting
we show that a simple upper bound on the likelihood can be computed using
convex optimization and we analyze conditions under which the bound is
guaranteed to be tight. This bound can then be used to assess the quality of
solutions found by EM (where the final result is projected on the discrete set)
or any other mixture estimation algorithm. For any dataset our method allows us
to find a finite mixture model together with a dataset-specific bound on how
far the likelihood of this mixture is from the global optimum of the likelihoodComment: icpr 201
On the Capacity of the Wiener Phase-Noise Channel: Bounds and Capacity Achieving Distributions
In this paper, the capacity of the additive white Gaussian noise (AWGN)
channel, affected by time-varying Wiener phase noise is investigated. Tight
upper and lower bounds on the capacity of this channel are developed. The upper
bound is obtained by using the duality approach, and considering a specific
distribution over the output of the channel. In order to lower-bound the
capacity, first a family of capacity-achieving input distributions is found by
solving a functional optimization of the channel mutual information. Then,
lower bounds on the capacity are obtained by drawing samples from the proposed
distributions through Monte-Carlo simulations. The proposed capacity-achieving
input distributions are circularly symmetric, non-Gaussian, and the input
amplitudes are correlated over time. The evaluated capacity bounds are tight
for a wide range of signal-to-noise-ratio (SNR) values, and thus they can be
used to quantify the capacity. Specifically, the bounds follow the well-known
AWGN capacity curve at low SNR, while at high SNR, they coincide with the
high-SNR capacity result available in the literature for the phase-noise
channel.Comment: IEEE Transactions on Communications, 201
Contraction of Locally Differentially Private Mechanisms
We investigate the contraction properties of locally differentially private
mechanisms. More specifically, we derive tight upper bounds on the divergence
between and output distributions of an
-LDP mechanism in terms of a divergence between the
corresponding input distributions and , respectively. Our first main
technical result presents a sharp upper bound on the -divergence
in terms of and
. We also show that the same result holds for a large family of
divergences, including KL-divergence and squared Hellinger distance. The second
main technical result gives an upper bound on
in terms of total variation distance
and . We then utilize these bounds to
establish locally private versions of the van Trees inequality, Le Cam's,
Assouad's, and the mutual information methods, which are powerful tools for
bounding minimax estimation risks. These results are shown to lead to better
privacy analyses than the state-of-the-arts in several statistical problems
such as entropy and discrete distribution estimation, non-parametric density
estimation, and hypothesis testing
- …