6 research outputs found
Learning from compressed observations
The problem of statistical learning is to construct a predictor of a random
variable as a function of a related random variable on the basis of an
i.i.d. training sample from the joint distribution of . Allowable
predictors are drawn from some specified class, and the goal is to approach
asymptotically the performance (expected loss) of the best predictor in the
class. We consider the setting in which one has perfect observation of the
-part of the sample, while the -part has to be communicated at some
finite bit rate. The encoding of the -values is allowed to depend on the
-values. Under suitable regularity conditions on the admissible predictors,
the underlying family of probability distributions and the loss function, we
give an information-theoretic characterization of achievable predictor
performance in terms of conditional distortion-rate functions. The ideas are
illustrated on the example of nonparametric regression in Gaussian noise.Comment: 6 pages; submitted to the 2007 IEEE Information Theory Workshop (ITW
2007
Are Slepian-Wolf Rates Necessary for Distributed Parameter Estimation?
We consider a distributed parameter estimation problem, in which multiple
terminals send messages related to their local observations using limited rates
to a fusion center who will obtain an estimate of a parameter related to
observations of all terminals. It is well known that if the transmission rates
are in the Slepian-Wolf region, the fusion center can fully recover all
observations and hence can construct an estimator having the same performance
as that of the centralized case. One natural question is whether Slepian-Wolf
rates are necessary to achieve the same estimation performance as that of the
centralized case. In this paper, we show that the answer to this question is
negative. We establish our result by explicitly constructing an asymptotically
minimum variance unbiased estimator (MVUE) that has the same performance as
that of the optimal estimator in the centralized case while requiring
information rates less than the conditions required in the Slepian-Wolf rate
region.Comment: Accepted in Allerton 201