24,105 research outputs found
Reducing statistical time-series problems to binary classification
We show how binary classification methods developed to work on i.i.d. data
can be used for solving statistical problems that are seemingly unrelated to
classification and concern highly-dependent time series. Specifically, the
problems of time-series clustering, homogeneity testing and the three-sample
problem are addressed. The algorithms that we construct for solving these
problems are based on a new metric between time-series distributions, which can
be evaluated using binary classification methods. Universal consistency of the
proposed algorithms is proven under most general assumptions. The theoretical
results are illustrated with experiments on synthetic and real-world data.Comment: In proceedings of NIPS 2012, pp. 2069-207
Spatial aggregation of local likelihood estimates with applications to classification
This paper presents a new method for spatially adaptive local (constant)
likelihood estimation which applies to a broad class of nonparametric models,
including the Gaussian, Poisson and binary response models. The main idea of
the method is, given a sequence of local likelihood estimates (``weak''
estimates), to construct a new aggregated estimate whose pointwise risk is of
order of the smallest risk among all ``weak'' estimates. We also propose a new
approach toward selecting the parameters of the procedure by providing the
prescribed behavior of the resulting estimate in the simple parametric
situation. We establish a number of important theoretical results concerning
the optimality of the aggregated estimate. In particular, our ``oracle'' result
claims that its risk is, up to some logarithmic multiplier, equal to the
smallest risk for the given family of estimates. The performance of the
procedure is illustrated by application to the classification problem. A
numerical study demonstrates its reasonable performance in simulated and
real-life examples.Comment: Published in at http://dx.doi.org/10.1214/009053607000000271 the
Annals of Statistics (http://www.imstat.org/aos/) by the Institute of
Mathematical Statistics (http://www.imstat.org
On the Convergence Speed of MDL Predictions for Bernoulli Sequences
We consider the Minimum Description Length principle for online sequence
prediction. If the underlying model class is discrete, then the total expected
square loss is a particularly interesting performance measure: (a) this
quantity is bounded, implying convergence with probability one, and (b) it
additionally specifies a `rate of convergence'. Generally, for MDL only
exponential loss bounds hold, as opposed to the linear bounds for a Bayes
mixture. We show that this is even the case if the model class contains only
Bernoulli distributions. We derive a new upper bound on the prediction error
for countable Bernoulli classes. This implies a small bound (comparable to the
one for Bayes mixtures) for certain important model classes. The results apply
to many Machine Learning tasks including classification and hypothesis testing.
We provide arguments that our theorems generalize to countable classes of
i.i.d. models.Comment: 17 page
- …