831 research outputs found
Improved Vapnik Cervonenkis bounds
We give a new proof of VC bounds where we avoid the use of symmetrization and
use a shadow sample of arbitrary size. We also improve on the variance term.
This results in better constants, as shown on numerical examples. Moreover our
bounds still hold for non identically distributed independent random variables.
Keywords: Statistical learning theory, PAC-Bayesian theorems, VC dimension
Toric grammars: a new statistical approach to natural language modeling
We propose a new statistical model for computational linguistics. Rather than
trying to estimate directly the probability distribution of a random sentence
of the language, we define a Markov chain on finite sets of sentences with many
finite recurrent communicating classes and define our language model as the
invariant probability measures of the chain on each recurrent communicating
class. This Markov chain, that we call a communication model, recombines at
each step randomly the set of sentences forming its current state, using some
grammar rules. When the grammar rules are fixed and known in advance instead of
being estimated on the fly, we can prove supplementary mathematical properties.
In particular, we can prove in this case that all states are recurrent states,
so that the chain defines a partition of its state space into finite recurrent
communicating classes. We show that our approach is a decisive departure from
Markov models at the sentence level and discuss its relationships with Context
Free Grammars. Although the toric grammars we use are closely related to
Context Free Grammars, the way we generate the language from the grammar is
qualitatively different. Our communication model has two purposes. On the one
hand, it is used to define indirectly the probability distribution of a random
sentence of the language. On the other hand it can serve as a (crude) model of
language transmission from one speaker to another speaker through the
communication of a (large) set of sentences
Robust linear least squares regression
We consider the problem of robustly predicting as well as the best linear
combination of given functions in least squares regression, and variants of
this problem including constraints on the parameters of the linear combination.
For the ridge estimator and the ordinary least squares estimator, and their
variants, we provide new risk bounds of order without logarithmic factor
unlike some standard results, where is the size of the training data. We
also provide a new estimator with better deviations in the presence of
heavy-tailed noise. It is based on truncating differences of losses in a
min--max framework and satisfies a risk bound both in expectation and in
deviations. The key common surprising factor of these results is the absence of
exponential moment condition on the output distribution while achieving
exponential deviations. All risk bounds are obtained through a PAC-Bayesian
analysis on truncated differences of losses. Experimental results strongly back
up our truncated min--max estimator.Comment: Published in at http://dx.doi.org/10.1214/11-AOS918 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org). arXiv admin note: significant text
overlap with arXiv:0902.173
Constant payoff in zero-sum stochastic games
In a zero-sum stochastic game, at each stage, two adversary players take
decisions and receive a stage payoff determined by them and by a random
variable representing the state of nature. The total payoff is the discounted
sum of the stage payoffs. Assume that the players are very patient and use
optimal strategies. We then prove that, at any point in the game, players get
essentially the same expected payoff: the payoff is constant. This solves a
conjecture by Sorin, Venel and Vigeral (2010). The proof relies on the
semi-algebraic approach for discounted stochastic games introduced by Bewley
and Kohlberg (1976), on the theory of Markov chains with rare transitions,
initiated by Friedlin and Wentzell (1984), and on some variational inequalities
for value functions inspired by the recent work of Davini, Fathi, Iturriaga and
Zavidovique (2016
- …