18 research outputs found
PAC-Bayesian Analysis of the Exploration-Exploitation Trade-off
We develop a coherent framework for integrative simultaneous analysis of the
exploration-exploitation and model order selection trade-offs. We improve over
our preceding results on the same subject (Seldin et al., 2011) by combining
PAC-Bayesian analysis with Bernstein-type inequality for martingales. Such a
combination is also of independent interest for studies of multiple
simultaneously evolving martingales.Comment: On-line Trading of Exploration and Exploitation 2 - ICML-2011
workshop. http://explo.cs.ucl.ac.uk/workshop
PAC-Bayesian Analysis of Martingales and Multiarmed Bandits
We present two alternative ways to apply PAC-Bayesian analysis to sequences
of dependent random variables. The first is based on a new lemma that enables
to bound expectations of convex functions of certain dependent random variables
by expectations of the same functions of independent Bernoulli random
variables. This lemma provides an alternative tool to Hoeffding-Azuma
inequality to bound concentration of martingale values. Our second approach is
based on integration of Hoeffding-Azuma inequality with PAC-Bayesian analysis.
We also introduce a way to apply PAC-Bayesian analysis in situation of limited
feedback. We combine the new tools to derive PAC-Bayesian generalization and
regret bounds for the multiarmed bandit problem. Although our regret bound is
not yet as tight as state-of-the-art regret bounds based on other
well-established techniques, our results significantly expand the range of
potential applications of PAC-Bayesian analysis and introduce a new analysis
tool to reinforcement learning and many other fields, where martingales and
limited feedback are encountered
Editors' Introduction to [Algorithmic Learning Theory: 21st International Conference, ALT 2010, Canberra, Australia, October 6-8, 2010. Proceedings]
Learning theory is an active research area that incorporates ideas,
problems, and techniques from a wide range of disciplines including
statistics, artificial intelligence, information theory, pattern
recognition, and theoretical computer science. The research reported
at the 21st International Conference on Algorithmic Learning Theory
(ALT 2010) ranges over areas such as query models, online learning,
inductive inference, boosting, kernel methods, complexity and
learning, reinforcement learning, unsupervised learning, grammatical
inference, and algorithmic forecasting. In this introduction we give
an overview of the five invited talks and the regular contributions
of ALT 2010
PAC-Bayesian Inequalities for Martingales
We present a set of high-probability inequalities that control the
concentration of weighted averages of multiple (possibly uncountably many)
simultaneously evolving and interdependent martingales. Our results extend the
PAC-Bayesian analysis in learning theory from the i.i.d. setting to martingales
opening the way for its application to importance weighted sampling,
reinforcement learning, and other interactive learning domains, as well as many
other domains in probability theory and statistics, where martingales are
encountered.
We also present a comparison inequality that bounds the expectation of a
convex function of a martingale difference sequence shifted to the [0,1]
interval by the expectation of the same function of independent Bernoulli
variables. This inequality is applied to derive a tighter analog of
Hoeffding-Azuma's inequality
PAC-Bayes bounds for stable algorithms with instance-dependent priors
PAC-Bayes bounds have been proposed to get risk estimates based on a training
sample. In this paper the PAC-Bayes approach is combined with stability of the
hypothesis learned by a Hilbert space valued algorithm. The PAC-Bayes setting
is used with a Gaussian prior centered at the expected output. Thus a novelty
of our paper is using priors defined in terms of the data-generating
distribution. Our main result estimates the risk of the randomized algorithm in
terms of the hypothesis stability coefficients. We also provide a new bound for
the SVM classifier, which is compared to other known bounds experimentally.
Ours appears to be the first stability-based bound that evaluates to
non-trivial values.Comment: 16 pages, discussion of theory and experiments in the main body,
detailed proofs and experimental details in the appendice
PAC-Bayes unleashed: generalisation bounds with unbounded losses
We present new PAC-Bayesian generalisation bounds for learning problems with
unbounded loss functions. This extends the relevance and applicability of the
PAC-Bayes learning framework, where most of the existing literature focuses on
supervised learning problems with a bounded loss function (typically assumed to
take values in the interval [0;1]). In order to relax this assumption, we
propose a new notion called HYPE (standing for \emph{HYPothesis-dependent
rangE}), which effectively allows the range of the loss to depend on each
predictor. Based on this new notion we derive a novel PAC-Bayesian
generalisation bound for unbounded loss functions, and we instantiate it on a
linear regression problem. To make our theory usable by the largest audience
possible, we include discussions on actual computation, practicality and
limitations of our assumptions.Comment: 24 page
A Primer on PAC-Bayesian Learning
International audienc