41,166 research outputs found
Optimality of Universal Bayesian Sequence Prediction for General Loss and Alphabet
Various optimality properties of universal sequence predictors based on
Bayes-mixtures in general, and Solomonoff's prediction scheme in particular,
will be studied. The probability of observing at time , given past
observations can be computed with the chain rule if the true
generating distribution of the sequences is known. If
is unknown, but known to belong to a countable or continuous class \M
one can base ones prediction on the Bayes-mixture defined as a
-weighted sum or integral of distributions \nu\in\M. The cumulative
expected loss of the Bayes-optimal universal prediction scheme based on
is shown to be close to the loss of the Bayes-optimal, but infeasible
prediction scheme based on . We show that the bounds are tight and that no
other predictor can lead to significantly smaller bounds. Furthermore, for
various performance measures, we show Pareto-optimality of and give an
Occam's razor argument that the choice for the weights
is optimal, where is the length of the shortest program describing
. The results are applied to games of chance, defined as a sequence of
bets, observations, and rewards. The prediction schemes (and bounds) are
compared to the popular predictors based on expert advice. Extensions to
infinite alphabets, partial, delayed and probabilistic prediction,
classification, and more active systems are briefly discussed.Comment: 34 page
Bayesian Bounds on Parameter Estimation Accuracy for Compact Coalescing Binary Gravitational Wave Signals
A global network of laser interferometric gravitational wave detectors is
projected to be in operation by around the turn of the century. Here, the noisy
output of a single instrument is examined. A gravitational wave is assumed to
have been detected in the data and we deal with the subsequent problem of
parameter estimation. Specifically, we investigate theoretical lower bounds on
the minimum mean-square errors associated with measuring the parameters of the
inspiral waveform generated by an orbiting system of neutron stars/black holes.
Three theoretical lower bounds on parameter estimation accuracy are considered:
the Cramer-Rao bound (CRB); the Weiss-Weinstein bound (WWB); and the Ziv-Zakai
bound (ZZB). We obtain the WWB and ZZB for the Newtonian-form of the coalescing
binary waveform, and compare them with published CRB and numerical Monte-Carlo
results. At large SNR, we find that the theoretical bounds are all identical
and are attained by the Monte-Carlo results. As SNR gradually drops below 10,
the WWB and ZZB are both found to provide increasingly tighter lower bounds
than the CRB. However, at these levels of moderate SNR, there is a significant
departure between all the bounds and the numerical Monte-Carlo results.Comment: 17 pages (LaTeX), 4 figures. Submitted to Physical Review
Lower Bounds on Exponential Moments of the Quadratic Error in Parameter Estimation
Considering the problem of risk-sensitive parameter estimation, we propose a
fairly wide family of lower bounds on the exponential moments of the quadratic
error, both in the Bayesian and the non--Bayesian regime. This family of
bounds, which is based on a change of measures, offers considerable freedom in
the choice of the reference measure, and our efforts are devoted to explore
this freedom to a certain extent. Our focus is mostly on signal models that are
relevant to communication problems, namely, models of a parameter-dependent
signal (modulated signal) corrupted by additive white Gaussian noise, but the
methodology proposed is also applicable to other types of parametric families,
such as models of linear systems driven by random input signals (white noise,
in most cases), and others. In addition to the well known motivations of the
risk-sensitive cost function (i.e., the exponential quadratic cost function),
which is most notably, the robustness to model uncertainty, we also view this
cost function as a tool for studying fundamental limits concerning the tail
behavior of the estimation error. Another interesting aspect, that we
demonstrate in a certain parametric model, is that the risk-sensitive cost
function may be subjected to phase transitions, owing to some analogies with
statistical mechanics.Comment: 28 pages; 4 figures; submitted for publicatio
PAC-Bayes and Domain Adaptation
We provide two main contributions in PAC-Bayesian theory for domain
adaptation where the objective is to learn, from a source distribution, a
well-performing majority vote on a different, but related, target distribution.
Firstly, we propose an improvement of the previous approach we proposed in
Germain et al. (2013), which relies on a novel distribution pseudodistance
based on a disagreement averaging, allowing us to derive a new tighter domain
adaptation bound for the target risk. While this bound stands in the spirit of
common domain adaptation works, we derive a second bound (introduced in Germain
et al., 2016) that brings a new perspective on domain adaptation by deriving an
upper bound on the target risk where the distributions' divergence-expressed as
a ratio-controls the trade-off between a source error measure and the target
voters' disagreement. We discuss and compare both results, from which we obtain
PAC-Bayesian generalization bounds. Furthermore, from the PAC-Bayesian
specialization to linear classifiers, we infer two learning algorithms, and we
evaluate them on real data.Comment: Neurocomputing, Elsevier, 2019. arXiv admin note: substantial text
overlap with arXiv:1503.0694
- …