78,569 research outputs found
Entropy and inference, revisited
We study properties of popular near-uniform (Dirichlet) priors for learning
undersampled probability distributions on discrete nonmetric spaces and show
that they lead to disastrous results. However, an Occam-style phase space
argument expands the priors into their infinite mixture and resolves most of
the observed problems. This leads to a surprisingly good estimator of entropies
of discrete distributions.Comment: LaTex2e, 9 pages, 5 figures; references added, minor revisions
introduced, formatting errors correcte
Bayesian Methods for Exoplanet Science
Exoplanet research is carried out at the limits of the capabilities of
current telescopes and instruments. The studied signals are weak, and often
embedded in complex systematics from instrumental, telluric, and astrophysical
sources. Combining repeated observations of periodic events, simultaneous
observations with multiple telescopes, different observation techniques, and
existing information from theory and prior research can help to disentangle the
systematics from the planetary signals, and offers synergistic advantages over
analysing observations separately. Bayesian inference provides a
self-consistent statistical framework that addresses both the necessity for
complex systematics models, and the need to combine prior information and
heterogeneous observations. This chapter offers a brief introduction to
Bayesian inference in the context of exoplanet research, with focus on time
series analysis, and finishes with an overview of a set of freely available
programming libraries.Comment: Invited revie
A Hierarchical Bayesian Trust Model based on Reputation and Group Behaviour
In many systems, agents must rely on their peers to achieve their goals. However, when trusted to perform an action, an agent may betray that trust by not behaving as required. Agents must therefore estimate the behaviour of their peers, so that they may identify reliable interaction partners. To this end, we present a Bayesian trust model (HABIT) for assessing trust based on direct experience and (potentially unreliable) reputation. Although existing approaches claim to achieve this, most rely on heuristics with little theoretical foundation. In contrast, HABIT is based on principled statistical techniques; can be used with any representation of behaviour; and can assess trust based on observed similarities between groups of agents. In this paper, we describe the theoretical aspects of the model and present experimental results in which HABIT was shown to be up to twice as accurate at predicting trustee performance as an existing state-of-the-art trust model
Threshold Choice Methods: the Missing Link
Many performance metrics have been introduced for the evaluation of
classification performance, with different origins and niches of application:
accuracy, macro-accuracy, area under the ROC curve, the ROC convex hull, the
absolute error, and the Brier score (with its decomposition into refinement and
calibration). One way of understanding the relation among some of these metrics
is the use of variable operating conditions (either in the form of
misclassification costs or class proportions). Thus, a metric may correspond to
some expected loss over a range of operating conditions. One dimension for the
analysis has been precisely the distribution we take for this range of
operating conditions, leading to some important connections in the area of
proper scoring rules. However, we show that there is another dimension which
has not received attention in the analysis of performance metrics. This new
dimension is given by the decision rule, which is typically implemented as a
threshold choice method when using scoring models. In this paper, we explore
many old and new threshold choice methods: fixed, score-uniform, score-driven,
rate-driven and optimal, among others. By calculating the loss of these methods
for a uniform range of operating conditions we get the 0-1 loss, the absolute
error, the Brier score (mean squared error), the AUC and the refinement loss
respectively. This provides a comprehensive view of performance metrics as well
as a systematic approach to loss minimisation, namely: take a model, apply
several threshold choice methods consistent with the information which is (and
will be) available about the operating condition, and compare their expected
losses. In order to assist in this procedure we also derive several connections
between the aforementioned performance metrics, and we highlight the role of
calibration in choosing the threshold choice method
Entropic Inference: some pitfalls and paradoxes we can avoid
The method of maximum entropy has been very successful but there are cases
where it has either failed or led to paradoxes that have cast doubt on its
general legitimacy. My more optimistic assessment is that such failures and
paradoxes provide us with valuable learning opportunities to sharpen our skills
in the proper way to deploy entropic methods. The central theme of this paper
revolves around the different ways in which constraints are used to capture the
information that is relevant to a problem. This leads us to focus on four
epistemically different types of constraints. I propose that the failure to
recognize the distinctions between them is a prime source of errors. I
explicitly discuss two examples. One concerns the dangers involved in replacing
expected values with sample averages. The other revolves around
misunderstanding ignorance. I discuss the Friedman-Shimony paradox as it is
manifested in the three-sided die problem and also in its original
thermodynamic formulation.Comment: 14 pages, 1 figure. Invited paper presented at MaxEnt 2012, The 32nd
International Workshop on Bayesian Inference and Maximum Entropy Methods in
Science and Engineering, (July 15--20, 2012, Garching, Germany
No Free Lunch for Noise Prediction
No-free-lunch theorems have shown that learning algorithms cannot be universally good. We show that no free funch exists for noise prediction as well. We show that when the noise is additive and the prior over target functions is uniform, a prior on the noise distribution cannot be updated, in the Bayesian sense, from any finite data set. We emphasize the importance of a prior over the target function in order to justify superior performance for learning systems
Entropy and information in neural spike trains: Progress on the sampling problem
The major problem in information theoretic analysis of neural responses and
other biological data is the reliable estimation of entropy--like quantities
from small samples. We apply a recently introduced Bayesian entropy estimator
to synthetic data inspired by experiments, and to real experimental spike
trains. The estimator performs admirably even very deep in the undersampled
regime, where other techniques fail. This opens new possibilities for the
information theoretic analysis of experiments, and may be of general interest
as an example of learning from limited data.Comment: 7 pages, 4 figures; referee suggested changes, accepted versio
- …