5,982 research outputs found
Joint Structure Learning of Multiple Non-Exchangeable Networks
Several methods have recently been developed for joint structure learning of
multiple (related) graphical models or networks. These methods treat individual
networks as exchangeable, such that each pair of networks are equally
encouraged to have similar structures. However, in many practical applications,
exchangeability in this sense may not hold, as some pairs of networks may be
more closely related than others, for example due to group and sub-group
structure in the data. Here we present a novel Bayesian formulation that
generalises joint structure learning beyond the exchangeable case. In addition
to a general framework for joint learning, we (i) provide a novel default prior
over the joint structure space that requires no user input; (ii) allow for
latent networks; (iii) give an efficient, exact algorithm for the case of time
series data and dynamic Bayesian networks. We present empirical results on
non-exchangeable populations, including a real data example from biology, where
cell-line-specific networks are related according to genomic features.Comment: To appear in Proceedings of the Seventeenth International Conference
on Artificial Intelligence and Statistics (AISTATS
Lightweight Probabilistic Deep Networks
Even though probabilistic treatments of neural networks have a long history,
they have not found widespread use in practice. Sampling approaches are often
too slow already for simple networks. The size of the inputs and the depth of
typical CNN architectures in computer vision only compound this problem.
Uncertainty in neural networks has thus been largely ignored in practice,
despite the fact that it may provide important information about the
reliability of predictions and the inner workings of the network. In this
paper, we introduce two lightweight approaches to making supervised learning
with probabilistic deep networks practical: First, we suggest probabilistic
output layers for classification and regression that require only minimal
changes to existing networks. Second, we employ assumed density filtering and
show that activation uncertainties can be propagated in a practical fashion
through the entire network, again with minor changes. Both probabilistic
networks retain the predictive power of the deterministic counterpart, but
yield uncertainties that correlate well with the empirical error induced by
their predictions. Moreover, the robustness to adversarial examples is
significantly increased.Comment: To appear at CVPR 201
Targeting Bayes factors with direct-path non-equilibrium thermodynamic integration
Thermodynamic integration (TI) for computing marginal likelihoods is based on an inverse annealing path from the prior to the posterior distribution. In many cases, the resulting estimator suffers from high variability, which particularly stems from the prior regime. When comparing complex models with differences in a comparatively small number of parameters, intrinsic errors from sampling fluctuations may outweigh the differences in the log marginal likelihood estimates. In the present article, we propose a thermodynamic integration scheme that directly targets the log Bayes factor. The method is based on a modified annealing path between the posterior distributions of the two models compared, which systematically avoids the high variance prior regime. We combine this scheme with the concept of non-equilibrium TI to minimise discretisation errors from numerical integration. Results obtained on Bayesian regression models applied to standard benchmark data, and a complex hierarchical model applied to biopathway inference, demonstrate a significant reduction in estimator variance over state-of-the-art TI methods
Probabilistic Logic Programming with Beta-Distributed Random Variables
We enable aProbLog---a probabilistic logical programming approach---to reason
in presence of uncertain probabilities represented as Beta-distributed random
variables. We achieve the same performance of state-of-the-art algorithms for
highly specified and engineered domains, while simultaneously we maintain the
flexibility offered by aProbLog in handling complex relational domains. Our
motivation is that faithfully capturing the distribution of probabilities is
necessary to compute an expected utility for effective decision making under
uncertainty: unfortunately, these probability distributions can be highly
uncertain due to sparse data. To understand and accurately manipulate such
probability distributions we need a well-defined theoretical framework that is
provided by the Beta distribution, which specifies a distribution of
probabilities representing all the possible values of a probability when the
exact value is unknown.Comment: Accepted for presentation at AAAI 201
- …