27,981 research outputs found
Probabilistic Graphical Models on Multi-Core CPUs using Java 8
In this paper, we discuss software design issues related to the development
of parallel computational intelligence algorithms on multi-core CPUs, using the
new Java 8 functional programming features. In particular, we focus on
probabilistic graphical models (PGMs) and present the parallelisation of a
collection of algorithms that deal with inference and learning of PGMs from
data. Namely, maximum likelihood estimation, importance sampling, and greedy
search for solving combinatorial optimisation problems. Through these concrete
examples, we tackle the problem of defining efficient data structures for PGMs
and parallel processing of same-size batches of data sets using Java 8
features. We also provide straightforward techniques to code parallel
algorithms that seamlessly exploit multi-core processors. The experimental
analysis, carried out using our open source AMIDST (Analysis of MassIve Data
STreams) Java toolbox, shows the merits of the proposed solutions.Comment: Pre-print version of the paper presented in the special issue on
Computational Intelligence Software at IEEE Computational Intelligence
Magazine journa
The Variational Homoencoder: Learning to learn high capacity generative models from few examples
Hierarchical Bayesian methods can unify many related tasks (e.g. k-shot
classification, conditional and unconditional generation) as inference within a
single generative model. However, when this generative model is expressed as a
powerful neural network such as a PixelCNN, we show that existing learning
techniques typically fail to effectively use latent variables. To address this,
we develop a modification of the Variational Autoencoder in which encoded
observations are decoded to new elements from the same class. This technique,
which we call a Variational Homoencoder (VHE), produces a hierarchical latent
variable model which better utilises latent variables. We use the VHE framework
to learn a hierarchical PixelCNN on the Omniglot dataset, which outperforms all
existing models on test set likelihood and achieves strong performance on
one-shot generation and classification tasks. We additionally validate the VHE
on natural images from the YouTube Faces database. Finally, we develop
extensions of the model that apply to richer dataset structures such as
factorial and hierarchical categories.Comment: UAI 2018 oral presentatio
Using Synthetic Data to Train Neural Networks is Model-Based Reasoning
We draw a formal connection between using synthetic training data to optimize
neural network parameters and approximate, Bayesian, model-based reasoning. In
particular, training a neural network using synthetic data can be viewed as
learning a proposal distribution generator for approximate inference in the
synthetic-data generative model. We demonstrate this connection in a
recognition task where we develop a novel Captcha-breaking architecture and
train it using synthetic data, demonstrating both state-of-the-art performance
and a way of computing task-specific posterior uncertainty. Using a neural
network trained this way, we also demonstrate successful breaking of real-world
Captchas currently used by Facebook and Wikipedia. Reasoning from these
empirical results and drawing connections with Bayesian modeling, we discuss
the robustness of synthetic data results and suggest important considerations
for ensuring good neural network generalization when training with synthetic
data.Comment: 8 pages, 4 figure
- …