88 research outputs found
Structured recognition for generative models with explaining away
A key goal of unsupervised learning is to go beyond density estimation and sample generation to reveal the structure inherent within observed data. Such structure can be expressed in the pattern of interactions between explanatory latent variables captured through a probabilistic graphical model. Although the learning of structured graphical models has a long history, much recent work in unsupervised modelling has instead emphasised flexible deep-network-based generation, either transforming independent latent generators to model complex data or assuming that distinct observed variables are derived from different latent nodes. Here, we extend amortised variational inference to incorporate structured factors over multiple variables, able to capture the observation-induced posterior dependence between latents that results from “explaining away” and thus allow complex observations to depend on multiple nodes of a structured graph. We show that appropriately parametrised factors can be combined efficiently with variational message passing in rich graphical structures. We instantiate the framework in nonlinear Gaussian Process Factor Analysis, evaluating the structured recognition framework using synthetic data from known generative processes. We fit the GPFA model to high-dimensional neural spike data from the hippocampus of freely moving rodents, where the model successfully identifies latent signals that correlate with behavioural covariates
End-to-end Sinkhorn Autoencoder with Noise Generator
In this work, we propose a novel end-to-end sinkhorn autoencoder with noise
generator for efficient data collection simulation. Simulating processes that
aim at collecting experimental data is crucial for multiple real-life
applications, including nuclear medicine, astronomy and high energy physics.
Contemporary methods, such as Monte Carlo algorithms, provide high-fidelity
results at a price of high computational cost. Multiple attempts are taken to
reduce this burden, e.g. using generative approaches based on Generative
Adversarial Networks or Variational Autoencoders. Although such methods are
much faster, they are often unstable in training and do not allow sampling from
an entire data distribution. To address these shortcomings, we introduce a
novel method dubbed end-to-end Sinkhorn Autoencoder, that leverages sinkhorn
algorithm to explicitly align distribution of encoded real data examples and
generated noise. More precisely, we extend autoencoder architecture by adding a
deterministic neural network trained to map noise from a known distribution
onto autoencoder latent space representing data distribution. We optimise the
entire model jointly. Our method outperforms competing approaches on a
challenging dataset of simulation data from Zero Degree Calorimeters of ALICE
experiment in LHC. as well as standard benchmarks, such as MNIST and CelebA
Topic Modelling Meets Deep Neural Networks: A Survey
Topic modelling has been a successful technique for text analysis for almost
twenty years. When topic modelling met deep neural networks, there emerged a
new and increasingly popular research area, neural topic models, with over a
hundred models developed and a wide range of applications in neural language
understanding such as text generation, summarisation and language models. There
is a need to summarise research developments and discuss open problems and
future directions. In this paper, we provide a focused yet comprehensive
overview of neural topic models for interested researchers in the AI community,
so as to facilitate them to navigate and innovate in this fast-growing research
area. To the best of our knowledge, ours is the first review focusing on this
specific topic.Comment: A review on Neural Topic Model
- …