35 research outputs found
Generating Sentences Using a Dynamic Canvas
We introduce the Attentive Unsupervised Text (W)riter (AUTR), which is a word
level generative model for natural language. It uses a recurrent neural network
with a dynamic attention and canvas memory mechanism to iteratively construct
sentences. By viewing the state of the memory at intermediate stages and where
the model is placing its attention, we gain insight into how it constructs
sentences. We demonstrate that AUTR learns a meaningful latent representation
for each sentence, and achieves competitive log-likelihood lower bounds whilst
being computationally efficient. It is effective at generating and
reconstructing sentences, as well as imputing missing words.Comment: AAAI 201
Generative Temporal Models with Spatial Memory for Partially Observed Environments
In model-based reinforcement learning, generative and temporal models of
environments can be leveraged to boost agent performance, either by tuning the
agent's representations during training or via use as part of an explicit
planning mechanism. However, their application in practice has been limited to
simplistic environments, due to the difficulty of training such models in
larger, potentially partially-observed and 3D environments. In this work we
introduce a novel action-conditioned generative model of such challenging
environments. The model features a non-parametric spatial memory system in
which we store learned, disentangled representations of the environment.
Low-dimensional spatial updates are computed using a state-space model that
makes use of knowledge on the prior dynamics of the moving agent, and
high-dimensional visual observations are modelled with a Variational
Auto-Encoder. The result is a scalable architecture capable of performing
coherent predictions over hundreds of time steps across a range of partially
observed 2D and 3D environments.Comment: ICML 201
Variational Autoencoders for New Physics Mining at the Large Hadron Collider
Using variational autoencoders trained on known physics processes, we develop
a one-sided threshold test to isolate previously unseen processes as outlier
events. Since the autoencoder training does not depend on any specific new
physics signature, the proposed procedure doesn't make specific assumptions on
the nature of new physics. An event selection based on this algorithm would be
complementary to classic LHC searches, typically based on model-dependent
hypothesis testing. Such an algorithm would deliver a list of anomalous events,
that the experimental collaborations could further scrutinize and even release
as a catalog, similarly to what is typically done in other scientific domains.
Event topologies repeating in this dataset could inspire new-physics model
building and new experimental searches. Running in the trigger system of the
LHC experiments, such an application could identify anomalous events that would
be otherwise lost, extending the scientific reach of the LHC.Comment: 29 pages, 12 figures, 5 table