65 research outputs found
Event Representations with Tensor-based Compositions
Robust and flexible event representations are important to many core areas in
language understanding. Scripts were proposed early on as a way of representing
sequences of events for such understanding, and has recently attracted renewed
attention. However, obtaining effective representations for modeling
script-like event sequences is challenging. It requires representations that
can capture event-level and scenario-level semantics. We propose a new
tensor-based composition method for creating event representations. The method
captures more subtle semantic interactions between an event and its entities
and yields representations that are effective at multiple event-related tasks.
With the continuous representations, we also devise a simple schema generation
method which produces better schemas compared to a prior discrete
representation based method. Our analysis shows that the tensors capture
distinct usages of a predicate even when there are only subtle differences in
their surface realizations.Comment: Accepted at AAAI 201
Hierarchical Quantized Representations for Script Generation
Scripts define knowledge about how everyday scenarios (such as going to a
restaurant) are expected to unfold. One of the challenges to learning scripts
is the hierarchical nature of the knowledge. For example, a suspect arrested
might plead innocent or guilty, and a very different track of events is then
expected to happen. To capture this type of information, we propose an
autoencoder model with a latent space defined by a hierarchy of categorical
variables. We utilize a recently proposed vector quantization based approach,
which allows continuous embeddings to be associated with each latent variable
value. This permits the decoder to softly decide what portions of the latent
hierarchy to condition on by attending over the value embeddings for a given
setting. Our model effectively encodes and generates scripts, outperforming a
recent language modeling-based method on several standard tasks, and allowing
the autoencoder model to achieve substantially lower perplexity scores compared
to the previous language modeling-based method.Comment: EMNLP 201
- …