327 research outputs found
Structural Neural Encoders for AMR-to-text Generation
AMR-to-text generation is a problem recently introduced to the NLP community,
in which the goal is to generate sentences from Abstract Meaning Representation
(AMR) graphs. Sequence-to-sequence models can be used to this end by converting
the AMR graphs to strings. Approaching the problem while working directly with
graphs requires the use of graph-to-sequence models that encode the AMR graph
into a vector representation. Such encoding has been shown to be beneficial in
the past, and unlike sequential encoding, it allows us to explicitly capture
reentrant structures in the AMR graphs. We investigate the extent to which
reentrancies (nodes with multiple parents) have an impact on AMR-to-text
generation by comparing graph encoders to tree encoders, where reentrancies are
not preserved. We show that improvements in the treatment of reentrancies and
long-range dependencies contribute to higher overall scores for graph encoders.
Our best model achieves 24.40 BLEU on LDC2015E86, outperforming the state of
the art by 1.1 points and 24.54 BLEU on LDC2017T10, outperforming the state of
the art by 1.24 points.Comment: Proceedings of NAACL 201
Graph-to-Sequence Learning using Gated Graph Neural Networks
Many NLP applications can be framed as a graph-to-sequence learning problem.
Previous work proposing neural architectures on this setting obtained promising
results compared to grammar-based approaches but still rely on linearisation
heuristics and/or standard recurrent networks to achieve the best performance.
In this work, we propose a new model that encodes the full structural
information contained in the graph. Our architecture couples the recently
proposed Gated Graph Neural Networks with an input transformation that allows
nodes and edges to have their own hidden representations, while tackling the
parameter explosion problem present in previous work. Experimental results show
that our model outperforms strong baselines in generation from AMR graphs and
syntax-based neural machine translation.Comment: ACL 201
Modeling Global and Local Node Contexts for Text Generation from Knowledge Graphs
Recent graph-to-text models generate text from graph-based data using either
global or local aggregation to learn node representations. Global node encoding
allows explicit communication between two distant nodes, thereby neglecting
graph topology as all nodes are directly connected. In contrast, local node
encoding considers the relations between neighbor nodes capturing the graph
structure, but it can fail to capture long-range relations. In this work, we
gather both encoding strategies, proposing novel neural models which encode an
input graph combining both global and local node contexts, in order to learn
better contextualized node embeddings. In our experiments, we demonstrate that
our approaches lead to significant improvements on two graph-to-text datasets
achieving BLEU scores of 18.01 on AGENDA dataset, and 63.69 on the WebNLG
dataset for seen categories, outperforming state-of-the-art models by 3.7 and
3.1 points, respectively.Comment: Accepted for publication in Transactions of the Association for
Computational Linguistics (TACL), 2020; Author's final version; pre-MIT Press
publication versio
Comparing Neural Meaning-to-Text Approaches for Dutch
The neural turn in computational linguistics has made it relatively easy to build systems for natural language generation, as long as suitable annotated corpora are available. But can such systems deliver the goods? Using Dutch data of the Parallel Meaning Bank, a corpus of (mostly short) texts annotated with language-neutral meaning representations, we investigate what challenges arise and what choices can be made when implementing sequence-to-sequence or graphto- sequence transformer models for generating Dutch texts from formal meaning representations. We compare the performance of linearized input graphs with graphs encoded in various formats and find that stacking encoders obtain the best results for the standard metrics used in natural language generation. A key challenge is dealing with unknown tokens that occur in the input meaning representation. We introduce a new method based on WordNet similarity to deal with out-of-vocab concepts
Investigating Pretrained Language Models for Graph-to-Text Generation
Graph-to-text generation aims to generate fluent texts from graph-based data.
In this paper, we investigate two recently proposed pretrained language models
(PLMs) and analyze the impact of different task-adaptive pretraining strategies
for PLMs in graph-to-text generation. We present a study across three graph
domains: meaning representations, Wikipedia knowledge graphs (KGs) and
scientific KGs. We show that the PLMs BART and T5 achieve new state-of-the-art
results and that task-adaptive pretraining strategies improve their performance
even further. In particular, we report new state-of-the-art BLEU scores of
49.72 on LDC2017T10, 59.70 on WebNLG, and 25.66 on AGENDA datasets - a relative
improvement of 31.8%, 4.5%, and 42.4%, respectively. In an extensive analysis,
we identify possible reasons for the PLMs' success on graph-to-text tasks. We
find evidence that their knowledge about true facts helps them perform well
even when the input graph representation is reduced to a simple bag of node and
edge labels.Comment: Our code and pretrained model checkpoints are available at
https://github.com/UKPLab/plms-graph2tex
- …