17 research outputs found
How to Train Your Agent to Read and Write
Reading and writing research papers is one of the most privileged abilities
that a qualified researcher should master. However, it is difficult for new
researchers (\eg{students}) to fully {grasp} this ability. It would be
fascinating if we could train an intelligent agent to help people read and
summarize papers, and perhaps even discover and exploit the potential knowledge
clues to write novel papers. Although there have been existing works focusing
on summarizing (\emph{i.e.}, reading) the knowledge in a given text or
generating (\emph{i.e.}, writing) a text based on the given knowledge, the
ability of simultaneously reading and writing is still under development.
Typically, this requires an agent to fully understand the knowledge from the
given text materials and generate correct and fluent novel paragraphs, which is
very challenging in practice. In this paper, we propose a Deep ReAder-Writer
(DRAW) network, which consists of a \textit{Reader} that can extract knowledge
graphs (KGs) from input paragraphs and discover potential knowledge, a
graph-to-text \textit{Writer} that generates a novel paragraph, and a
\textit{Reviewer} that reviews the generated paragraph from three different
aspects. Extensive experiments show that our DRAW network outperforms
considered baselines and several state-of-the-art methods on AGENDA and
M-AGENDA datasets. Our code and supplementary are released at
https://github.com/menggehe/DRAW
Modeling Graph Structure via Relative Position for Text Generation from Knowledge Graphs
We present Graformer, a novel Transformer-based encoder-decoder architecture
for graph-to-text generation. With our novel graph self-attention, the encoding
of a node relies on all nodes in the input graph - not only direct neighbors -
facilitating the detection of global patterns. We represent the relation
between two nodes as the length of the shortest path between them. Graformer
learns to weight these node-node relations differently for different attention
heads, thus virtually learning differently connected views of the input graph.
We evaluate Graformer on two popular graph-to-text generation benchmarks,
AGENDA and WebNLG, where it achieves strong performance while using many fewer
parameters than other approaches
Graph Transformer for Graph-to-Sequence Learning
The dominant graph-to-sequence transduction models employ graph neural
networks for graph representation learning, where the structural information is
reflected by the receptive field of neurons. Unlike graph neural networks that
restrict the information exchange between immediate neighborhood, we propose a
new model, known as Graph Transformer, that uses explicit relation encoding and
allows direct communication between two distant nodes. It provides a more
efficient way for global graph structure modeling. Experiments on the
applications of text generation from Abstract Meaning Representation (AMR) and
syntax-based neural machine translation show the superiority of our proposed
model. Specifically, our model achieves 27.4 BLEU on LDC2015E86 and 29.7 BLEU
on LDC2017T10 for AMR-to-text generation, outperforming the state-of-the-art
results by up to 2.2 points. On the syntax-based translation tasks, our model
establishes new single-model state-of-the-art BLEU scores, 21.3 for
English-to-German and 14.1 for English-to-Czech, improving over the existing
best results, including ensembles, by over 1 BLEU.Comment: accepted by AAAI202