182 research outputs found
Graph-to-Sequence Learning using Gated Graph Neural Networks
Many NLP applications can be framed as a graph-to-sequence learning problem.
Previous work proposing neural architectures on this setting obtained promising
results compared to grammar-based approaches but still rely on linearisation
heuristics and/or standard recurrent networks to achieve the best performance.
In this work, we propose a new model that encodes the full structural
information contained in the graph. Our architecture couples the recently
proposed Gated Graph Neural Networks with an input transformation that allows
nodes and edges to have their own hidden representations, while tackling the
parameter explosion problem present in previous work. Experimental results show
that our model outperforms strong baselines in generation from AMR graphs and
syntax-based neural machine translation.Comment: ACL 201
Graph Pre-training for AMR Parsing and Generation
Abstract meaning representation (AMR) highlights the core semantic
information of text in a graph structure. Recently, pre-trained language models
(PLMs) have advanced tasks of AMR parsing and AMR-to-text generation,
respectively. However, PLMs are typically pre-trained on textual data, thus are
sub-optimal for modeling structural knowledge. To this end, we investigate
graph self-supervised training to improve the structure awareness of PLMs over
AMR graphs. In particular, we introduce two graph auto-encoding strategies for
graph-to-graph pre-training and four tasks to integrate text and graph
information during pre-training. We further design a unified framework to
bridge the gap between pre-training and fine-tuning tasks. Experiments on both
AMR parsing and AMR-to-text generation show the superiority of our model. To
our knowledge, we are the first to consider pre-training on semantic graphs.Comment: ACL2022 camera-ready final versio
Guiding AMR Parsing with Reverse Graph Linearization
Abstract Meaning Representation (AMR) parsing aims to extract an abstract
semantic graph from a given sentence. The sequence-to-sequence approaches,
which linearize the semantic graph into a sequence of nodes and edges and
generate the linearized graph directly, have achieved good performance.
However, we observed that these approaches suffer from structure loss
accumulation during the decoding process, leading to a much lower F1-score for
nodes and edges decoded later compared to those decoded earlier. To address
this issue, we propose a novel Reverse Graph Linearization (RGL) enhanced
framework. RGL defines both default and reverse linearization orders of an AMR
graph, where most structures at the back part of the default order appear at
the front part of the reversed order and vice versa. RGL incorporates the
reversed linearization to the original AMR parser through a two-pass
self-distillation mechanism, which guides the model when generating the default
linearizations. Our analysis shows that our proposed method significantly
mitigates the problem of structure loss accumulation, outperforming the
previously best AMR parsing model by 0.8 and 0.5 Smatch scores on the AMR 2.0
and AMR 3.0 dataset, respectively. The code are available at
https://github.com/pkunlp-icler/AMR_reverse_graph_linearization.Comment: Findings of EMNLP202
- …