14 research outputs found
Bipartite Flat-Graph Network for Nested Named Entity Recognition
In this paper, we propose a novel bipartite flat-graph network (BiFlaG) for
nested named entity recognition (NER), which contains two subgraph modules: a
flat NER module for outermost entities and a graph module for all the entities
located in inner layers. Bidirectional LSTM (BiLSTM) and graph convolutional
network (GCN) are adopted to jointly learn flat entities and their inner
dependencies. Different from previous models, which only consider the
unidirectional delivery of information from innermost layers to outer ones (or
outside-to-inside), our model effectively captures the bidirectional
interaction between them. We first use the entities recognized by the flat NER
module to construct an entity graph, which is fed to the next graph module. The
richer representation learned from graph module carries the dependencies of
inner entities and can be exploited to improve outermost entity predictions.
Experimental results on three standard nested NER datasets demonstrate that our
BiFlaG outperforms previous state-of-the-art models.Comment: Accepted by ACL202
Cross-Lingual Semantic Role Labeling with High-Quality Translated Training Corpus
Many efforts of research are devoted to semantic role labeling (SRL) which is
crucial for natural language understanding. Supervised approaches have achieved
impressing performances when large-scale corpora are available for
resource-rich languages such as English. While for the low-resource languages
with no annotated SRL dataset, it is still challenging to obtain competitive
performances. Cross-lingual SRL is one promising way to address the problem,
which has achieved great advances with the help of model transferring and
annotation projection. In this paper, we propose a novel alternative based on
corpus translation, constructing high-quality training datasets for the target
languages from the source gold-standard SRL annotations. Experimental results
on Universal Proposition Bank show that the translation-based method is highly
effective, and the automatic pseudo datasets can improve the target-language
SRL performances significantly.Comment: Accepted at ACL 202
InVeRo: Making Semantic Role Labeling Accessible with Intelligible Verbs and Roles
Semantic Role Labeling (SRL) is deeply dependent on complex linguistic resources and sophisticated neural models, which makes the task difficult to approach for non-experts. To address this issue we present a new platform named Intelligible Verbs and Roles (InVeRo). This platform provides access to a new verb resource, VerbAtlas, and a state-of-the-art pre-trained implementation of a neural, span-based architecture for SRL. Both the resource and the system provide human-readable verb sense and semantic role information, with an easy to use Web interface and RESTful APIs available at http://nlp.uniroma1.it/invero
A Tour of Explicit Multilingual Semantics: Word Sense Disambiguation, Semantic Role Labeling and Semantic Parsing
The recent advent of modern pretrained language models has sparked a revolution in Natural Language Processing (NLP), especially in multilingual and cross-lingual applications. Today, such language models have become the de facto standard for providing rich input representations to neural systems, achieving unprecedented results in an increasing range of benchmarks. However, questions that often arise are: firstly, whether current language models are, indeed, able to capture explicit, symbolic meaning; secondly, if they are, to what extent; thirdly, and perhaps more importantly, whether current approaches are capable of scaling across languages. In this cutting-edge tutorial, we will review recent efforts that have aimed at shedding light on meaning in NLP, with a focus on three key open problems in lexical and sentence-level semantics: Word Sense Disambiguation, Semantic Role Labeling, and Semantic Parsing. After a brief introduction, we will spotlight how state-of-the-art models tackle these tasks in multiple languages, showing where they excel and where they fail. We hope that this tutorial will broaden the audience interested in multilingual semantics and inspire researchers to further advance the field
SG-Net: Syntax-Guided Machine Reading Comprehension
For machine reading comprehension, the capacity of effectively modeling the
linguistic knowledge from the detail-riddled and lengthy passages and getting
ride of the noises is essential to improve its performance. Traditional
attentive models attend to all words without explicit constraint, which results
in inaccurate concentration on some dispensable words. In this work, we propose
using syntax to guide the text modeling by incorporating explicit syntactic
constraints into attention mechanism for better linguistically motivated word
representations. In detail, for self-attention network (SAN) sponsored
Transformer-based encoder, we introduce syntactic dependency of interest (SDOI)
design into the SAN to form an SDOI-SAN with syntax-guided self-attention.
Syntax-guided network (SG-Net) is then composed of this extra SDOI-SAN and the
SAN from the original Transformer encoder through a dual contextual
architecture for better linguistics inspired representation. To verify its
effectiveness, the proposed SG-Net is applied to typical pre-trained language
model BERT which is right based on a Transformer encoder. Extensive experiments
on popular benchmarks including SQuAD 2.0 and RACE show that the proposed
SG-Net design helps achieve substantial performance improvement over strong
baselines.Comment: Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-2020
Syntax-Aware Graph-to-Graph Transformer for Semantic Role Labelling
Recent models have shown that incorporating syntactic knowledge into the
semantic role labelling (SRL) task leads to a significant improvement. In this
paper, we propose Syntax-aware Graph-to-Graph Transformer (SynG2G-Tr) model,
which encodes the syntactic structure using a novel way to input graph
relations as embeddings, directly into the self-attention mechanism of
Transformer. This approach adds a soft bias towards attention patterns that
follow the syntactic structure but also allows the model to use this
information to learn alternative patterns. We evaluate our model on both
span-based and dependency-based SRL datasets, and outperform previous
alternative methods in both in-domain and out-of-domain settings, on CoNLL 2005
and CoNLL 2009 datasets.Comment: Accepted to Rep4NLP at ACL 202
Hierarchical Contextualized Representation for Named Entity Recognition
Named entity recognition (NER) models are typically based on the architecture
of Bi-directional LSTM (BiLSTM). The constraints of sequential nature and the
modeling of single input prevent the full utilization of global information
from larger scope, not only in the entire sentence, but also in the entire
document (dataset). In this paper, we address these two deficiencies and
propose a model augmented with hierarchical contextualized representation:
sentence-level representation and document-level representation. In
sentence-level, we take different contributions of words in a single sentence
into consideration to enhance the sentence representation learned from an
independent BiLSTM via label embedding attention mechanism. In document-level,
the key-value memory network is adopted to record the document-aware
information for each unique word which is sensitive to similarity of context
information. Our two-level hierarchical contextualized representations are
fused with each input token embedding and corresponding hidden state of BiLSTM,
respectively. The experimental results on three benchmark NER datasets
(CoNLL-2003 and Ontonotes 5.0 English datasets, CoNLL-2002 Spanish dataset)
show that we establish new state-of-the-art results.Comment: Accepted by AAAI 202
Bridging the Gap in Multilingual Semantic Role Labeling: A Language-Agnostic Approach
Recent research indicates that taking advantage of complex syntactic features leads to favorable results in Semantic Role Labeling. Nonetheless, an analysis of the latest state-of-the-art multilingual systems reveals the difficulty of bridging the wide gap in performance between high-resource (e.g., English) and low-resource (e.g., German) settings. To overcome this issue, we propose a fully language-agnostic model that does away with morphological and syntactic features to achieve robustness across languages. Our approach outperforms the state of the art in all the languages of the CoNLL-2009 benchmark dataset, especially whenever a scarce amount of training data is available. Our objective is not to reject approaches that rely on syntax, rather to set a strong and consistent language-independent baseline for future innovations in Semantic Role Labeling. We release our model code and checkpoints at https://github.com/SapienzaNLP/multi-srl