27 research outputs found
Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling
Semantic role labeling (SRL) is the task of identifying the
predicate-argument structure of a sentence. It is typically regarded as an
important step in the standard NLP pipeline. As the semantic representations
are closely related to syntactic ones, we exploit syntactic information in our
model. We propose a version of graph convolutional networks (GCNs), a recent
class of neural networks operating on graphs, suited to model syntactic
dependency graphs. GCNs over syntactic dependency trees are used as sentence
encoders, producing latent feature representations of words in a sentence. We
observe that GCN layers are complementary to LSTM ones: when we stack both GCN
and LSTM layers, we obtain a substantial improvement over an already
state-of-the-art LSTM SRL model, resulting in the best reported scores on the
standard benchmark (CoNLL-2009) both for Chinese and English.Comment: To appear in EMNLP 201
A Simple and Accurate Syntax-Agnostic Neural Model for Dependency-based Semantic Role Labeling
We introduce a simple and accurate neural model for dependency-based semantic
role labeling. Our model predicts predicate-argument dependencies relying on
states of a bidirectional LSTM encoder. The semantic role labeler achieves
competitive performance on English, even without any kind of syntactic
information and only using local inference. However, when automatically
predicted part-of-speech tags are provided as input, it substantially
outperforms all previous local models and approaches the best reported results
on the English CoNLL-2009 dataset. We also consider Chinese, Czech and Spanish
where our approach also achieves competitive results. Syntactic parsers are
unreliable on out-of-domain data, so standard (i.e., syntactically-informed)
SRL models are hindered when tested in this setting. Our syntax-agnostic model
appears more robust, resulting in the best reported results on standard
out-of-domain test sets.Comment: To appear in CoNLL 201
Bias Beyond English: Counterfactual Tests for Bias in Sentiment Analysis in Four Languages
Sentiment analysis (SA) systems are used in many products and hundreds of languages. Gender and racial biases are well-studied in English SA systems, but understudied in other languages, with few resources for such studies. To remedy this, we build a counterfactual evaluation corpus for gender and racial/migrant bias in four languages. We demonstrate its usefulness by answering a simple but important question that an engineer might need to answer when deploying a system: What biases do systems import from pre-trained models when compared to a baseline with no pre-training? Our evaluation corpus, by virtue of being counterfactual, not only reveals which models have less bias, but also pinpoints changes in model bias behaviour, which enables more targeted mitigation strategies. We release our code and evaluation corpora to facilitate future research