7,053 research outputs found
Gradient-based Inference for Networks with Output Constraints
Practitioners apply neural networks to increasingly complex problems in
natural language processing, such as syntactic parsing and semantic role
labeling that have rich output structures. Many such structured-prediction
problems require deterministic constraints on the output values; for example,
in sequence-to-sequence syntactic parsing, we require that the sequential
outputs encode valid trees. While hidden units might capture such properties,
the network is not always able to learn such constraints from the training data
alone, and practitioners must then resort to post-processing. In this paper, we
present an inference method for neural networks that enforces deterministic
constraints on outputs without performing rule-based post-processing or
expensive discrete search. Instead, in the spirit of gradient-based training,
we enforce constraints with gradient-based inference (GBI): for each input at
test-time, we nudge continuous model weights until the network's unconstrained
inference procedure generates an output that satisfies the constraints. We
study the efficacy of GBI on three tasks with hard constraints: semantic role
labeling, syntactic parsing, and sequence transduction. In each case, the
algorithm not only satisfies constraints but improves accuracy, even when the
underlying network is state-of-the-art.Comment: AAAI 201
Graphene: Semantically-Linked Propositions in Open Information Extraction
We present an Open Information Extraction (IE) approach that uses a
two-layered transformation stage consisting of a clausal disembedding layer and
a phrasal disembedding layer, together with rhetorical relation identification.
In that way, we convert sentences that present a complex linguistic structure
into simplified, syntactically sound sentences, from which we can extract
propositions that are represented in a two-layered hierarchy in the form of
core relational tuples and accompanying contextual information which are
semantically linked via rhetorical relations. In a comparative evaluation, we
demonstrate that our reference implementation Graphene outperforms
state-of-the-art Open IE systems in the construction of correct n-ary
predicate-argument structures. Moreover, we show that existing Open IE
approaches can benefit from the transformation process of our framework.Comment: 27th International Conference on Computational Linguistics (COLING
2018
- …