25 research outputs found
When is multitask learning effective? Semantic sequence prediction under varying data conditions
Multitask learning has been applied successfully to a range of tasks, mostly
morphosyntactic. However, little is known on when MTL works and whether there
are data characteristics that help to determine its success. In this paper we
evaluate a range of semantic sequence labeling tasks in a MTL setup. We examine
different auxiliary tasks, amongst which a novel setup, and correlate their
impact to data-dependent conditions. Our results show that MTL is not always
effective, significant improvements are obtained only for 1 out of 5 tasks.
When successful, auxiliary tasks with compact and more uniform label
distributions are preferable.Comment: In EACL 201
When is multitask learning effective? Semantic sequence prediction under varying data conditions
International audienceMultitask learning has been applied successfully to a range of tasks, mostly mor-phosyntactic. However, little is known on when MTL works and whether there are data characteristics that help to determine its success. In this paper we evaluate a range of semantic sequence labeling tasks in a MTL setup. We examine different auxiliary tasks, amongst which a novel setup, and correlate their impact to data-dependent conditions. Our results show that MTL is not always effective, significant improvements are obtained only for 1 out of 5 tasks. When successful, auxiliary tasks with compact and more uniform label distributions are preferable
Parsing Thai Social Data: A New Challenge for Thai NLP
Dependency parsing (DP) is a task that analyzes text for syntactic structure
and relationship between words. DP is widely used to improve natural language
processing (NLP) applications in many languages such as English. Previous works
on DP are generally applicable to formally written languages. However, they do
not apply to informal languages such as the ones used in social networks.
Therefore, DP has to be researched and explored with such social network data.
In this paper, we explore and identify a DP model that is suitable for Thai
social network data. After that, we will identify the appropriate linguistic
unit as an input. The result showed that, the transition based model called,
improve Elkared dependency parser outperform the others at UAS of 81.42%.Comment: 7 Pages, 8 figures, to be published in The 14th International Joint
Symposium on Artificial Intelligence and Natural Language Processing
(iSAI-NLP 2019
Gated Task Interaction Framework for Multi-task Sequence Tagging
Recent studies have shown that neural models can achieve high performance on
several sequence labelling/tagging problems without the explicit use of
linguistic features such as part-of-speech (POS) tags. These models are trained
only using the character-level and the word embedding vectors as inputs. Others
have shown that linguistic features can improve the performance of neural
models on tasks such as chunking and named entity recognition (NER). However,
the change in performance depends on the degree of semantic relatedness between
the linguistic features and the target task; in some instances, linguistic
features can have a negative impact on performance. This paper presents an
approach to jointly learn these linguistic features along with the target
sequence labelling tasks with a new multi-task learning (MTL) framework called
Gated Tasks Interaction (GTI) network for solving multiple sequence tagging
tasks. The GTI network exploits the relations between the multiple tasks via
neural gate modules. These gate modules control the flow of information between
the different tasks. Experiments on benchmark datasets for chunking and NER
show that our framework outperforms other competitive baselines trained with
and without external training resources.Comment: 8 page
Neural Multi-Task Learning for Citation Function and Provenance
Citation function and provenance are two cornerstone tasks in citation
analysis. Given a citation, the former task determines its rhetorical role,
while the latter locates the text in the cited paper that contains the relevant
cited information. We hypothesize that these two tasks are synergistically
related, and build a model that validates this claim. For both tasks, we show
that a single-layer convolutional neural network (CNN) outperforms existing
state-of-the-art baselines. More importantly, we show that the two tasks are
indeed synergistic: by jointly training both of the tasks in a multi-task
learning setup, we demonstrate additional performance gains. Altogether, our
models improve the current state-of-the-arts up to 2\%, with statistical
significance for both citation function and provenance prediction tasks