8,547 research outputs found
Jointly Learning to Label Sentences and Tokens
Learning to construct text representations in end-to-end systems can be difficult, as natural languages are highly compositional and task-specific annotated datasets are often limited in size.
Methods for directly supervising language composition can allow us to guide the models based on existing knowledge, regularizing them towards more robust and interpretable representations.
In this paper, we investigate how objectives at different granularities can be used to learn better language representations and we propose an architecture for jointly learning to label sentences and tokens.
The predictions at each level are combined together using an attention mechanism, with token-level labels also acting as explicit supervision for composing sentence-level representations.
Our experiments show that by learning to perform these tasks jointly on multiple levels, the model achieves substantial improvements for both sentence classification and sequence labeling
Recommended from our members
Jointly Learning to Label Sentences and Tokens
Learning to construct text representations in end-to-end systems can be difficult, as natural languages are highly compositional and task-specific annotated datasets are often limited in size.
Methods for directly supervising language composition can allow us to guide the models based on existing knowledge, regularizing them towards more robust and interpretable representations.
In this paper, we investigate how objectives at different granularities can be used to learn better language representations and we propose an architecture for jointly learning to label sentences and tokens.
The predictions at each level are combined together using an attention mechanism, with token-level labels also acting as explicit supervision for composing sentence-level representations.
Our experiments show that by learning to perform these tasks jointly on multiple levels, the model achieves substantial improvements for both sentence classification and sequence labeling
Jointly Modeling Topics and Intents with Global Order Structure
Modeling document structure is of great importance for discourse analysis and
related applications. The goal of this research is to capture the document
intent structure by modeling documents as a mixture of topic words and
rhetorical words. While the topics are relatively unchanged through one
document, the rhetorical functions of sentences usually change following
certain orders in discourse. We propose GMM-LDA, a topic modeling based
Bayesian unsupervised model, to analyze the document intent structure
cooperated with order information. Our model is flexible that has the ability
to combine the annotations and do supervised learning. Additionally, entropic
regularization can be introduced to model the significant divergence between
topics and intents. We perform experiments in both unsupervised and supervised
settings, results show the superiority of our model over several
state-of-the-art baselines.Comment: Accepted by AAAI 201
Syntactically Look-Ahead Attention Network for Sentence Compression
Sentence compression is the task of compressing a long sentence into a short
one by deleting redundant words. In sequence-to-sequence (Seq2Seq) based
models, the decoder unidirectionally decides to retain or delete words. Thus,
it cannot usually explicitly capture the relationships between decoded words
and unseen words that will be decoded in the future time steps. Therefore, to
avoid generating ungrammatical sentences, the decoder sometimes drops important
words in compressing sentences. To solve this problem, we propose a novel
Seq2Seq model, syntactically look-ahead attention network (SLAHAN), that can
generate informative summaries by explicitly tracking both dependency parent
and child words during decoding and capturing important words that will be
decoded in the future. The results of the automatic evaluation on the Google
sentence compression dataset showed that SLAHAN achieved the best
kept-token-based-F1, ROUGE-1, ROUGE-2 and ROUGE-L scores of 85.5, 79.3, 71.3
and 79.1, respectively. SLAHAN also improved the summarization performance on
longer sentences. Furthermore, in the human evaluation, SLAHAN improved
informativeness without losing readability.Comment: AAAI 202
- …