1,749 research outputs found
A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations
Matching natural language sentences is central for many applications such as
information retrieval and question answering. Existing deep models rely on a
single sentence representation or multiple granularity representations for
matching. However, such methods cannot well capture the contextualized local
information in the matching process. To tackle this problem, we present a new
deep architecture to match two sentences with multiple positional sentence
representations. Specifically, each positional sentence representation is a
sentence representation at this position, generated by a bidirectional long
short term memory (Bi-LSTM). The matching score is finally produced by
aggregating interactions between these different positional sentence
representations, through -Max pooling and a multi-layer perceptron. Our
model has several advantages: (1) By using Bi-LSTM, rich context of the whole
sentence is leveraged to capture the contextualized local information in each
positional sentence representation; (2) By matching with multiple positional
sentence representations, it is flexible to aggregate different important
contextualized local information in a sentence to support the matching; (3)
Experiments on different tasks such as question answering and sentence
completion demonstrate the superiority of our model.Comment: Accepted by AAAI-201
Recurrent Memory Networks for Language Modeling
Recurrent Neural Networks (RNN) have obtained excellent result in many
natural language processing (NLP) tasks. However, understanding and
interpreting the source of this success remains a challenge. In this paper, we
propose Recurrent Memory Network (RMN), a novel RNN architecture, that not only
amplifies the power of RNN but also facilitates our understanding of its
internal functioning and allows us to discover underlying patterns in data. We
demonstrate the power of RMN on language modeling and sentence completion
tasks. On language modeling, RMN outperforms Long Short-Term Memory (LSTM)
network on three large German, Italian, and English dataset. Additionally we
perform in-depth analysis of various linguistic dimensions that RMN captures.
On Sentence Completion Challenge, for which it is essential to capture sentence
coherence, our RMN obtains 69.2% accuracy, surpassing the previous
state-of-the-art by a large margin.Comment: 8 pages, 6 figures. Accepted at NAACL 201
Learning Word Representations with Hierarchical Sparse Coding
We propose a new method for learning word representations using hierarchical
regularization in sparse coding inspired by the linguistic study of word
meanings. We show an efficient learning algorithm based on stochastic proximal
methods that is significantly faster than previous approaches, making it
possible to perform hierarchical sparse coding on a corpus of billions of word
tokens. Experiments on various benchmark tasks---word similarity ranking,
analogies, sentence completion, and sentiment analysis---demonstrate that the
method outperforms or is competitive with state-of-the-art methods. Our word
representations are available at
\url{http://www.ark.cs.cmu.edu/dyogatam/wordvecs/}
- …