956 research outputs found
Compositional Distributional Semantics with Long Short Term Memory
We are proposing an extension of the recursive neural network that makes use
of a variant of the long short-term memory architecture. The extension allows
information low in parse trees to be stored in a memory register (the `memory
cell') and used much later higher up in the parse tree. This provides a
solution to the vanishing gradient problem and allows the network to capture
long range dependencies. Experimental results show that our composition
outperformed the traditional neural-network composition on the Stanford
Sentiment Treebank.Comment: 10 pages, 7 figure
Dynamic Compositional Neural Networks over Tree Structure
Tree-structured neural networks have proven to be effective in learning
semantic representations by exploiting syntactic information. In spite of their
success, most existing models suffer from the underfitting problem: they
recursively use the same shared compositional function throughout the whole
compositional process and lack expressive power due to inability to capture the
richness of compositionality. In this paper, we address this issue by
introducing the dynamic compositional neural networks over tree structure
(DC-TreeNN), in which the compositional function is dynamically generated by a
meta network. The role of meta-network is to capture the metaknowledge across
the different compositional rules and formulate them. Experimental results on
two typical tasks show the effectiveness of the proposed models.Comment: Accepted by IJCAI 201
MS-TR: A Morphologically Enriched Sentiment Treebank and Recursive Deep Models for Compositional Semantics in Turkish
Recursive Deep Models have been used as powerful models to learn
compositional representations of text for many natural language processing tasks.
However, they require structured input (i.e. sentiment treebank) to encode sentences
based on their tree-based structure to enable them to learn latent semantics
of words using recursive composition functions. In this paper, we present our
contributions and efforts for the Turkish Sentiment Treebank construction. We
introduce MS-TR, a Morphologically Enriched Sentiment Treebank, which was
implemented for training Recursive Deep Models to address compositional sentiment
analysis for Turkish, which is one of the well-known Morphologically Rich
Language (MRL). We propose a semi-supervised automatic annotation, as a distantsupervision
approach, using morphological features of words to infer the polarity of
the inner nodes of MS-TR as positive and negative. The proposed annotation model
has four different annotation levels: morph-level, stem-level, token-level, and
review-level. Each annotation level’s contribution was tested using three different
domain datasets, including product reviews, movie reviews, and the Turkish Natural
Corpus essays. Comparative results were obtained with the Recursive Neural Tensor Networks (RNTN) model which is operated over MS-TR, and conventional machine learning methods. Experiments proved that RNTN outperformed the baseline methods and achieved much better accuracy results compared to the baseline methods, which cannot accurately capture the aggregated sentiment information
- …