1,565 research outputs found
Sentiment analysis based on rhetorical structure theory: Learning deep neural networks from discourse trees
Prominent applications of sentiment analysis are countless, covering areas
such as marketing, customer service and communication. The conventional
bag-of-words approach for measuring sentiment merely counts term frequencies;
however, it neglects the position of the terms within the discourse. As a
remedy, we develop a discourse-aware method that builds upon the discourse
structure of documents. For this purpose, we utilize rhetorical structure
theory to label (sub-)clauses according to their hierarchical relationships and
then assign polarity scores to individual leaves. To learn from the resulting
rhetorical structure, we propose a tensor-based, tree-structured deep neural
network (named Discourse-LSTM) in order to process the complete discourse tree.
The underlying tensors infer the salient passages of narrative materials. In
addition, we suggest two algorithms for data augmentation (node reordering and
artificial leaf insertion) that increase our training set and reduce
overfitting. Our benchmarks demonstrate the superior performance of our
approach. Moreover, our tensor structure reveals the salient text passages and
thereby provides explanatory insights
Semi-supervised Sequence Learning
We present two approaches that use unlabeled data to improve sequence
learning with recurrent networks. The first approach is to predict what comes
next in a sequence, which is a conventional language model in natural language
processing. The second approach is to use a sequence autoencoder, which reads
the input sequence into a vector and predicts the input sequence again. These
two algorithms can be used as a "pretraining" step for a later supervised
sequence learning algorithm. In other words, the parameters obtained from the
unsupervised step can be used as a starting point for other supervised training
models. In our experiments, we find that long short term memory recurrent
networks after being pretrained with the two approaches are more stable and
generalize better. With pretraining, we are able to train long short term
memory recurrent networks up to a few hundred timesteps, thereby achieving
strong performance in many text classification tasks, such as IMDB, DBpedia and
20 Newsgroups
Inducing Distant Supervision in Suggestion Mining through Part-of-Speech Embeddings
Mining suggestion expressing sentences from a given text is a less
investigated sentence classification task, and therefore lacks hand labeled
benchmark datasets. In this work, we propose and evaluate two approaches for
distant supervision in suggestion mining. The distant supervision is obtained
through a large silver standard dataset, constructed using the text from
wikiHow and Wikipedia. Both the approaches use a LSTM based neural network
architecture to learn a classification model for suggestion mining, but vary in
their method to use the silver standard dataset. The first approach directly
trains the classifier using this dataset, while the second approach only learns
word embeddings from this dataset. In the second approach, we also learn POS
embeddings, which interestingly gives the best classification accuracy
Deep-Sentiment: Sentiment Analysis Using Ensemble of CNN and Bi-LSTM Models
With the popularity of social networks, and e-commerce websites, sentiment
analysis has become a more active area of research in the past few years. On a
high level, sentiment analysis tries to understand the public opinion about a
specific product or topic, or trends from reviews or tweets. Sentiment analysis
plays an important role in better understanding customer/user opinion, and also
extracting social/political trends. There has been a lot of previous works for
sentiment analysis, some based on hand-engineering relevant textual features,
and others based on different neural network architectures. In this work, we
present a model based on an ensemble of long-short-term-memory (LSTM), and
convolutional neural network (CNN), one to capture the temporal information of
the data, and the other one to extract the local structure thereof. Through
experimental results, we show that using this ensemble model we can outperform
both individual models. We are also able to achieve a very high accuracy rate
compared to the previous works
A Novel Way of Identifying Cyber Predators
Recurrent Neural Networks with Long Short-Term Memory cell (LSTM-RNN) have
impressive ability in sequence data processing, particularly for language model
building and text classification. This research proposes the combination of
sentiment analysis, new approach of sentence vectors and LSTM-RNN as a novel
way for Sexual Predator Identification (SPI). LSTM-RNN language model is
applied to generate sentence vectors which are the last hidden states in the
language model. Sentence vectors are fed into another LSTM-RNN classifier, so
as to capture suspicious conversations. Hidden state enables to generate
vectors for sentences never seen before. Fasttext is used to filter the
contents of conversations and generate a sentiment score so as to identify
potential predators. The experiment achieves a record-breaking accuracy and
precision of 100% with recall of 81.10%, exceeding the top-ranked result in the
SPI competition.Comment: 6 page
Deep Learning for Sentiment Analysis : A Survey
Deep learning has emerged as a powerful machine learning technique that
learns multiple layers of representations or features of the data and produces
state-of-the-art prediction results. Along with the success of deep learning in
many other application domains, deep learning is also popularly used in
sentiment analysis in recent years. This paper first gives an overview of deep
learning and then provides a comprehensive survey of its current applications
in sentiment analysis.Comment: 34 pages, 9 figures, 2 table
Leveraging Sparse and Dense Feature Combinations for Sentiment Classification
Neural networks are one of the most popular approaches for many natural
language processing tasks such as sentiment analysis. They often outperform
traditional machine learning models and achieve the state-of-art results on
most tasks. However, many existing deep learning models are complex, difficult
to train and provide a limited improvement over simpler methods. We propose a
simple, robust and powerful model for sentiment classification. This model
outperforms many deep learning models and achieves comparable results to other
deep learning models with complex architectures on sentiment analysis datasets.
We publish the code online.Comment: 4 page
Gated Convolutional Neural Networks for Domain Adaptation
Domain Adaptation explores the idea of how to maximize performance on a
target domain, distinct from source domain, upon which the classifier was
trained. This idea has been explored for the task of sentiment analysis
extensively. The training of reviews pertaining to one domain and evaluation on
another domain is widely studied for modeling a domain independent algorithm.
This further helps in understanding correlation between domains. In this paper,
we show that Gated Convolutional Neural Networks (GCN) perform effectively at
learning sentiment analysis in a manner where domain dependant knowledge is
filtered out using its gates. We perform our experiments on multiple gate
architectures: Gated Tanh ReLU Unit (GTRU), Gated Tanh Unit (GTU) and Gated
Linear Unit (GLU). Extensive experimentation on two standard datasets relevant
to the task, reveal that training with Gated Convolutional Neural Networks give
significantly better performance on target domains than regular convolution and
recurrent based architectures. While complex architectures like attention,
filter domain specific knowledge as well, their complexity order is remarkably
high as compared to gated architectures. GCNs rely on convolution hence gaining
an upper hand through parallelization.Comment: Accepted Long Paper at 24th International Conference on Applications
of Natural Language to Information Systems, June 2019, MediaCityUK Campus,
United Kingdo
Assessing State-of-the-Art Sentiment Models on State-of-the-Art Sentiment Datasets
There has been a good amount of progress in sentiment analysis over the past
10 years, including the proposal of new methods and the creation of benchmark
datasets. In some papers, however, there is a tendency to compare models only
on one or two datasets, either because of time restraints or because the model
is tailored to a specific task. Accordingly, it is hard to understand how well
a certain model generalizes across different tasks and datasets. In this paper,
we contribute to this situation by comparing several models on six different
benchmarks, which belong to different domains and additionally have different
levels of granularity (binary, 3-class, 4-class and 5-class). We show that
Bi-LSTMs perform well across datasets and that both LSTMs and Bi-LSTMs are
particularly good at fine-grained sentiment tasks (i. e., with more than two
classes). Incorporating sentiment information into word embeddings during
training gives good results for datasets that are lexically similar to the
training data. With our experiments, we contribute to a better understanding of
the performance of different model architectures on different data sets.
Consequently, we detect novel state-of-the-art results on the SenTube datasets.Comment: Presented at WASSA 201
Learning Semantic Sentence Embeddings using Sequential Pair-wise Discriminator
In this paper, we propose a method for obtaining sentence-level embeddings.
While the problem of securing word-level embeddings is very well studied, we
propose a novel method for obtaining sentence-level embeddings. This is
obtained by a simple method in the context of solving the paraphrase generation
task. If we use a sequential encoder-decoder model for generating paraphrase,
we would like the generated paraphrase to be semantically close to the original
sentence. One way to ensure this is by adding constraints for true paraphrase
embeddings to be close and unrelated paraphrase candidate sentence embeddings
to be far. This is ensured by using a sequential pair-wise discriminator that
shares weights with the encoder that is trained with a suitable loss function.
Our loss function penalizes paraphrase sentence embedding distances from being
too large. This loss is used in combination with a sequential encoder-decoder
network. We also validated our method by evaluating the obtained embeddings for
a sentiment analysis task. The proposed method results in semantic embeddings
and outperforms the state-of-the-art on the paraphrase generation and sentiment
analysis task on standard datasets. These results are also shown to be
statistically significant.Comment: COLING 2018 (accepted
- …