2,744 research outputs found
W2VLDA: Almost Unsupervised System for Aspect Based Sentiment Analysis
With the increase of online customer opinions in specialised websites and
social networks, the necessity of automatic systems to help to organise and
classify customer reviews by domain-specific aspect/categories and sentiment
polarity is more important than ever. Supervised approaches to Aspect Based
Sentiment Analysis obtain good results for the domain/language their are
trained on, but having manually labelled data for training supervised systems
for all domains and languages are usually very costly and time consuming. In
this work we describe W2VLDA, an almost unsupervised system based on topic
modelling, that combined with some other unsupervised methods and a minimal
configuration, performs aspect/category classifiation,
aspect-terms/opinion-words separation and sentiment polarity classification for
any given domain and language. We evaluate the performance of the aspect and
sentiment classification in the multilingual SemEval 2016 task 5 (ABSA)
dataset. We show competitive results for several languages (English, Spanish,
French and Dutch) and domains (hotels, restaurants, electronic-devices)
Decision support from financial disclosures with deep neural networks and transfer learning
Company disclosures greatly aid in the process of financial decision-making;
therefore, they are consulted by financial investors and automated traders
before exercising ownership in stocks. While humans are usually able to
correctly interpret the content, the same is rarely true of computerized
decision support systems, which struggle with the complexity and ambiguity of
natural language. A possible remedy is represented by deep learning, which
overcomes several shortcomings of traditional methods of text mining. For
instance, recurrent neural networks, such as long short-term memories, employ
hierarchical structures, together with a large number of hidden layers, to
automatically extract features from ordered sequences of words and capture
highly non-linear relationships such as context-dependent meanings. However,
deep learning has only recently started to receive traction, possibly because
its performance is largely untested. Hence, this paper studies the use of deep
neural networks for financial decision support. We additionally experiment with
transfer learning, in which we pre-train the network on a different corpus with
a length of 139.1 million words. Our results reveal a higher directional
accuracy as compared to traditional machine learning when predicting stock
price movements in response to financial disclosures. Our work thereby helps to
highlight the business value of deep learning and provides recommendations to
practitioners and executives
A Deep Neural Architecture for Sentence-level Sentiment Classification in Twitter Social Networking
This paper introduces a novel deep learning framework including a
lexicon-based approach for sentence-level prediction of sentiment label
distribution. We propose to first apply semantic rules and then use a Deep
Convolutional Neural Network (DeepCNN) for character-level embeddings in order
to increase information for word-level embedding. After that, a Bidirectional
Long Short-Term Memory Network (Bi-LSTM) produces a sentence-wide feature
representation from the word-level embedding. We evaluate our approach on three
Twitter sentiment classification datasets. Experimental results show that our
model can improve the classification accuracy of sentence-level sentiment
analysis in Twitter social networking.Comment: PACLING Conference 2017, 6 page
Evaluation of sentence embeddings in downstream and linguistic probing tasks
Despite the fast developmental pace of new sentence embedding methods, it is
still challenging to find comprehensive evaluations of these different
techniques. In the past years, we saw significant improvements in the field of
sentence embeddings and especially towards the development of universal
sentence encoders that could provide inductive transfer to a wide variety of
downstream tasks. In this work, we perform a comprehensive evaluation of recent
methods using a wide variety of downstream and linguistic feature probing
tasks. We show that a simple approach using bag-of-words with a recently
introduced language model for deep context-dependent word embeddings proved to
yield better results in many tasks when compared to sentence encoders trained
on entailment datasets. We also show, however, that we are still far away from
a universal encoder that can perform consistently across several downstream
tasks.Comment: 15 pages, 3 figures, 11 table
Cross-Lingual Word Embeddings for Turkic Languages
There has been an increasing interest in learning cross-lingual word
embeddings to transfer knowledge obtained from a resource-rich language, such
as English, to lower-resource languages for which annotated data is scarce,
such as Turkish, Russian, and many others. In this paper, we present the first
viability study of established techniques to align monolingual embedding spaces
for Turkish, Uzbek, Azeri, Kazakh and Kyrgyz, members of the Turkic family
which is heavily affected by the low-resource constraint. Those techniques are
known to require little explicit supervision, mainly in the form of bilingual
dictionaries, hence being easily adaptable to different domains, including
low-resource ones. We obtain new bilingual dictionaries and new word embeddings
for these languages and show the steps for obtaining cross-lingual word
embeddings using state-of-the-art techniques. Then, we evaluate the results
using the bilingual dictionary induction task. Our experiments confirm that the
obtained bilingual dictionaries outperform previously-available ones, and that
word embeddings from a low-resource language can benefit from resource-rich
closely-related languages when they are aligned together. Furthermore,
evaluation on an extrinsic task (Sentiment analysis on Uzbek) proves that
monolingual word embeddings can, although slightly, benefit from cross-lingual
alignments.Comment: Final version, published in the proceedings of LREC 202
Basic tasks of sentiment analysis
Subjectivity detection is the task of identifying objective and subjective
sentences. Objective sentences are those which do not exhibit any sentiment.
So, it is desired for a sentiment analysis engine to find and separate the
objective sentences for further analysis, e.g., polarity detection. In
subjective sentences, opinions can often be expressed on one or multiple
topics. Aspect extraction is a subtask of sentiment analysis that consists in
identifying opinion targets in opinionated text, i.e., in detecting the
specific aspects of a product or service the opinion holder is either praising
or complaining about
Sentiment analysis based on rhetorical structure theory: Learning deep neural networks from discourse trees
Prominent applications of sentiment analysis are countless, covering areas
such as marketing, customer service and communication. The conventional
bag-of-words approach for measuring sentiment merely counts term frequencies;
however, it neglects the position of the terms within the discourse. As a
remedy, we develop a discourse-aware method that builds upon the discourse
structure of documents. For this purpose, we utilize rhetorical structure
theory to label (sub-)clauses according to their hierarchical relationships and
then assign polarity scores to individual leaves. To learn from the resulting
rhetorical structure, we propose a tensor-based, tree-structured deep neural
network (named Discourse-LSTM) in order to process the complete discourse tree.
The underlying tensors infer the salient passages of narrative materials. In
addition, we suggest two algorithms for data augmentation (node reordering and
artificial leaf insertion) that increase our training set and reduce
overfitting. Our benchmarks demonstrate the superior performance of our
approach. Moreover, our tensor structure reveals the salient text passages and
thereby provides explanatory insights
Learning Emoji Embeddings using Emoji Co-occurrence Network Graph
Usage of emoji in social media platforms has seen a rapid increase over the
last few years. Majority of the social media posts are laden with emoji and
users often use more than one emoji in a single social media post to express
their emotions and to emphasize certain words in a message. Utilizing the emoji
co-occurrence can be helpful to understand how emoji are used in social media
posts and their meanings in the context of social media posts. In this paper,
we investigate whether emoji co-occurrences can be used as a feature to learn
emoji embeddings which can be used in many downstream applications such
sentiment analysis and emotion identification in social media text. We utilize
147 million tweets which have emojis in them and build an emoji co-occurrence
network. Then, we train a network embedding model to embed emojis into a low
dimensional vector space. We evaluate our embeddings using sentiment analysis
and emoji similarity experiments, and experimental results show that our
embeddings outperform the current state-of-the-art results for sentiment
analysis tasks.Comment: Accepted at the 1st International Workshop on Emoji Understanding and
Applications in Social Media Co located with ICWSM 201
Learning Cross-lingual Embeddings from Twitter via Distant Supervision
Cross-lingual embeddings represent the meaning of words from different
languages in the same vector space. Recent work has shown that it is possible
to construct such representations by aligning independently learned monolingual
embedding spaces, and that accurate alignments can be obtained even without
external bilingual data. In this paper we explore a research direction that has
been surprisingly neglected in the literature: leveraging noisy user-generated
text to learn cross-lingual embeddings particularly tailored towards social
media applications. While the noisiness and informal nature of the social media
genre poses additional challenges to cross-lingual embedding methods, we find
that it also provides key opportunities due to the abundance of code-switching
and the existence of a shared vocabulary of emoji and named entities. Our
contribution consists of a very simple post-processing step that exploits these
phenomena to significantly improve the performance of state-of-the-art
alignment methods.Comment: Accepted to ICWSM 2020. 11 pages, 1 appendix. Pre-trained embeddings
available at https://github.com/pedrada88/crossembeddings-twitte
Unsupervised Document Embedding With CNNs
We propose a new model for unsupervised document embedding. Leading existing
approaches either require complex inference or use recurrent neural networks
(RNN) that are difficult to parallelize. We take a different route and develop
a convolutional neural network (CNN) embedding model. Our CNN architecture is
fully parallelizable resulting in over 10x speedup in inference time over RNN
models. Parallelizable architecture enables to train deeper models where each
successive layer has increasingly larger receptive field and models longer
range semantic structure within the document. We additionally propose a fully
unsupervised learning algorithm to train this model based on stochastic forward
prediction. Empirical results on two public benchmarks show that our approach
produces comparable to state-of-the-art accuracy at a fraction of computational
cost.Comment: Major revision with additional experiments and model descriptio
- …