230 research outputs found
Cross-domain sentiment classification using a sentiment sensitive thesaurus
Automatic classification of sentiment is important for numerous applications such as opinion mining, opinion summarization, contextual advertising, and market analysis. However, sentiment is expressed differently in different domains, and annotating corpora for every possible domain of interest is costly. Applying a sentiment classifier trained using labeled data for a particular domain to classify sentiment of user reviews on a different domain often results in poor performance. We propose a method to overcome this problem in cross-domain sentiment classification. First, we create a sentiment sensitive distributional thesaurus using labeled data for the source domains and unlabeled data for both source and target domains. Sentiment sensitivity is achieved in the thesaurus by incorporating document level sentiment labels in the context vectors used as the basis for measuring the distributional similarity between words. Next, we use the created thesaurus to expand feature vectors during train and test times in a binary classifier. The proposed method significantly outperforms numerous baselines and returns results that are comparable with previously proposed cross-domain sentiment classification methods. We conduct an extensive empirical analysis of the proposed method on single and multi-source domain adaptation, unsupervised and supervised domain adaptation, and numerous similarity measures for creating the sentiment sensitive thesaurus
Learning to predict distributions of words across domains
Although the distributional hypothesis has been applied successfully in many natural language processing tasks, systems using distributional information have been limited to a single domain because the distribution of a word can vary between domains as the word’s predominant meaning changes. However, if it were possible to predict how the distribution of a word changes from one domain to another, the predictions could be used to adapt a system trained in one domain to work in another. We propose an unsupervised method to predict the distribution of a word in one domain, given its distribution in another domain. We evaluate our method on two tasks: cross-domain part-of-speech tagging and cross-domain sentiment classification. In both tasks, our method significantly outperforms competitive baselines and returns results that are statistically comparable to current state-of-the-art methods, while requiring no task-specific customisations
Embedding Semantic Relations into Word Representations
Learning representations for semantic relations is important for various
tasks such as analogy detection, relational search, and relation
classification. Although there have been several proposals for learning
representations for individual words, learning word representations that
explicitly capture the semantic relations between words remains under
developed. We propose an unsupervised method for learning vector
representations for words such that the learnt representations are sensitive to
the semantic relations that exist between two words. First, we extract lexical
patterns from the co-occurrence contexts of two words in a corpus to represent
the semantic relations that exist between those two words. Second, we represent
a lexical pattern as the weighted sum of the representations of the words that
co-occur with that lexical pattern. Third, we train a binary classifier to
detect relationally similar vs. non-similar lexical pattern pairs. The proposed
method is unsupervised in the sense that the lexical pattern pairs we use as
train data are automatically sampled from a corpus, without requiring any
manual intervention. Our proposed method statistically significantly
outperforms the current state-of-the-art word representations on three
benchmark datasets for proportional analogy detection, demonstrating its
ability to accurately capture the semantic relations among words.Comment: International Joint Conferences in AI (IJCAI) 201
Meta-Embedding as Auxiliary Task Regularization.
Word embeddings have been shown to benefit from ensambling several word
embedding sources, often carried out using straightforward mathematical
operations over the set of word vectors. More recently, self-supervised
learning has been used to find a lower-dimensional representation, similar in
size to the individual word embeddings within the ensemble. However, these
methods do not use the available manually labeled datasets that are often used
solely for the purpose of evaluation. We propose to reconstruct an ensemble of
word embeddings as an auxiliary task that regularises a main task while both
tasks share the learned meta-embedding layer. We carry out intrinsic evaluation
(6 word similarity datasets and 3 analogy datasets) and extrinsic evaluation (4
downstream tasks). For intrinsic task evaluation, supervision comes from
various labeled word similarity datasets. Our experimental results show that
the performance is improved for all word similarity datasets when compared to
self-supervised learning methods with a mean increase of in Spearman
correlation. Specifically, the proposed method shows the best performance in 4
out of 6 of word similarity datasets when using a cosine reconstruction loss
and Brier's word similarity loss. Moreover, improvements are also made when
performing word meta-embedding reconstruction in sequence tagging and sentence
meta-embedding for sentence classification
- …