382 research outputs found
Multi-Label Learning with Label Enhancement
The task of multi-label learning is to predict a set of relevant labels for
the unseen instance. Traditional multi-label learning algorithms treat each
class label as a logical indicator of whether the corresponding label is
relevant or irrelevant to the instance, i.e., +1 represents relevant to the
instance and -1 represents irrelevant to the instance. Such label represented
by -1 or +1 is called logical label. Logical label cannot reflect different
label importance. However, for real-world multi-label learning problems, the
importance of each possible label is generally different. For the real
applications, it is difficult to obtain the label importance information
directly. Thus we need a method to reconstruct the essential label importance
from the logical multilabel data. To solve this problem, we assume that each
multi-label instance is described by a vector of latent real-valued labels,
which can reflect the importance of the corresponding labels. Such label is
called numerical label. The process of reconstructing the numerical labels from
the logical multi-label data via utilizing the logical label information and
the topological structure in the feature space is called Label Enhancement. In
this paper, we propose a novel multi-label learning framework called LEMLL,
i.e., Label Enhanced Multi-Label Learning, which incorporates regression of the
numerical labels and label enhancement into a unified framework. Extensive
comparative studies validate that the performance of multi-label learning can
be improved significantly with label enhancement and LEMLL can effectively
reconstruct latent label importance information from logical multi-label data.Comment: ICDM 201
Learning task specific distributed paragraph representations using a 2-tier convolutional neural network
We introduce a type of 2-tier convolutional neural network model for learning distributed paragraph representations for a special task (e.g. paragraph or short document level sentiment analysis and text topic categorization). We decompose the paragraph semantics into 3 cascaded constitutes: word representation, sentence composition and document composition. Specifically, we learn distributed word representations by a continuous bag-of-words model from a large unstructured text corpus. Then, using these word representations as pre-trained vectors, distributed task specific sentence representations are learned from a sentence level corpus with task-specific labels by the first tier of our model. Using these sentence representations as distributed paragraph representation vectors, distributed paragraph representations are learned from a paragraph-level corpus by the second tier of our model. It is evaluated on DBpedia ontology classification dataset and Amazon review dataset. Empirical results show the effectiveness of our proposed learning model for generating distributed paragraph representations
Reducing Spurious Correlations for Aspect-Based Sentiment Analysis with Variational Information Bottleneck and Contrastive Learning
Deep learning techniques have dominated the literature on aspect-based
sentiment analysis (ABSA), yielding state-of-the-art results. However, these
deep models generally suffer from spurious correlation problems between input
features and output labels, which creates significant barriers to robustness
and generalization capability. In this paper, we propose a novel Contrastive
Variational Information Bottleneck framework (called CVIB) to reduce spurious
correlations for ABSA. The proposed CVIB framework is composed of an original
network and a self-pruned network, and these two networks are optimized
simultaneously via contrastive learning. Concretely, we employ the Variational
Information Bottleneck (VIB) principle to learn an informative and compressed
network (self-pruned network) from the original network, which discards the
superfluous patterns or spurious correlations between input features and
prediction labels. Then, self-pruning contrastive learning is devised to pull
together semantically similar positive pairs and push away dissimilar pairs,
where the representations of the anchor learned by the original and self-pruned
networks respectively are regarded as a positive pair while the representations
of two different sentences within a mini-batch are treated as a negative pair.
To verify the effectiveness of our CVIB method, we conduct extensive
experiments on five benchmark ABSA datasets and the experimental results show
that our approach achieves better performance than the strong competitors in
terms of overall prediction performance, robustness, and generalization
A convolutional attentional neural network for sentiment classification
Neural network models with attention mechanism have shown their efficiencies on various tasks. However, there is little research work on attention mechanism for text classification and existing attention model for text classification lacks of cognitive intuition and mathematical explanation. In this paper, we propose a new architecture of neural network based on the attention model for text classification. In particular, we show that the convolutional neural network (CNN) is a reasonable model for extracting attentions from text sequences in mathematics. We then propose a novel attention model base on CNN and introduce a new network architecture which combines recurrent neural network with our CNN-based attention model. Experimental results on five datasets show that our proposed models can accurately capture the salient parts of sentences to improve the performance of text classification
Recommended from our members
Automatic labelling of topic models learned from Twitter by summarisation
Latent topics derived by topic models such as Latent Dirichlet Allocation (LDA) are the result of hidden thematic structures which provide further insights into the data. The automatic labelling of such topics derived from social media poses however new challenges since topics may characterise novel events happening in the real world. Existing automatic topic labelling approaches which depend on external knowledge sources become less applicable here since relevant articles/concepts of the extracted topics may not exist in external sources. In this paper we propose to address the problem of automatic labelling of latent topics learned from Twitter as a summarisation problem. We introduce a framework which apply summarisation algorithms to generate topic labels. These algorithms are independent of external sources and only rely on the identification of dominant terms in documents related to the latent topic. We compare the efficiency of existing state of the art summarisation algorithms. Our results suggest that summarisation algorithms generate better topic labels which capture event-related context compared to the top-n terms returned by LDA
Biogenesis of iron–sulfur clusters and their role in DNA metabolism
Iron–sulfur (Fe/S) clusters (ISCs) are redox-active protein cofactors that their synthesis, transfer, and insertion into target proteins require many components. Mitochondrial ISC assembly is the foundation of all cellular ISCs in eukaryotic cells. The mitochondrial ISC cooperates with the cytosolic Fe/S protein assembly (CIA) systems to accomplish the cytosolic and nuclear Fe/S clusters maturation. ISCs are needed for diverse cellular functions, including nitrogen fixation, oxidative phosphorylation, mitochondrial respiratory pathways, and ribosome assembly. Recent research advances have confirmed the existence of different ISCs in enzymes that regulate DNA metabolism, including helicases, nucleases, primases, DNA polymerases, and glycosylases. Here we outline the synthesis of mitochondrial, cytosolic and nuclear ISCs and highlight their functions in DNA metabolism
Learning user and product distributed representations using a sequence model for sentiment analysis
In product reviews, it is observed that the distribution of polarity ratings over reviews written by different users or evaluated based on different products are often skewed in the real world. As such, incorporating user and product information would be helpful for the task of sentiment classification of reviews. However, existing approaches ignored the temporal nature of reviews posted by the same user or evaluated on the same product. We argue that the temporal relations of reviews might be potentially useful for learning user and product embedding and thus propose employing a sequence model to embed these temporal relations into user and product representations so as to improve the performance of document-level sentiment analysis. Specifically, we first learn a distributed representation of each review by a one-dimensional convolutional neural network. Then, taking these representations as pretrained vectors, we use a recurrent neural network with gated recurrent units to learn distributed representations of users and products. Finally, we feed the user, product and review representations into a machine learning classifier for sentiment classification. Our approach has been evaluated on three large-scale review datasets from the IMDB and Yelp. Experimental results show that: (1) sequence modeling for the purposes of distributed user and product representation learning can improve the performance of document-level sentiment classification; (2) the proposed approach achieves state-of-The-Art results on these benchmark datasets
Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: A Preliminary Empirical Study
Evaluating the quality of generated text is a challenging task in natural
language processing. This difficulty arises from the inherent complexity and
diversity of text. Recently, OpenAI's ChatGPT, a powerful large language model
(LLM), has garnered significant attention due to its impressive performance in
various tasks. Therefore, we present this report to investigate the
effectiveness of LLMs, especially ChatGPT, and explore ways to optimize their
use in assessing text quality. We compared three kinds of reference-free
evaluation methods based on ChatGPT or similar LLMs. The experimental results
prove that ChatGPT is capable to evaluate text quality effectively from various
perspectives without reference and demonstrates superior performance than most
existing automatic metrics. In particular, the Explicit Score, which utilizes
ChatGPT to generate a numeric score measuring text quality, is the most
effective and reliable method among the three exploited approaches. However,
directly comparing the quality of two texts using ChatGPT may lead to
suboptimal results. We hope this report will provide valuable insights into
selecting appropriate methods for evaluating text quality with LLMs such as
ChatGPT.Comment: Technical Report, 13 page
- …