121 research outputs found
Multi-theme sentiment analysis with sentiment shifting
Business reviews contain rich sentiment on multiple themes, disclosing more interesting information than the overall polarities of documents. When it comes to fine-grained sentiment analysis, given any segment of text, we are not only interested in overall polarity of such segment, but also the sentiment words play major effects. However, sentiment analysis at the word level poses significant challenges due to the complexity of reviews, the inconsistency of sentiment in different themes, and the sentiment shifting resulting from linguistic patterns---contextual valence shifters.
To simultaneously resolve the multi-theme and sentiment shifting dilemma, a unified explainable sentiment analysis model, MTSA, is proposed in this paper, which enables both classification of sentiment polarity and discovery of quantified sentiment-shifting patterns. MTSA formulates multi-theme sentiment by learning embeddings (i.e., vector representations) for both themes and words, and derives the shifter effect learning algorithm by modeling the shifted sentiment in a logistic regression model.
Extensive experiments have been conducted on Yelp business reviews and IMDB movie reviews. The improvement of sentiment polarity classification demonstrates the effectiveness of MTSA at rectifying word feature representations of reviews, and the human evaluation shows its successful discovery of multi-theme sentiment words and automatic effect quantification of contextual valence shifters
Enable Language Models to Implicitly Learn Self-Improvement From Data
Large Language Models (LLMs) have demonstrated remarkable capabilities in
open-ended text generation tasks. However, the inherent open-ended nature of
these tasks implies that there is always room for improvement in the quality of
model responses. To address this challenge, various approaches have been
proposed to enhance the performance of LLMs. There has been a growing focus on
enabling LLMs to self-improve their response quality, thereby reducing the
reliance on extensive human annotation efforts for collecting diverse and
high-quality training data. Recently, prompting-based methods have been widely
explored among self-improvement methods owing to their effectiveness,
efficiency, and convenience. However, those methods usually require explicitly
and thoroughly written rubrics as inputs to LLMs. It is expensive and
challenging to manually derive and provide all necessary rubrics with a
real-world complex goal for improvement (e.g., being more helpful and less
harmful). To this end, we propose an ImPlicit Self-ImprovemenT (PIT) framework
that implicitly learns the improvement goal from human preference data. PIT
only requires preference data that are used to train reward models without
extra human efforts. Specifically, we reformulate the training objective of
reinforcement learning from human feedback (RLHF) -- instead of maximizing
response quality for a given input, we maximize the quality gap of the response
conditioned on a reference response. In this way, PIT is implicitly trained
with the improvement goal of better aligning with human preferences.
Experiments on two real-world datasets and one synthetic dataset show that our
method significantly outperforms prompting-based methods.Comment: 28 pages, 5 figures, 4 table
Augmentation with Projection: Towards an Effective and Efficient Data Augmentation Paradigm for Distillation
Knowledge distillation is one of the primary methods of transferring
knowledge from large to small models. However, it requires massive
task-specific data, which may not be plausible in many real-world applications.
Data augmentation methods such as representation interpolation, token
replacement, or augmentation with models are applied to tackle this problem.
However, these data augmentation methods either potentially cause shifts in
decision boundaries (representation interpolation), are not expressive enough
(token replacement), or introduce too much computational overhead (augmentation
with models). To this end, we propose AugPro (Augmentation with Projection), an
effective and efficient data augmentation method for distillation. Our method
builds on top of representation interpolation augmentation methods to maintain
the diversity of expressions and converts the augmented data to tokens to avoid
shifting decision boundaries. It uses simple operations that come with little
computational overhead. The results on multiple GLUE tasks show that our
methods can improve distillation performance by a large margin at a low time
cost. Codes are available at
https://github.com/google-research/google-research/tree/master/augpro.Comment: 20 pages, 5 figures. Accepted by ICLR 202
Analysis of General Network Coding Conditions and Design of a Free-Ride-Oriented Routing Metric
Recommended from our members
Tailoring Light-Matter Interaction with a Nanoscale Plasmon Resonator
We propose and demonstrate a new approach for achieving enhanced light-matter interactions with quantum emitters. Our approach makes use of a plasmon resonator composed of defect-free, highly crystalline silver nanowires surrounded by patterned dielectric distributed Bragg reflectors. These resonators have an effective mode volume (Veff) 2 orders of magnitude below the diffraction limit and a quality factor (Q) approaching 100, enabling enhancement of spontaneous emission rates by a factor exceeding 75 at the cavity resonance. We also show that these resonators can be used to convert a broadband quantum emitter to a narrow-band single-photon source with color-selective emission enhancement.Physic
Latent factor transition for dynamic collaborative filtering
User preferences change over time and capturing such changes is essential for developing accurate recom-mender systems. Despite its importance, only a few works in collaborative filtering have addressed this is-sue. In this paper, we consider evolving preferences and we model user dynamics by introducing and learning a transition matrix for each user’s latent vectors between consecutive time windows. Intuitively, the transition matrix for a user summarizes the time-invariant pat-tern of the evolution for the user. We first extend the conventional probabilistic matrix factorization and then improve upon this solution through its fully Bayesian model. These solutions take advantage of the model complexity and scalability of conventional Bayesian ma-trix factorization, yet adapt dynamically to user’s evolv-ing preferences. We evaluate the effectiveness of these solutions through empirical studies on six large-scale real life data sets
- …