300 research outputs found
A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithm
Domain incremental learning aims to adapt to a sequence of domains with
access to only a small subset of data (i.e., memory) from previous domains.
Various methods have been proposed for this problem, but it is still unclear
how they are related and when practitioners should choose one method over
another. In response, we propose a unified framework, dubbed Unified Domain
Incremental Learning (UDIL), for domain incremental learning with memory. Our
UDIL **unifies** various existing methods, and our theoretical analysis shows
that UDIL always achieves a tighter generalization error bound compared to
these methods. The key insight is that different existing methods correspond to
our bound with different **fixed** coefficients; based on insights from this
unification, our UDIL allows **adaptive** coefficients during training, thereby
always achieving the tightest bound. Empirical results show that our UDIL
outperforms the state-of-the-art domain incremental learning methods on both
synthetic and real-world datasets. Code will be available at
https://github.com/Wang-ML-Lab/unified-continual-learning.Comment: Accepted at NeurIPS 202
Relational Sentence Embedding for Flexible Semantic Matching
We present Relational Sentence Embedding (RSE), a new paradigm to further
discover the potential of sentence embeddings. Prior work mainly models the
similarity between sentences based on their embedding distance. Because of the
complex semantic meanings conveyed, sentence pairs can have various relation
types, including but not limited to entailment, paraphrasing, and
question-answer. It poses challenges to existing embedding methods to capture
such relational information. We handle the problem by learning associated
relational embeddings. Specifically, a relation-wise translation operation is
applied to the source sentence to infer the corresponding target sentence with
a pre-trained Siamese-based encoder. The fine-grained relational similarity
scores can be computed from learned embeddings. We benchmark our method on 19
datasets covering a wide range of tasks, including semantic textual similarity,
transfer, and domain-specific tasks. Experimental results show that our method
is effective and flexible in modeling sentence relations and outperforms a
series of state-of-the-art sentence embedding methods.
https://github.com/BinWang28/RSEComment: RepL4NLP at ACL 202
Recommended from our members
Conversion of p–n conduction type by spinodal decomposition in Zn-Sb-Bi phase-change alloys
Phase-change films with multiple resistance levels are promising for increasing the storage density in phase-change memory technology. Diffusion-dominated Zn2Sb3 films undergo transitions across three states, from high through intermediate to low resistance, upon annealing. The properties of the Zn2Sb3 material can be further optimized by doping with Bi. Based on scanning transmission electron microscopy combined with electrical transport measurements, at a particular Bi concentration, the conduction of Zn-Sb-Bi compounds changes from p- to n-type, originating from spinodal decomposition. Simultaneously, the change in the temperature coefficient of resistivity shows a metal-to-insulator transition. Further analysis of microstructure characteristics reveals that the distribution of the Bi-Sb phase may be the origin of the driving force for the p–n conduction and metal-to-insulator transitions and therefore may provide us with another way to improve multilevel data storage. Moreover, the Bi doping promotes the thermoelectric properties of the studied alloys, leading to higher values of the power factor compared to known reported structures. The present study sheds valuable light on the spinodal decomposition process caused by Bi doping, which can also occur in a wide variety of chalcogenide-based phase-change materials. In addition, the study provides a new strategy for realizing novel p–n heterostructures for multilevel data storage and thermoelectric applications
A Focused Study on Sequence Length for Dialogue Summarization
Output length is critical to dialogue summarization systems. The dialogue
summary length is determined by multiple factors, including dialogue
complexity, summary objective, and personal preferences. In this work, we
approach dialogue summary length from three perspectives. First, we analyze the
length differences between existing models' outputs and the corresponding human
references and find that summarization models tend to produce more verbose
summaries due to their pretraining objectives. Second, we identify salient
features for summary length prediction by comparing different model settings.
Third, we experiment with a length-aware summarizer and show notable
improvement on existing models if summary length can be well incorporated.
Analysis and experiments are conducted on popular DialogSum and SAMSum datasets
to validate our findings.Comment: Preprint version - ICASSP submissio
Multimodal Short Video Rumor Detection System Based on Contrastive Learning
With short video platforms becoming one of the important channels for news
sharing, major short video platforms in China have gradually become new
breeding grounds for fake news. However, it is not easy to distinguish short
video rumors due to the great amount of information and features contained in
short videos, as well as the serious homogenization and similarity of features
among videos. In order to mitigate the spread of short video rumors, our group
decides to detect short video rumors by constructing multimodal feature fusion
and introducing external knowledge after considering the advantages and
disadvantages of each algorithm. The ideas of detection are as follows: (1)
dataset creation: to build a short video dataset with multiple features; (2)
multimodal rumor detection model: firstly, we use TSN (Temporal Segment
Networks) video coding model to extract video features; then, we use OCR
(Optical Character Recognition) and ASR (Automatic Character Recognition) to
extract video features. Recognition) and ASR (Automatic Speech Recognition)
fusion to extract text, and then use the BERT model to fuse text features with
video features (3) Finally, use contrast learning to achieve distinction: first
crawl external knowledge, then use the vector database to achieve the
introduction of external knowledge and the final structure of the
classification output. Our research process is always oriented to practical
needs, and the related knowledge results will play an important role in many
practical scenarios such as short video rumor identification and social opinion
control
- …