599 research outputs found
AdaER: An Adaptive Experience Replay Approach for Continual Lifelong Learning
Continual lifelong learning is an machine learning framework inspired by
human learning, where learners are trained to continuously acquire new
knowledge in a sequential manner. However, the non-stationary nature of
streaming training data poses a significant challenge known as catastrophic
forgetting, which refers to the rapid forgetting of previously learned
knowledge when new tasks are introduced. While some approaches, such as
experience replay (ER), have been proposed to mitigate this issue, their
performance remains limited, particularly in the class-incremental scenario
which is considered natural and highly challenging. In this paper, we present a
novel algorithm, called adaptive-experience replay (AdaER), to address the
challenge of continual lifelong learning. AdaER consists of two stages: memory
replay and memory update. In the memory replay stage, AdaER introduces a
contextually-cued memory recall (C-CMR) strategy, which selectively replays
memories that are most conflicting with the current input data in terms of both
data and task. Additionally, AdaER incorporates an entropy-balanced reservoir
sampling (E-BRS) strategy to enhance the performance of the memory buffer by
maximizing information entropy. To evaluate the effectiveness of AdaER, we
conduct experiments on established supervised continual lifelong learning
benchmarks, specifically focusing on class-incremental learning scenarios. The
results demonstrate that AdaER outperforms existing continual lifelong learning
baselines, highlighting its efficacy in mitigating catastrophic forgetting and
improving learning performance.Comment: 18 pages, 26 figure
Information recovery in the Hayden-Preskill protocol
We revisit information retrieval from evaporating black holes in the
Hayden-Preskill protocol, treating the black hole dynamics as Haar-random. We
compute, down to the first exponentially suppressed terms, all integer-indexed
R\'enyi mutual informations between a black hole, its radiation, and a
reference that catalogues Alice's diaries. We find that dropping a diary into a
young black hole effectively delays the Page time. We also compute the
radiation : diary reflected R\'enyi entropies, and identify a technical reason
why they cannot be continued to the reflected entropy by the replica trick.Comment: 24 pages plus appendice
Reducing Sensitivity on Speaker Names for Text Generation from Dialogues
Changing speaker names consistently throughout a dialogue should not affect
its meaning and corresponding outputs for text generation from dialogues.
However, pre-trained language models, serving as the backbone for
dialogue-processing tasks, have shown to be sensitive to nuances. This may
result in unfairness in real-world applications. No comprehensive analysis of
this problem has been done in the past. In this work, we propose to
quantitatively measure a model's sensitivity on speaker names, and
comprehensively evaluate a number of known methods for reducing speaker name
sensitivity, including a novel approach of our own. Extensive experiments on
multiple datasets provide a benchmark for this problem and show the favorable
performance of our approach in sensitivity reduction and quality of generation.Comment: findings of ACL'2
Knowledge Represent and Reconstruction by “Fundamentals of Materials Science” Classroom Teaching Mode Reform
AbstractClassroom teaching is the main form of teaching organization and activity way, and is also the main base on the classroom teaching mode reform. This article by “Fundamentals of Materials Science” as an example, generalizing the knowledge representation of three types and advantages in the classroom teaching, points out that the teacher's role in this progresss. We analyze that the feasibility and the ideal effect on rebuilding the students of materials science knowledge by the inquiry learning new knowledge, hierarchical practice and the freedom of assignments. The teachers can link of knowledge and new knowledge from participating in the generation of new knowledge; The teachers help students from standing in “the shoulders of giants” and not on “beach” by the careful design “training”; The teachers ensure that all students get interesting on learning “Fundamentals of Materials Science” by flexible free homework
Unrevealing hardening and strengthening mechanisms in high-entropy ceramics from lattice distortion
Revealing the hardening and strengthening mechanisms is crucial for
facilitating the design of superhard and high-strength high-entropy ceramics
(HECs). Here, we take high-entropy diborides (HEB) as the prototype to
thoroughly investigate the hardening and strengthening mechanisms of HECs.
Specifically, the equiatomic 4- to 9-cation single-phase HEB ceramics
(4-9HEB) are fabricated by an ultra-fast high-temperature sintering method.
The as-fabricated 4-9HEB samples possess similar grain sizes, comparable
relative densities (up to ~98%), uniform compositions, and clean grain
boundaries without any impurities. The experimental results show that the
hardness and flexural strength of the as-fabricated 4-9HEB samples have an
increasing tendency with the increase of metal components. The first-principles
calculations find that lattice distortion is essential to the hardness and
strength of HEB. With the increase of metal components, an aggravation of
lattice distortion accompanied by B-B bond strengthening is determined,
resulting in the enhancement of the hardness and flexural strength. Moreover,
the correlation between other potential indicators and the hardness/flexural
strength of HEB has been disproved, including valence electron
concentration, electronegativity mismatch, and metallic states. Our results
unravel the hardening and strengthening mechanisms of HECs by intensifying
lattice distortion, which may provide guidance for developing superhard and
high-strength HECs
In-sample Curriculum Learning by Sequence Completion for Natural Language Generation
Curriculum learning has shown promising improvements in multiple domains by
training machine learning models from easy samples to hard ones. Previous works
which either design rules or train models for scoring the difficulty highly
rely on task-specific expertise, and cannot generalize. Inspired by the
``easy-to-hard'' intuition, we propose to do in-sample curriculum learning for
natural language generation tasks. Our learning strategy starts training the
model to generate the last few words, i.e., do sequence completion, and
gradually extends to generate the whole output sequence. Comprehensive
experiments show that it generalizes well to different tasks and achieves
significant improvements over strong baselines
- …