589 research outputs found
AdaER: An Adaptive Experience Replay Approach for Continual Lifelong Learning
Continual lifelong learning is an machine learning framework inspired by
human learning, where learners are trained to continuously acquire new
knowledge in a sequential manner. However, the non-stationary nature of
streaming training data poses a significant challenge known as catastrophic
forgetting, which refers to the rapid forgetting of previously learned
knowledge when new tasks are introduced. While some approaches, such as
experience replay (ER), have been proposed to mitigate this issue, their
performance remains limited, particularly in the class-incremental scenario
which is considered natural and highly challenging. In this paper, we present a
novel algorithm, called adaptive-experience replay (AdaER), to address the
challenge of continual lifelong learning. AdaER consists of two stages: memory
replay and memory update. In the memory replay stage, AdaER introduces a
contextually-cued memory recall (C-CMR) strategy, which selectively replays
memories that are most conflicting with the current input data in terms of both
data and task. Additionally, AdaER incorporates an entropy-balanced reservoir
sampling (E-BRS) strategy to enhance the performance of the memory buffer by
maximizing information entropy. To evaluate the effectiveness of AdaER, we
conduct experiments on established supervised continual lifelong learning
benchmarks, specifically focusing on class-incremental learning scenarios. The
results demonstrate that AdaER outperforms existing continual lifelong learning
baselines, highlighting its efficacy in mitigating catastrophic forgetting and
improving learning performance.Comment: 18 pages, 26 figure
Information recovery in the Hayden-Preskill protocol
We revisit information retrieval from evaporating black holes in the
Hayden-Preskill protocol, treating the black hole dynamics as Haar-random. We
compute, down to the first exponentially suppressed terms, all integer-indexed
R\'enyi mutual informations between a black hole, its radiation, and a
reference that catalogues Alice's diaries. We find that dropping a diary into a
young black hole effectively delays the Page time. We also compute the
radiation : diary reflected R\'enyi entropies, and identify a technical reason
why they cannot be continued to the reflected entropy by the replica trick.Comment: 24 pages plus appendice
Reducing Sensitivity on Speaker Names for Text Generation from Dialogues
Changing speaker names consistently throughout a dialogue should not affect
its meaning and corresponding outputs for text generation from dialogues.
However, pre-trained language models, serving as the backbone for
dialogue-processing tasks, have shown to be sensitive to nuances. This may
result in unfairness in real-world applications. No comprehensive analysis of
this problem has been done in the past. In this work, we propose to
quantitatively measure a model's sensitivity on speaker names, and
comprehensively evaluate a number of known methods for reducing speaker name
sensitivity, including a novel approach of our own. Extensive experiments on
multiple datasets provide a benchmark for this problem and show the favorable
performance of our approach in sensitivity reduction and quality of generation.Comment: findings of ACL'2
Knowledge Represent and Reconstruction by “Fundamentals of Materials Science” Classroom Teaching Mode Reform
AbstractClassroom teaching is the main form of teaching organization and activity way, and is also the main base on the classroom teaching mode reform. This article by “Fundamentals of Materials Science” as an example, generalizing the knowledge representation of three types and advantages in the classroom teaching, points out that the teacher's role in this progresss. We analyze that the feasibility and the ideal effect on rebuilding the students of materials science knowledge by the inquiry learning new knowledge, hierarchical practice and the freedom of assignments. The teachers can link of knowledge and new knowledge from participating in the generation of new knowledge; The teachers help students from standing in “the shoulders of giants” and not on “beach” by the careful design “training”; The teachers ensure that all students get interesting on learning “Fundamentals of Materials Science” by flexible free homework
In-sample Curriculum Learning by Sequence Completion for Natural Language Generation
Curriculum learning has shown promising improvements in multiple domains by
training machine learning models from easy samples to hard ones. Previous works
which either design rules or train models for scoring the difficulty highly
rely on task-specific expertise, and cannot generalize. Inspired by the
``easy-to-hard'' intuition, we propose to do in-sample curriculum learning for
natural language generation tasks. Our learning strategy starts training the
model to generate the last few words, i.e., do sequence completion, and
gradually extends to generate the whole output sequence. Comprehensive
experiments show that it generalizes well to different tasks and achieves
significant improvements over strong baselines
Experimental investigation on plugging performance of nanospheres in low-permeability reservoir with bottom water
The oil production rate decreases rapidly after a short period of high yield from acidizing or fracturing in low-permeability reservoirs. In this paper, nanospheres are applied before the fracturing step, which possess the ability to absorb water and expand in the water layer, reducing the flow capacity of bottom water and finally enhancing the oil recovery. The plugging performance is investigated by nanosphere displacement experiments in cores and sand-packs, which explores the plugging effect in the oil layer, the oil-water transition zones, the water layer and the fracturing zones. In addition, a nuclear magnetic resonance experiment is conducted to study the flow mechanism of nanospheres and determine the plugging rates, which can characterize the plugging performance of nanospheres in porous media. The results show that the plugging rate is 85.84% and 78.65% on the water layer and oil-water transition zone, respectively, and 94.36% in the fracturing zone. Meanwhile, the nanospheres cannot plug the oil layer. The formation pressure has a less considerable effect on the plugging performance of nanospheres. The nanospheres have good injectivity, and the intensity variations in small, medium and large pores account for 34.46%, 13.22% and 52.32%, respectively. Overall, this paper explores the feasibility of applying nanospheres for water plugging and enhanced oil recovery.Cited as: Tang, M., Wang, C., Deng, X., Yang, H., Lu, J., Yu, H. Experimental investigation on plugging performance of nanospheres in low-permeability reservoir with bottom water. Advances in Geo-Energy Research, 2022, 6(2): 95-103. https://doi.org/10.46690/ager.2022.02.0
- …