4 research outputs found
Using Task Descriptions in Lifelong Machine Learning for Improved Performance and Zero-Shot Transfer
Knowledge transfer between tasks can improve the performance of learned
models, but requires an accurate estimate of the inter-task relationships to
identify the relevant knowledge to transfer. These inter-task relationships are
typically estimated based on training data for each task, which is inefficient
in lifelong learning settings where the goal is to learn each consecutive task
rapidly from as little data as possible. To reduce this burden, we develop a
lifelong learning method based on coupled dictionary learning that utilizes
high-level task descriptions to model the inter-task relationships. We show
that using task descriptors improves the performance of the learned task
policies, providing both theoretical justification for the benefit and
empirical demonstration of the improvement across a variety of learning
problems. Given only the descriptor for a new task, the lifelong learner is
also able to accurately predict a model for the new task through zero-shot
learning using the coupled dictionary, eliminating the need to gather training
data before addressing the task.Comment: 28 page
Towards Lifelong Learning of End-to-end ASR
Automatic speech recognition (ASR) technologies today are primarily optimized
for given datasets; thus, any changes in the application environment (e.g.,
acoustic conditions or topic domains) may inevitably degrade the performance.
We can collect new data describing the new environment and fine-tune the
system, but this naturally leads to higher error rates for the earlier
datasets, referred to as catastrophic forgetting. The concept of lifelong
learning (LLL) aiming to enable a machine to sequentially learn new tasks from
new datasets describing the changing real world without forgetting the
previously learned knowledge is thus brought to attention. This paper reports,
to our knowledge, the first effort to extensively consider and analyze the use
of various approaches of LLL in end-to-end (E2E) ASR, including proposing novel
methods in saving data for past domains to mitigate the catastrophic forgetting
problem. An overall relative reduction of 28.7% in WER was achieved compared to
the fine-tuning baseline when sequentially learning on three very different
benchmark corpora. This can be the first step toward the highly desired ASR
technologies capable of synchronizing with the continuously changing real
world.Comment: Interspeech 2021. We acknowledge the support of Salesforce Research
Deep Learning Gran
CLOPS: Continual Learning of Physiological Signals
Deep learning algorithms are known to experience destructive interference
when instances violate the assumption of being independent and identically
distributed (i.i.d). This violation, however, is ubiquitous in clinical
settings where data are streamed temporally and from a multitude of
physiological sensors. To overcome this obstacle, we propose CLOPS, a
replay-based continual learning strategy. In three continual learning scenarios
based on three publically-available datasets, we show that CLOPS can outperform
the state-of-the-art methods, GEM and MIR. Moreover, we propose end-to-end
trainable parameters, which we term task-instance parameters, that can be used
to quantify task difficulty and similarity. This quantification yields insights
into both network interpretability and clinical applications, where task
difficulty is poorly quantified
PAC Bounds for Imitation and Model-based Batch Learning of Contextual Markov Decision Processes
We consider the problem of batch multi-task reinforcement learning with
observed context descriptors, motivated by its application to personalized
medical treatment. In particular, we study two general classes of learning
algorithms: direct policy learning (DPL), an imitation-learning based approach
which learns from expert trajectories, and model-based learning. First, we
derive sample complexity bounds for DPL, and then show that model-based
learning from expert actions can, even with a finite model class, be
impossible. After relaxing the conditions under which the model-based approach
is expected to learn by allowing for greater coverage of state-action space, we
provide sample complexity bounds for model-based learning with finite model
classes, showing that there exist model classes with sample complexity
exponential in their statistical complexity. We then derive a sample complexity
upper bound for model-based learning based on a measure of concentration of the
data distribution. Our results give formal justification for imitation learning
over model-based learning in this setting