92,217 research outputs found
Transfer Learning for Speech and Language Processing
Transfer learning is a vital technique that generalizes models trained for
one setting or task to other settings or tasks. For example in speech
recognition, an acoustic model trained for one language can be used to
recognize speech in another language, with little or no re-training data.
Transfer learning is closely related to multi-task learning (cross-lingual vs.
multilingual), and is traditionally studied in the name of `model adaptation'.
Recent advance in deep learning shows that transfer learning becomes much
easier and more effective with high-level abstract features learned by deep
models, and the `transfer' can be conducted not only between data distributions
and data types, but also between model structures (e.g., shallow nets and deep
nets) or even model types (e.g., Bayesian models and neural models). This
review paper summarizes some recent prominent research towards this direction,
particularly for speech and language processing. We also report some results
from our group and highlight the potential of this very interesting research
field.Comment: 13 pages, APSIPA 201
Recent Advances in Transfer Learning for Cross-Dataset Visual Recognition: A Problem-Oriented Perspective
This paper takes a problem-oriented perspective and presents a comprehensive
review of transfer learning methods, both shallow and deep, for cross-dataset
visual recognition. Specifically, it categorises the cross-dataset recognition
into seventeen problems based on a set of carefully chosen data and label
attributes. Such a problem-oriented taxonomy has allowed us to examine how
different transfer learning approaches tackle each problem and how well each
problem has been researched to date. The comprehensive problem-oriented review
of the advances in transfer learning with respect to the problem has not only
revealed the challenges in transfer learning for visual recognition, but also
the problems (e.g. eight of the seventeen problems) that have been scarcely
studied. This survey not only presents an up-to-date technical review for
researchers, but also a systematic approach and a reference for a machine
learning practitioner to categorise a real problem and to look up for a
possible solution accordingly
Deep transfer learning for partial differential equations under conditional shift with DeepONet
Traditional machine learning algorithms are designed to learn in isolation,
i.e. address single tasks. The core idea of transfer learning (TL) is that
knowledge gained in learning to perform one task (source) can be leveraged to
improve learning performance in a related, but different, task (target). TL
leverages and transfers previously acquired knowledge to address the expense of
data acquisition and labeling, potential computational power limitations, and
the dataset distribution mismatches. Although significant progress has been
made in the fields of image processing, speech recognition, and natural
language processing (for classification and regression) for TL, little work has
been done in the field of scientific machine learning for functional regression
and uncertainty quantification in partial differential equations. In this work,
we propose a novel TL framework for task-specific learning under conditional
shift with a deep operator network (DeepONet). Inspired by the conditional
embedding operator theory, we measure the statistical distance between the
source domain and the target feature domain by embedding conditional
distributions onto a reproducing kernel Hilbert space. Task-specific operator
learning is accomplished by fine-tuning task-specific layers of the target
DeepONet using a hybrid loss function that allows for the matching of
individual target samples while also preserving the global properties of the
conditional distribution of target data. We demonstrate the advantages of our
approach for various TL scenarios involving nonlinear PDEs under conditional
shift. Our results include geometry domain adaptation and show that the
proposed TL framework enables fast and efficient multi-task operator learning,
despite significant differences between the source and target domains.Comment: 19 pages, 3 figure
Transfer Learning via Contextual Invariants for One-to-Many Cross-Domain Recommendation
The rapid proliferation of new users and items on the social web has
aggravated the gray-sheep user/long-tail item challenge in recommender systems.
Historically, cross-domain co-clustering methods have successfully leveraged
shared users and items across dense and sparse domains to improve inference
quality. However, they rely on shared rating data and cannot scale to multiple
sparse target domains (i.e., the one-to-many transfer setting). This, combined
with the increasing adoption of neural recommender architectures, motivates us
to develop scalable neural layer-transfer approaches for cross-domain learning.
Our key intuition is to guide neural collaborative filtering with
domain-invariant components shared across the dense and sparse domains,
improving the user and item representations learned in the sparse domains. We
leverage contextual invariances across domains to develop these shared modules,
and demonstrate that with user-item interaction context, we can learn-to-learn
informative representation spaces even with sparse interaction data. We show
the effectiveness and scalability of our approach on two public datasets and a
massive transaction dataset from Visa, a global payments technology company
(19% Item Recall, 3x faster vs. training separate models for each domain). Our
approach is applicable to both implicit and explicit feedback settings.Comment: SIGIR 202
Adaptive mesh refinement techniques for high-order finite-volume WENO schemes
This paper demonstrates the capabilities of Adaptive Mesh Refinement Techniques (AMR) on 2D hybrid unstructured meshes, for high order finite volume WENO methods. The AMR technique developed is a conformal adapting unstructured hybrid quadrilaterals and triangles (quads & tris) technique for resolving sharp flow features in accurate manner for steady-state and time dependent flow problems. In this method, the mesh can be refined or coarsened which depends on an error estimator, making decision at the parent level whilst maintaining a conformal mesh, the unstructured hybrid mesh refinement is done hierarchically.When a numerical method can work on a fixed conformal mesh this can be applied to do dynamic mesh adaptation. Two Refinement strategies have been devised both following a H-P refinement technique, which can be applied for providing better resolution to strong gradient dominated problems. The AMR algorithm has been tested on cylindrical explosion test and forward facing step problems
- …