2,372 research outputs found
Domain Generalization by Marginal Transfer Learning
In the problem of domain generalization (DG), there are labeled training data
sets from several related prediction problems, and the goal is to make accurate
predictions on future unlabeled data sets that are not known to the learner.
This problem arises in several applications where data distributions fluctuate
because of environmental, technical, or other sources of variation. We
introduce a formal framework for DG, and argue that it can be viewed as a kind
of supervised learning problem by augmenting the original feature space with
the marginal distribution of feature vectors. While our framework has several
connections to conventional analysis of supervised learning algorithms, several
unique aspects of DG require new methods of analysis.
This work lays the learning theoretic foundations of domain generalization,
building on our earlier conference paper where the problem of DG was introduced
Blanchard et al., 2011. We present two formal models of data generation,
corresponding notions of risk, and distribution-free generalization error
analysis. By focusing our attention on kernel methods, we also provide more
quantitative results and a universally consistent algorithm. An efficient
implementation is provided for this algorithm, which is experimentally compared
to a pooling strategy on one synthetic and three real-world data sets
Lifelong Bandit Optimization: No Prior and No Regret
In practical applications, machine learning algorithms are often repeatedly
applied to problems with similar structure over and over again. We focus on
solving a sequence of bandit optimization tasks and develop LiBO, an algorithm
which adapts to the environment by learning from past experience and becoming
more sample-efficient in the process. We assume a kernelized structure where
the kernel is unknown but shared across all tasks. LiBO sequentially
meta-learns a kernel that approximates the true kernel and simultaneously
solves the incoming tasks with the latest kernel estimate. Our algorithm can be
paired with any kernelized bandit algorithm and guarantees oracle optimal
performance, meaning that as more tasks are solved, the regret of LiBO on each
task converges to the regret of the bandit algorithm with oracle knowledge of
the true kernel. Naturally, if paired with a sublinear bandit algorithm, LiBO
yields a sublinear lifelong regret. We also show that direct access to the data
from each task is not necessary for attaining sublinear regret. The lifelong
problem can thus be solved in a federated manner, while keeping the data of
each task private.Comment: 32 pages, 6 figures, preprin
Progressive growing of self-organized hierarchical representations for exploration
Designing agent that can autonomously discover and learn a diversity of
structures and skills in unknown changing environments is key for lifelong
machine learning. A central challenge is how to learn incrementally
representations in order to progressively build a map of the discovered
structures and re-use it to further explore. To address this challenge, we
identify and target several key functionalities. First, we aim to build lasting
representations and avoid catastrophic forgetting throughout the exploration
process. Secondly we aim to learn a diversity of representations allowing to
discover a "diversity of diversity" of structures (and associated skills) in
complex high-dimensional environments. Thirdly, we target representations that
can structure the agent discoveries in a coarse-to-fine manner. Finally, we
target the reuse of such representations to drive exploration toward an
"interesting" type of diversity, for instance leveraging human guidance.
Current approaches in state representation learning rely generally on
monolithic architectures which do not enable all these functionalities.
Therefore, we present a novel technique to progressively construct a Hierarchy
of Observation Latent Models for Exploration Stratification, called HOLMES.
This technique couples the use of a dynamic modular model architecture for
representation learning with intrinsically-motivated goal exploration processes
(IMGEPs). The paper shows results in the domain of automated discovery of
diverse self-organized patterns, considering as testbed the experimental
framework from Reinke et al. (2019)
- …