8,134 research outputs found
Lifelong learning of concepts in CRAFT
La planification à des niveaux d’abstraction plus élevés est essentielle lorsqu’il s’agit de
résoudre des tâches à long horizon avec des complexités hiérarchiques. Pour planifier avec
succès à un niveau d’abstraction donné, un agent doit comprendre le fonctionnement de
l’environnement à ce niveau particulier. Cette compréhension peut être implicite en termes de
politiques, de fonctions de valeur et de modèles, ou elle peut être définie explicitement. Dans
ce travail, nous introduisons les concepts comme un moyen de représenter et d’accumuler
explicitement des informations sur l’environnement.
Les concepts sont définis en termes de transition d’état et des conditions requises pour
que cette transition ait lieu. La simplicité de cette définition offre flexibilité et contrôle
sur le processus d’apprentissage. Étant donné que les concepts sont de nature hautement
interprétable, il est facile d’encoder les connaissances antérieures et d’intervenir au cours
du processus d’apprentissage si nécessaire. Cette définition facilite également le transfert
de concepts entre différents domaines. Les concepts, à un niveau d’abstraction donné, sont
intimement liés aux compétences, ou actions temporellement abstraites. Toutes les transitions
d’état suffisamment importantes pour être représentées par un concept se produisent après
l’exécution réussie d’une compétence. En exploitant cette relation, nous introduisons un
cadre qui facilite l’apprentissage tout au long de la vie et le raffinement des concepts Ă
différents niveaux d’abstraction. Le cadre comporte trois volets:
Le sytème 1 segmente un flux d’expérience (par exemple une démonstration) en
une séquence de compétences. Cette segmentation peut se faire à différents niveaux
d’abstraction.
Le sytème 2 analyse ces segments pour affiner et mettre à niveau son ensemble de
concepts, lorsqu’applicable.
Le sytème 3 utilise les concepts disponibles pour générer un graphe de dépendance de
sous-tâches. Ce graphe peut être utilisé pour planifier à différents niveaux d’abstraction.
Nous démontrons l’applicabilité de ce cadre dans l’environnement hiérarchique 2D CRAFT. Nous effectuons des expériences pour explorer comment les concepts peuvent être appris
de différents flux d’expérience et comment la qualité de la base de concepts affecte l’optimalité
du plan général. Dans les tâches avec des dépendances de sous-tâches complexes, où
la plupart des algorithmes ne parviennent pas à se généraliser ou prennent un temps impraticable
à converger, nous démontrons que les concepts peuvent être utilisés pour simplifier
considérablement la planification. Ce cadre peut également être utilisé pour comprendre
l’intention d’une démonstration donnée en termes de concepts. Cela permet à l’agent de
répliquer facilement la démonstration dans différents environnements. Nous montrons que
cette méthode d’imitation est beaucoup plus robuste aux changements de configuration de
l’environnement que les méthodes traditionnelles. Dans notre formulation du problème, nous
faisons deux hypothèses: 1) que nous avons accès à un ensemble de compétences suffisamment
exhaustif, et 2) que notre agent a accès à des environnements de pratique, qui peuvent
être utilisés pour affiner les concepts en cas de besoin. L’objectif de ce travail est d’explorer
l’aspect pratique des concepts d’apprentissage comme moyen d’améliorer la compréhension
de l’environnement. Dans l’ensemble, nous démontrons que les concepts d’apprentissagePlanning at higher levels of abstraction is critical when it comes to solving long horizon tasks with hierarchical complexities. To plan successfully at a given level of abstraction, an agent must have an understanding of how the environment functions at that particular level. This understanding may be implicit in terms of policies, value functions, and world models, or it can be defined explicitly. In this work, we introduce concepts as a means to explicitly represent and accumulate information about the environment. Concepts are defined in terms of a state transition and the conditions required for that transition to take place. The simplicity of this definition offers flexibility and control over the learning process. Since concepts are highly interpretable in nature, it is easy to encode prior knowledge and intervene during the learning process if necessary. This definition also makes it relatively straightforward to transfer concepts across different domains wherever applicable. Concepts, at a given level of abstraction, are intricately linked to skills, or temporally abstracted actions. All the state transitions significant enough to be represented by a concept occur only after the successful execution of a skill. Exploiting this relationship, we introduce a framework that aids in lifelong learning and refining of concepts across different levels of abstraction. The framework has three components: - System 1 segments a stream of experience (e.g. a demonstration) into a sequence of skills. This segmentation can be done at different levels of abstraction. - System 2 analyses these segments to refine and upgrade its set of concepts, whenever applicable. - System 3 utilises the available concepts to generate a sub-task dependency graph. This graph can be used for planning at different levels of abstraction We demonstrate the applicability of this framework in the 2D hierarchical environment CRAFT. We perform experiments to explore how concepts can be learned from different streams of experience, and how the quality of the concept base affects the optimality of the overall plan. In tasks with complex sub-task dependencies, where most algorithms fail to generalise or take an impractical amount of time to converge, we demonstrate that concepts can be used to significantly simplify planning. This framework can also be used to understand the intention of a given demonstration in terms of concepts. This makes it easy for the agent to replicate a demonstration in different environments. We show that this method of imitation is much more robust to changes in the environment configurations than traditional methods. In our problem formulation, we make two assumptions: 1) that we have access to a sufficiently exhaustive set of skills, and 2) that our agent has access to practice environments, which can be used to refine concepts when needed. The objective behind this work is to explore the practicality of learning concepts as a means to improve one’s understanding about the environment. Overall, we demonstrate that learning concepts can be a light-weight yet efficient way to increase the capability of a system
Learning Disentangled Representations in the Imaging Domain
Disentangled representation learning has been proposed as an approach to
learning general representations even in the absence of, or with limited,
supervision. A good general representation can be fine-tuned for new target
tasks using modest amounts of data, or used directly in unseen domains
achieving remarkable performance in the corresponding task. This alleviation of
the data and annotation requirements offers tantalising prospects for
applications in computer vision and healthcare. In this tutorial paper, we
motivate the need for disentangled representations, present key theory, and
detail practical building blocks and criteria for learning such
representations. We discuss applications in medical imaging and computer vision
emphasising choices made in exemplar key works. We conclude by presenting
remaining challenges and opportunities.Comment: Submitted. This paper follows a tutorial style but also surveys a
considerable (more than 200 citations) number of work
Recommended from our members
Evaluating service supply in conditional cash transfers
textConditional cash transfers are poverty reduction mechanisms that seek to increase demand of social services by combining an income effect with a health or education requirement. This demand-side strategy relies on a tacit assumption about the quality of and access to those services as a path to improve human capital outcomes. Some conditional cash transfers have included supply-side complementary incentives to ensure that services are suitable to deliver a good education and better health. This study reviews the existing evidence on the impact of supply-side incentives in the context of conditional cash transfers. The review finds that a limited number of studies estimate effects of supply in human capital outcomes and only a few impact evaluations consider the role of schools or health centers in enabling access. The evaluations revised find no evidence that supply side interventions coupled with conditional cash transfers directly improve program outcomes. Nonetheless, several studies highlight the relevance of school organization, in terms of school modalities and student/teacher ratios in school enrollment and attendance. Impact estimations as well as the implementation of the supply-side programs also signal the need for a more nuanced understanding of how school management influences a variety of schooling outcomes. In general, the small number of impact estimations and the restricted set of variables used limits the generalizability of the results. For this reason, a principal conclusion of the review is the need for further research on the topic, as well as consistency across impact measures and a more in-depth analysis of school supply and their influence on learning outcomes.Global Policy Studie
Causal Disentangled Recommendation Against User Preference Shifts
Recommender systems easily face the issue of user preference shifts. User
representations will become out-of-date and lead to inappropriate
recommendations if user preference has shifted over time. To solve the issue,
existing work focuses on learning robust representations or predicting the
shifting pattern. There lacks a comprehensive view to discover the underlying
reasons for user preference shifts. To understand the preference shift, we
abstract a causal graph to describe the generation procedure of user
interaction sequences. Assuming user preference is stable within a short
period, we abstract the interaction sequence as a set of chronological
environments. From the causal graph, we find that the changes of some
unobserved factors (e.g., becoming pregnant) cause preference shifts between
environments. Besides, the fine-grained user preference over categories
sparsely affects the interactions with different items. Inspired by the causal
graph, our key considerations to handle preference shifts lie in modeling the
interaction generation procedure by: 1) capturing the preference shifts across
environments for accurate preference prediction, and 2) disentangling the
sparse influence from user preference to interactions for accurate effect
estimation of preference. To this end, we propose a Causal Disentangled
Recommendation (CDR) framework, which captures preference shifts via a temporal
variational autoencoder and learns the sparse influence from multiple
environments. Specifically, an encoder is adopted to infer the unobserved
factors from user interactions while a decoder is to model the interaction
generation process. Besides, we introduce two learnable matrices to disentangle
the sparse influence from user preference to interactions. Lastly, we devise a
multi-objective loss to optimize CDR. Extensive experiments on three datasets
show the superiority of CDR.Comment: This paper has been accepted for publication in Transactions on
Information System
Disentangled Representation Learning
Disentangled Representation Learning (DRL) aims to learn a model capable of
identifying and disentangling the underlying factors hidden in the observable
data in representation form. The process of separating underlying factors of
variation into variables with semantic meaning benefits in learning explainable
representations of data, which imitates the meaningful understanding process of
humans when observing an object or relation. As a general learning strategy,
DRL has demonstrated its power in improving the model explainability,
controlability, robustness, as well as generalization capacity in a wide range
of scenarios such as computer vision, natural language processing, data mining
etc. In this article, we comprehensively review DRL from various aspects
including motivations, definitions, methodologies, evaluations, applications
and model designs. We discuss works on DRL based on two well-recognized
definitions, i.e., Intuitive Definition and Group Theory Definition. We further
categorize the methodologies for DRL into four groups, i.e., Traditional
Statistical Approaches, Variational Auto-encoder Based Approaches, Generative
Adversarial Networks Based Approaches, Hierarchical Approaches and Other
Approaches. We also analyze principles to design different DRL models that may
benefit different tasks in practical applications. Finally, we point out
challenges in DRL as well as potential research directions deserving future
investigations. We believe this work may provide insights for promoting the DRL
research in the community.Comment: 22 pages,9 figure
Disentangled Generative Causal Representation Learning
This paper proposes a Disentangled gEnerative cAusal Representation (DEAR)
learning method. Unlike existing disentanglement methods that enforce
independence of the latent variables, we consider the general case where the
underlying factors of interests can be causally correlated. We show that
previous methods with independent priors fail to disentangle causally
correlated factors. Motivated by this finding, we propose a new disentangled
learning method called DEAR that enables causal controllable generation and
causal representation learning. The key ingredient of this new formulation is
to use a structural causal model (SCM) as the prior for a bidirectional
generative model. The prior is then trained jointly with a generator and an
encoder using a suitable GAN loss incorporated with supervision. We provide
theoretical justification on the identifiability and asymptotic consistency of
the proposed method, which guarantees disentangled causal representation
learning under appropriate conditions. We conduct extensive experiments on both
synthesized and real data sets to demonstrate the effectiveness of DEAR in
causal controllable generation, and the benefits of the learned representations
for downstream tasks in terms of sample efficiency and distributional
robustness
Inductive Biases for Deep Learning of Higher-Level Cognition
A fascinating hypothesis is that human and animal intelligence could be
explained by a few principles (rather than an encyclopedic list of heuristics).
If that hypothesis was correct, we could more easily both understand our own
intelligence and build intelligent machines. Just like in physics, the
principles themselves would not be sufficient to predict the behavior of
complex systems like brains, and substantial computation might be needed to
simulate human-like intelligence. This hypothesis would suggest that studying
the kind of inductive biases that humans and animals exploit could help both
clarify these principles and provide inspiration for AI research and
neuroscience theories. Deep learning already exploits several key inductive
biases, and this work considers a larger list, focusing on those which concern
mostly higher-level and sequential conscious processing. The objective of
clarifying these particular principles is that they could potentially help us
build AI systems benefiting from humans' abilities in terms of flexible
out-of-distribution and systematic generalization, which is currently an area
where a large gap exists between state-of-the-art machine learning and human
intelligence.Comment: This document contains a review of authors research as part of the
requirement of AG's predoctoral exam, an overview of the main contributions
of the authors few recent papers (co-authored with several other co-authors)
as well as a vision of proposed future researc
- …