239,210 research outputs found
Learning Independent Causal Mechanisms
Statistical learning relies upon data sampled from a distribution, and we
usually do not care what actually generated it in the first place. From the
point of view of causal modeling, the structure of each distribution is induced
by physical mechanisms that give rise to dependences between observables.
Mechanisms, however, can be meaningful autonomous modules of generative models
that make sense beyond a particular entailed data distribution, lending
themselves to transfer between problems. We develop an algorithm to recover a
set of independent (inverse) mechanisms from a set of transformed data points.
The approach is unsupervised and based on a set of experts that compete for
data generated by the mechanisms, driving specialization. We analyze the
proposed method in a series of experiments on image data. Each expert learns to
map a subset of the transformed data back to a reference distribution. The
learned mechanisms generalize to novel domains. We discuss implications for
transfer learning and links to recent trends in generative modeling.Comment: ICML 201
Learning Causally Disentangled Representations via the Principle of Independent Causal Mechanisms
Learning disentangled causal representations is a challenging problem that
has gained significant attention recently due to its implications for
extracting meaningful information for downstream tasks. In this work, we define
a new notion of causal disentanglement from the perspective of independent
causal mechanisms. We propose ICM-VAE, a framework for learning causally
disentangled representations supervised by causally related observed labels. We
model causal mechanisms using learnable flow-based diffeomorphic functions to
map noise variables to latent causal variables. Further, to promote the
disentanglement of causal factors, we propose a causal disentanglement prior
that utilizes the known causal structure to encourage learning a causally
factorized distribution in the latent space. Under relatively mild conditions,
we provide theoretical results showing the identifiability of causal factors
and mechanisms up to permutation and elementwise reparameterization. We
empirically demonstrate that our framework induces highly disentangled causal
factors, improves interventional robustness, and is compatible with
counterfactual generation
Causal Component Analysis
Independent Component Analysis (ICA) aims to recover independent latent
variables from observed mixtures thereof. Causal Representation Learning (CRL)
aims instead to infer causally related (thus often statistically dependent)
latent variables, together with the unknown graph encoding their causal
relationships. We introduce an intermediate problem termed Causal Component
Analysis (CauCA). CauCA can be viewed as a generalization of ICA, modelling the
causal dependence among the latent components, and as a special case of CRL. In
contrast to CRL, it presupposes knowledge of the causal graph, focusing solely
on learning the unmixing function and the causal mechanisms. Any impossibility
results regarding the recovery of the ground truth in CauCA also apply for CRL,
while possibility results may serve as a stepping stone for extensions to CRL.
We characterize CauCA identifiability from multiple datasets generated through
different types of interventions on the latent causal variables. As a
corollary, this interventional perspective also leads to new identifiability
results for nonlinear ICA -- a special case of CauCA with an empty graph --
requiring strictly fewer datasets than previous results. We introduce a
likelihood-based approach using normalizing flows to estimate both the unmixing
function and the causal mechanisms, and demonstrate its effectiveness through
extensive synthetic experiments in the CauCA and ICA setting
Semi-Supervised Learning, Causality and the Conditional Cluster Assumption
While the success of semi-supervised learning (SSL) is still not fully
understood, Sch\"olkopf et al. (2012) have established a link to the principle
of independent causal mechanisms. They conclude that SSL should be impossible
when predicting a target variable from its causes, but possible when predicting
it from its effects. Since both these cases are somewhat restrictive, we extend
their work by considering classification using cause and effect features at the
same time, such as predicting disease from both risk factors and symptoms.
While standard SSL exploits information contained in the marginal distribution
of all inputs (to improve the estimate of the conditional distribution of the
target given inputs), we argue that in our more general setting we should use
information in the conditional distribution of effect features given causal
features. We explore how this insight generalises the previous understanding,
and how it relates to and can be exploited algorithmically for SSL.Comment: 36th Conference on Uncertainty in Artificial Intelligence (2020)
(Previously presented at the NeurIPS 2019 workshop "Do the right thing":
machine learning and causal inference for improved decision making,
Vancouver, Canada.
Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learning
Recent years have seen a surge of interest in learning high-level causal
representations from low-level image pairs under interventions. Yet, existing
efforts are largely limited to simple synthetic settings that are far away from
real-world problems. In this paper, we present Causal Triplet, a causal
representation learning benchmark featuring not only visually more complex
scenes, but also two crucial desiderata commonly overlooked in previous works:
(i) an actionable counterfactual setting, where only certain object-level
variables allow for counterfactual observations whereas others do not; (ii) an
interventional downstream task with an emphasis on out-of-distribution
robustness from the independent causal mechanisms principle. Through extensive
experiments, we find that models built with the knowledge of disentangled or
object-centric representations significantly outperform their distributed
counterparts. However, recent causal representation learning methods still
struggle to identify such latent structures, indicating substantial challenges
and opportunities for future work. Our code and datasets will be available at
https://sites.google.com/view/causaltriplet.Comment: Conference on Causal Learning and Reasoning (CLeaR) 202
- …