4 research outputs found
Causal Inference Using LLM-Guided Discovery
At the core of causal inference lies the challenge of determining reliable
causal graphs solely based on observational data. Since the well-known backdoor
criterion depends on the graph, any errors in the graph can propagate
downstream to effect inference. In this work, we initially show that complete
graph information is not necessary for causal effect inference; the topological
order over graph variables (causal order) alone suffices. Further, given a node
pair, causal order is easier to elicit from domain experts compared to graph
edges since determining the existence of an edge can depend extensively on
other variables. Interestingly, we find that the same principle holds for Large
Language Models (LLMs) such as GPT-3.5-turbo and GPT-4, motivating an automated
method to obtain causal order (and hence causal effect) with LLMs acting as
virtual domain experts. To this end, we employ different prompting strategies
and contextual cues to propose a robust technique of obtaining causal order
from LLMs. Acknowledging LLMs' limitations, we also study possible techniques
to integrate LLMs with established causal discovery algorithms, including
constraint-based and score-based methods, to enhance their performance.
Extensive experiments demonstrate that our approach significantly improves
causal ordering accuracy as compared to discovery algorithms, highlighting the
potential of LLMs to enhance causal inference across diverse fields
Rethinking Counterfactual Data Augmentation Under Confounding
Counterfactual data augmentation has recently emerged as a method to mitigate
confounding biases in the training data for a machine learning model. These
biases, such as spurious correlations, arise due to various observed and
unobserved confounding variables in the data generation process. In this paper,
we formally analyze how confounding biases impact downstream classifiers and
present a causal viewpoint to the solutions based on counterfactual data
augmentation. We explore how removing confounding biases serves as a means to
learn invariant features, ultimately aiding in generalization beyond the
observed data distribution. Additionally, we present a straightforward yet
powerful algorithm for generating counterfactual images, which effectively
mitigates the influence of confounding effects on downstream classifiers.
Through experiments on MNIST variants and the CelebA datasets, we demonstrate
the effectiveness and practicality of our approach
Building a Winning Team: Selecting Source Model Ensembles using a Submodular Transferability Estimation Approach
Estimating the transferability of publicly available pretrained models to a
target task has assumed an important place for transfer learning tasks in
recent years. Existing efforts propose metrics that allow a user to choose one
model from a pool of pre-trained models without having to fine-tune each model
individually and identify one explicitly. With the growth in the number of
available pre-trained models and the popularity of model ensembles, it also
becomes essential to study the transferability of multiple-source models for a
given target task. The few existing efforts study transferability in such
multi-source ensemble settings using just the outputs of the classification
layer and neglect possible domain or task mismatch. Moreover, they overlook the
most important factor while selecting the source models, viz., the cohesiveness
factor between them, which can impact the performance and confidence in the
prediction of the ensemble. To address these gaps, we propose a novel Optimal
tranSport-based suBmOdular tRaNsferability metric (OSBORN) to estimate the
transferability of an ensemble of models to a downstream task. OSBORN
collectively accounts for image domain difference, task difference, and
cohesiveness of models in the ensemble to provide reliable estimates of
transferability. We gauge the performance of OSBORN on both image
classification and semantic segmentation tasks. Our setup includes 28 source
datasets, 11 target datasets, 5 model architectures, and 2 pre-training
methods. We benchmark our method against current state-of-the-art metrics
MS-LEEP and E-LEEP, and outperform them consistently using the proposed
approach.Comment: To appear at ICCV 202
Towards Learning and Explaining Indirect Causal Effects in Neural Networks
Recently, there has been a growing interest in learning and explaining causal
effects within Neural Network (NN) models. By virtue of NN architectures,
previous approaches consider only direct and total causal effects assuming
independence among input variables. We view an NN as a structural causal model
(SCM) and extend our focus to include indirect causal effects by introducing
feedforward connections among input neurons. We propose an ante-hoc method that
captures and maintains direct, indirect, and total causal effects during NN
model training. We also propose an algorithm for quantifying learned causal
effects in an NN model and efficient approximation strategies for quantifying
causal effects in high-dimensional data. Extensive experiments conducted on
synthetic and real-world datasets demonstrate that the causal effects learned
by our ante-hoc method better approximate the ground truth effects compared to
existing methods