Causal inference plays a vital role in diverse domains like epidemiology,
healthcare, and economics. De-confounding and counterfactual prediction in
observational data has emerged as a prominent concern in causal inference
research. While existing models tackle observed confounders, the presence of
unobserved confounders remains a significant challenge, distorting causal
inference and impacting counterfactual outcome accuracy. To address this, we
propose a novel variational learning model of unobserved confounders for
counterfactual inference (VLUCI), which generates the posterior distribution of
unobserved confounders. VLUCI relaxes the unconfoundedness assumption often
overlooked by most causal inference methods. By disentangling observed and
unobserved confounders, VLUCI constructs a doubly variational inference model
to approximate the distribution of unobserved confounders, which are used for
inferring more accurate counterfactual outcomes. Extensive experiments on
synthetic and semi-synthetic datasets demonstrate VLUCI's superior performance
in inferring unobserved confounders. It is compatible with state-of-the-art
counterfactual inference models, significantly improving inference accuracy at
both group and individual levels. Additionally, VLUCI provides confidence
intervals for counterfactual outcomes, aiding decision-making in risk-sensitive
domains. We further clarify the considerations when applying VLUCI to cases
where unobserved confounders don't strictly conform to our model assumptions
using the public IHDP dataset as an example, highlighting the practical
advantages of VLUCI.Comment: 15 pages, 8 figure