Counterfactual explanations utilize feature perturbations to analyze the
outcome of an original decision and recommend an actionable recourse. We argue
that it is beneficial to provide several alternative explanations rather than a
single point solution and propose a probabilistic paradigm to estimate a
diverse set of counterfactuals. Specifically, we treat the perturbations as
random variables endowed with prior distribution functions. This allows
sampling multiple counterfactuals from the posterior density, with the added
benefit of incorporating inductive biases, preserving domain specific
constraints and quantifying uncertainty in estimates. More importantly, we
leverage Bayesian hierarchical modeling to share information across different
subgroups of a population, which can both improve robustness and measure
fairness. A gradient based sampler with superior convergence characteristics
efficiently computes the posterior samples. Experiments across several datasets
demonstrate that the counterfactuals estimated using our approach are valid,
sparse, diverse and feasible