Causal Sensitivity Analysis for Decision Trees

Ventilator assignments in the pediatric intensive care unit (PICU) are made by medical experts; however, for some patients the relationship between ventilator assignment and patient health status is not well understood. Using observational data collected by Virtual PICU Systems (VPS) (58,772 PICU visits with covariates and different ventilator assignments conducted by clinicians), we attempt to identify which patients would derive the greatest clinical benefit from ventilators by providing a concise model to help clinicians estimate a ventilator's potential effect on individual patients, in the event that patients need to be prioritized due to limited ventilator availability. Effectively allocating ventilators requires estimating the effect of ventilation on different patients; this is known as individual treatment effect estimation. However, we only have access to non-randomized data, which is confounded by the fact that sicker patients are more likely to be ventilated. In order to reduce bias due to potential confounding to estimate the average treatment effect, propensity score matching has been widely studied and applied to estimate the average treatment effect, which matches patients from treated group with patients from control group based on similar conditional probability of ventilator assignment given an individual patient's features. This matching process assumes no unmeasured confounding, meaning there must be no unobserved covariates influencing both treatment assignment and patient's outcome. However, this is not guaranteed to be true, and if it is not, the average treatment effect estimation using propensity score matching approach can be fragile given an unmeasured confounder with strong influences. Rosenbaum and Dual Sensitivity Analysis is specifically designed for potential unmeasured confounder problems in propensity score matching, assuming confounder's existence it evaluates how "sensitive" the treatment effect estimation after matching can be. This sensitivity analysis method has been well-studied to evaluate the estimated average treatment effect based on propensity score matching, specifically, using generalized linear models as the propensity score model. However, both estimating treatment effect via propensity score matching and its sensitivity analysis have their limitations: first, propensity score matching only helps in estimating the average treatment effect, while it does not provide much information about individual treatment effect on each patient; second, Rosenbaum and Dual Sensitivity Analysis only evaluates the robustness of estimated average treatment effect from propensity score matching, while it cannot evaluate the robustness of a complex model estimating the individual treatment effect, such as a decision tree model. To solve this problem, we attempt to estimate the individual treatment effect from observational study, by proposing the treatment effect tree (TET) model. TET can be estimated through learning a Node-Level-Stabilizing decision tree based on matched pairs from potential outcome matching, which is a matching approach inspired by propensity score matching. With synthetic data generated to mimic the real-world clinical setting, we show that TET performs very well in estimating individual treatment effect, and the structure of TET can be estimated by conducting potential outcome matching in observational data. There is a matching process in TET estimation, and to evaluate the robustness of the estimated TET learned through potential outcome matching in observational data, we propose an empirical sensitivity analysis method to show how sensitive the estimated TET's structure and predictive power can be in situations with strong levels of confounding described by Rosenbaum and Dual Sensitivity Analysis. We use the same synthetic dataset with different levels of confounding encoded as boolean confounders to experiment with this sensitivity analysis method. We show the experimental results of estimating TET from observational data, as well as their performances in sensitivity analysis. The experimental results show that with strong covariates setting, the estimated TET from observational data can be very stable against strong levels of confounding described by Rosenbaum and Dual Sensitivity Analysis encoded as boolean confounders. In this work, we propose TET model for individual treatment effect estimation with observational data, we show that TET can be learned from matching individuals based on potential outcome. We designed an empirical sensitivity analysis method to evaluate the robustness of TET with different levels of confounding described by Rosenbaum and Dual Sensitivity Analysis, and the experimental results show the learned TET can be stable against strong levels of confounding

Invariant Causal Set Covering Machines

Rule-based models, such as decision trees, appeal to practitioners due to their interpretable nature. However, the learning algorithms that produce such models are often vulnerable to spurious associations and thus, they are not guaranteed to extract causally-relevant insights. In this work, we build on ideas from the invariant causal prediction literature to propose Invariant Causal Set Covering Machines, an extension of the classical Set Covering Machine algorithm for conjunctions/disjunctions of binary-valued rules that provably avoids spurious associations. We demonstrate both theoretically and empirically that our method can identify the causal parents of a variable of interest in polynomial time

On the Pointwise Behavior of Recursive Partitioning and Its Implications for Heterogeneous Causal Effect Estimation

Decision tree learning is increasingly being used for pointwise inference. Important applications include causal heterogenous treatment effects and dynamic policy decisions, as well as conditional quantile regression and design of experiments, where tree estimation and inference is conducted at specific values of the covariates. In this paper, we call into question the use of decision trees (trained by adaptive recursive partitioning) for such purposes by demonstrating that they can fail to achieve polynomial rates of convergence in uniform norm, even with pruning. Instead, the convergence may be poly-logarithmic or, in some important special cases, such as honest regression trees, fail completely. We show that random forests can remedy the situation, turning poor performing trees into nearly optimal procedures, at the cost of losing interpretability and introducing two additional tuning parameters. The two hallmarks of random forests, subsampling and the random feature selection mechanism, are seen to each distinctively contribute to achieving nearly optimal performance for the model class considered