4 research outputs found
Sharp Oracle Inequalities for Aggregation of Affine Estimators
We consider the problem of combining a (possibly uncountably infinite) set of
affine estimators in non-parametric regression model with heteroscedastic
Gaussian noise. Focusing on the exponentially weighted aggregate, we prove a
PAC-Bayesian type inequality that leads to sharp oracle inequalities in
discrete but also in continuous settings. The framework is general enough to
cover the combinations of various procedures such as least square regression,
kernel ridge regression, shrinking estimators and many other estimators used in
the literature on statistical inverse problems. As a consequence, we show that
the proposed aggregate provides an adaptive estimator in the exact minimax
sense without neither discretizing the range of tuning parameters nor splitting
the set of observations. We also illustrate numerically the good performance
achieved by the exponentially weighted aggregate
Causal Q-Aggregation for CATE Model Selection
Accurate estimation of conditional average treatment effects (CATE) is at the
core of personalized decision making. While there is a plethora of models for
CATE estimation, model selection is a nontrivial task, due to the fundamental
problem of causal inference. Recent empirical work provides evidence in favor
of proxy loss metrics with double robust properties and in favor of model
ensembling. However, theoretical understanding is lacking. Direct application
of prior theoretical work leads to suboptimal oracle model selection rates due
to the non-convexity of the model selection problem. We provide regret rates
for the major existing CATE ensembling approaches and propose a new CATE model
ensembling approach based on Q-aggregation using the doubly robust loss. Our
main result shows that causal Q-aggregation achieves statistically optimal
oracle model selection regret rates of (with models and
samples), with the addition of higher-order estimation error terms related
to products of errors in the nuisance functions. Crucially, our regret rate
does not require that any of the candidate CATE models be close to the truth.
We validate our new method on many semi-synthetic datasets and also provide
extensions of our work to CATE model selection with instrumental variables and
unobserved confounding.Comment: The main text is 9 pages, and we include the Appendix at the end
(totaling 51 pages
Causal Inference under Data Restrictions
This dissertation focuses on modern causal inference under uncertainty and
data restrictions, with applications to neoadjuvant clinical trials,
distributed data networks, and robust individualized decision making.
In the first project, we propose a method under the principal stratification
framework to identify and estimate the average treatment effects on a binary
outcome, conditional on the counterfactual status of a post-treatment
intermediate response. Under mild assumptions, the treatment effect of interest
can be identified. We extend the approach to address censored outcome data. The
proposed method is applied to a neoadjuvant clinical trial and its performance
is evaluated via simulation studies.
In the second project, we propose a tree-based model averaging approach to
improve the estimation accuracy of conditional average treatment effects at a
target site by leveraging models derived from other potentially heterogeneous
sites, without them sharing subject-level data. The performance of this
approach is demonstrated by a study of the causal effects of oxygen therapy on
hospital survival rates and backed up by comprehensive simulations.
In the third project, we propose a robust individualized decision learning
framework with sensitive variables to improve the worst-case outcomes of
individuals caused by sensitive variables that are unavailable at the time of
decision. Unlike most existing work that uses mean-optimal objectives, we
propose a robust learning framework by finding a newly defined quantile- or
infimum-optimal decision rule. From a causal perspective, we also generalize
the classic notion of (average) fairness to conditional fairness for individual
subjects. The reliable performance of the proposed method is demonstrated
through synthetic experiments and three real-data applications.Comment: PhD dissertation, University of Pittsburgh. The contents are mostly
based on arXiv:2211.06569, arXiv:2103.06261 and arXiv:2103.04175 with
extended discussion