56 research outputs found
Nearly optimal Bayesian Shrinkage for High Dimensional Regression
During the past decade, shrinkage priors have received much attention in
Bayesian analysis of high-dimensional data. In this paper, we study the problem
for high-dimensional linear regression models. We show that if the shrinkage
prior has a heavy and flat tail, and allocates a sufficiently large probability
mass in a very small neighborhood of zero, then its posterior properties are as
good as those of the spike-and-slab prior. While enjoying its efficiency in
Bayesian computation, the shrinkage prior can lead to a nearly optimal
contraction rate and selection consistency as the spike-and-slab prior. Our
numerical results show that under posterior consistency, Bayesian methods can
yield much better results in variable selection than the regularization
methods, such as Lasso and SCAD. We also establish a Bernstein von-Mises type
results comparable to Castillo et al (2015), this result leads to a convenient
way to quantify uncertainties of the regression coefficient estimates, which
has been beyond the ability of regularization methods
Optimal False Discovery Control of Minimax Estimator
In the analysis of high dimensional regression models, there are two
important objectives: statistical estimation and variable selection. In
literature, most works focus on either optimal estimation, e.g., minimax
error, or optimal selection behavior, e.g., minimax Hamming loss. However in
this study, we investigate the subtle interplay between the estimation accuracy
and selection behavior. Our result shows that an estimator's error rate
critically depends on its performance of type I error control. Essentially, the
minimax convergence rate of false discovery rate over all rate-minimax
estimators is a polynomial of the true sparsity ratio. This result helps us to
characterize the false positive control of rate-optimal estimators under
different sparsity regimes. More specifically, under near-linear sparsity, the
number of yielded false positives always explodes to infinity under worst
scenario, but the false discovery rate still converges to 0; under linear
sparsity, even the false discovery rate doesn't asymptotically converge to 0.
On the other side, in order to asymptotically eliminate all false discoveries,
the estimator must be sub-optimal in terms of its convergence rate. This work
attempts to offer rigorous analysis on the incompatibility phenomenon between
selection consistency and rate-minimaxity observed in the high dimensional
regression literature
Fair Supervised Learning with A Simple Random Sampler of Sensitive Attributes
As the data-driven decision process becomes dominating for industrial
applications, fairness-aware machine learning arouses great attention in
various areas. This work proposes fairness penalties learned by neural networks
with a simple random sampler of sensitive attributes for non-discriminatory
supervised learning. In contrast to many existing works that critically rely on
the discreteness of sensitive attributes and response variables, the proposed
penalty is able to handle versatile formats of the sensitive attributes, so it
is more extensively applicable in practice than many existing algorithms. This
penalty enables us to build a computationally efficient group-level
in-processing fairness-aware training framework. Empirical evidence shows that
our framework enjoys better utility and fairness measures on popular benchmark
data sets than competing methods. We also theoretically characterize estimation
errors and loss of utility of the proposed neural-penalized risk minimization
problem
Personalized Federated X -armed Bandit
In this work, we study the personalized federated -armed bandit
problem, where the heterogeneous local objectives of the clients are optimized
simultaneously in the federated learning paradigm. We propose the
\texttt{PF-PNE} algorithm with a unique double elimination strategy, which
safely eliminates the non-optimal regions while encouraging federated
collaboration through biased but effective evaluations of the local objectives.
The proposed \texttt{PF-PNE} algorithm is able to optimize local objectives
with arbitrary levels of heterogeneity, and its limited communications protects
the confidentiality of the client-wise reward data. Our theoretical analysis
shows the benefit of the proposed algorithm over single-client algorithms.
Experimentally, \texttt{PF-PNE} outperforms multiple baselines on both
synthetic and real life datasets
- β¦