1,874 research outputs found
Large Margin Multiclass Gaussian Classification with Differential Privacy
As increasing amounts of sensitive personal information is aggregated into
data repositories, it has become important to develop mechanisms for processing
the data without revealing information about individual data instances. The
differential privacy model provides a framework for the development and
theoretical analysis of such mechanisms. In this paper, we propose an algorithm
for learning a discriminatively trained multi-class Gaussian classifier that
satisfies differential privacy using a large margin loss function with a
perturbed regularization term. We present a theoretical upper bound on the
excess risk of the classifier introduced by the perturbation.Comment: 14 page
Mitigating Group Bias in Federated Learning for Heterogeneous Devices
Federated Learning is emerging as a privacy-preserving model training
approach in distributed edge applications. As such, most edge deployments are
heterogeneous in nature i.e., their sensing capabilities and environments vary
across deployments. This edge heterogeneity violates the independence and
identical distribution (IID) property of local data across clients and produces
biased global models i.e. models that contribute to unfair decision-making and
discrimination against a particular community or a group. Existing bias
mitigation techniques only focus on bias generated from label heterogeneity in
non-IID data without accounting for domain variations due to feature
heterogeneity and do not address global group-fairness property.
Our work proposes a group-fair FL framework that minimizes group-bias while
preserving privacy and without resource utilization overhead. Our main idea is
to leverage average conditional probabilities to compute a cross-domain group
\textit{importance weights} derived from heterogeneous training data to
optimize the performance of the worst-performing group using a modified
multiplicative weights update method. Additionally, we propose regularization
techniques to minimize the difference between the worst and best-performing
groups while making sure through our thresholding mechanism to strike a balance
between bias reduction and group performance degradation. Our evaluation of
human emotion recognition and image classification benchmarks assesses the fair
decision-making of our framework in real-world heterogeneous settings
On PAC Learning Halfspaces in Non-interactive Local Privacy Model with Public Unlabeled Data
In this paper, we study the problem of PAC learning halfspaces in the
non-interactive local differential privacy model (NLDP). To breach the barrier
of exponential sample complexity, previous results studied a relaxed setting
where the server has access to some additional public but unlabeled data. We
continue in this direction. Specifically, we consider the problem under the
standard setting instead of the large margin setting studied before. Under
different mild assumptions on the underlying data distribution, we propose two
approaches that are based on the Massart noise model and self-supervised
learning and show that it is possible to achieve sample complexities that are
only linear in the dimension and polynomial in other terms for both private and
public data, which significantly improve the previous results. Our methods
could also be used for other private PAC learning problems.Comment: To appear in The 14th Asian Conference on Machine Learning (ACML
2022
Statistical Active Learning Algorithms for Noise Tolerance and Differential Privacy
We describe a framework for designing efficient active learning algorithms
that are tolerant to random classification noise and are
differentially-private. The framework is based on active learning algorithms
that are statistical in the sense that they rely on estimates of expectations
of functions of filtered random examples. It builds on the powerful statistical
query framework of Kearns (1993).
We show that any efficient active statistical learning algorithm can be
automatically converted to an efficient active learning algorithm which is
tolerant to random classification noise as well as other forms of
"uncorrelated" noise. The complexity of the resulting algorithms has
information-theoretically optimal quadratic dependence on , where
is the noise rate.
We show that commonly studied concept classes including thresholds,
rectangles, and linear separators can be efficiently actively learned in our
framework. These results combined with our generic conversion lead to the first
computationally-efficient algorithms for actively learning some of these
concept classes in the presence of random classification noise that provide
exponential improvement in the dependence on the error over their
passive counterparts. In addition, we show that our algorithms can be
automatically converted to efficient active differentially-private algorithms.
This leads to the first differentially-private active learning algorithms with
exponential label savings over the passive case.Comment: Extended abstract appears in NIPS 201
- …