3 research outputs found
Differentially Private Empirical Risk Minimization with Input Perturbation
We propose a novel framework for the differentially private ERM, input
perturbation. Existing differentially private ERM implicitly assumed that the
data contributors submit their private data to a database expecting that the
database invokes a differentially private mechanism for publication of the
learned model. In input perturbation, each data contributor independently
randomizes her/his data by itself and submits the perturbed data to the
database. We show that the input perturbation framework theoretically
guarantees that the model learned with the randomized data eventually satisfies
differential privacy with the prescribed privacy parameters. At the same time,
input perturbation guarantees that local differential privacy is guaranteed to
the server. We also show that the excess risk bound of the model learned with
input perturbation is under a certain condition, where is the
sample size. This is the same as the excess risk bound of the state-of-the-art.Comment: 22 pages, 4 figure
Differentially Private Convex Optimization with Feasibility Guarantees
This paper develops a novel differentially private framework to solve convex
optimization problems with sensitive optimization data and complex physical or
operational constraints. Unlike standard noise-additive algorithms, that act
primarily on the problem data, objective or solution, and disregard the problem
constraints, this framework requires the optimization variables to be a
function of the noise and exploits a chance-constrained problem reformulation
with formal feasibility guarantees. The noise is calibrated to provide
differential privacy for identity and linear queries on the optimization
solution. For many applications, including resource allocation problems, the
proposed framework provides a trade-off between the expected optimality loss
and the variance of optimization results
Input Perturbation: A New Paradigm between Central and Local Differential Privacy
Traditionally, there are two models on differential privacy: the central
model and the local model. The central model focuses on the machine learning
model and the local model focuses on the training data. In this paper, we study
the \textit{input perturbation} method in differentially private empirical risk
minimization (DP-ERM), preserving privacy of the central model. By adding noise
to the original training data and training with the `perturbed data', we
achieve (,)-differential privacy on the final model, along
with some kind of privacy on the original data. We observe that there is an
interesting connection between the local model and the central model: the
perturbation on the original data causes the perturbation on the gradient, and
finally the model parameters. This observation means that our method builds a
bridge between local and central model, protecting the data, the gradient and
the model simultaneously, which is more superior than previous central methods.
Detailed theoretical analysis and experiments show that our method achieves
almost the same (or even better) performance as some of the best previous
central methods with more protections on privacy, which is an attractive
result. Moreover, we extend our method to a more general case: the loss
function satisfies the Polyak-Lojasiewicz condition, which is more general than
strong convexity, the constraint on the loss function in most previous work