240 research outputs found
Heavy Hitters and the Structure of Local Privacy
We present a new locally differentially private algorithm for the heavy
hitters problem which achieves optimal worst-case error as a function of all
standardly considered parameters. Prior work obtained error rates which depend
optimally on the number of users, the size of the domain, and the privacy
parameter, but depend sub-optimally on the failure probability.
We strengthen existing lower bounds on the error to incorporate the failure
probability, and show that our new upper bound is tight with respect to this
parameter as well. Our lower bound is based on a new understanding of the
structure of locally private protocols. We further develop these ideas to
obtain the following general results beyond heavy hitters.
Advanced Grouposition: In the local model, group privacy for
users degrades proportionally to , instead of linearly in
as in the central model. Stronger group privacy yields improved max-information
guarantees, as well as stronger lower bounds (via "packing arguments"), over
the central model.
Building on a transformation of Bassily and Smith (STOC 2015), we
give a generic transformation from any non-interactive approximate-private
local protocol into a pure-private local protocol. Again in contrast with the
central model, this shows that we cannot obtain more accurate algorithms by
moving from pure to approximate local privacy
Private Multiplicative Weights Beyond Linear Queries
A wide variety of fundamental data analyses in machine learning, such as
linear and logistic regression, require minimizing a convex function defined by
the data. Since the data may contain sensitive information about individuals,
and these analyses can leak that sensitive information, it is important to be
able to solve convex minimization in a privacy-preserving way.
A series of recent results show how to accurately solve a single convex
minimization problem in a differentially private manner. However, the same data
is often analyzed repeatedly, and little is known about solving multiple convex
minimization problems with differential privacy. For simpler data analyses,
such as linear queries, there are remarkable differentially private algorithms
such as the private multiplicative weights mechanism (Hardt and Rothblum, FOCS
2010) that accurately answer exponentially many distinct queries. In this work,
we extend these results to the case of convex minimization and show how to give
accurate and differentially private solutions to *exponentially many* convex
minimization problems on a sensitive dataset
Private Incremental Regression
Data is continuously generated by modern data sources, and a recent challenge
in machine learning has been to develop techniques that perform well in an
incremental (streaming) setting. In this paper, we investigate the problem of
private machine learning, where as common in practice, the data is not given at
once, but rather arrives incrementally over time.
We introduce the problems of private incremental ERM and private incremental
regression where the general goal is to always maintain a good empirical risk
minimizer for the history observed under differential privacy. Our first
contribution is a generic transformation of private batch ERM mechanisms into
private incremental ERM mechanisms, based on a simple idea of invoking the
private batch ERM procedure at some regular time intervals. We take this
construction as a baseline for comparison. We then provide two mechanisms for
the private incremental regression problem. Our first mechanism is based on
privately constructing a noisy incremental gradient function, which is then
used in a modified projected gradient procedure at every timestep. This
mechanism has an excess empirical risk of , where is the
dimensionality of the data. While from the results of [Bassily et al. 2014]
this bound is tight in the worst-case, we show that certain geometric
properties of the input and constraint set can be used to derive significantly
better results for certain interesting regression problems.Comment: To appear in PODS 201
Privacy Amplification for Matrix Mechanisms
Privacy amplification exploits randomness in data selection to provide
tighter differential privacy (DP) guarantees. This analysis is key to DP-SGD's
success in machine learning, but, is not readily applicable to the newer
state-of-the-art algorithms. This is because these algorithms, known as
DP-FTRL, use the matrix mechanism to add correlated noise instead of
independent noise as in DP-SGD.
In this paper, we propose "MMCC", the first algorithm to analyze privacy
amplification via sampling for any generic matrix mechanism. MMCC is nearly
tight in that it approaches a lower bound as . To analyze
correlated outputs in MMCC, we prove that they can be analyzed as if they were
independent, by conditioning them on prior outputs. Our "conditional
composition theorem" has broad utility: we use it to show that the noise added
to binary-tree-DP-FTRL can asymptotically match the noise added to DP-SGD with
amplification. Our amplification algorithm also has practical empirical
utility: we show it leads to significant improvement in the privacy-utility
trade-offs for DP-FTRL algorithms on standard benchmarks
Multi-Epoch Matrix Factorization Mechanisms for Private Machine Learning
We introduce new differentially private (DP) mechanisms for gradient-based
machine learning (ML) with multiple passes (epochs) over a dataset,
substantially improving the achievable privacy-utility-computation tradeoffs.
We formalize the problem of DP mechanisms for adaptive streams with multiple
participations and introduce a non-trivial extension of online matrix
factorization DP mechanisms to our setting. This includes establishing the
necessary theory for sensitivity calculations and efficient computation of
optimal matrices. For some applications like SGD steps, applying
these optimal techniques becomes computationally expensive. We thus design an
efficient Fourier-transform-based mechanism with only a minor utility loss.
Extensive empirical evaluation on both example-level DP for image
classification and user-level DP for language modeling demonstrate substantial
improvements over all previous methods, including the widely-used DP-SGD .
Though our primary application is to ML, our main DP results are applicable to
arbitrary linear queries and hence may have much broader applicability.Comment: 9 pages main-text, 3 figures. 40 pages with 13 figures tota
Correlated Noise Provably Beats Independent Noise for Differentially Private Learning
Differentially private learning algorithms inject noise into the learning
process. While the most common private learning algorithm, DP-SGD, adds
independent Gaussian noise in each iteration, recent work on matrix
factorization mechanisms has shown empirically that introducing correlations in
the noise can greatly improve their utility. We characterize the asymptotic
learning utility for any choice of the correlation function, giving precise
analytical bounds for linear regression and as the solution to a convex program
for general convex functions. We show, using these bounds, how correlated noise
provably improves upon vanilla DP-SGD as a function of problem parameters such
as the effective dimension and condition number. Moreover, our analytical
expression for the near-optimal correlation function circumvents the cubic
complexity of the semi-definite program used to optimize the noise correlation
matrix in previous work. We validate our theory with experiments on private
deep learning. Our work matches or outperforms prior work while being efficient
both in terms of compute and memory.Comment: Christopher A. Choquette-Choo, Krishnamurthy Dvijotham, and Krishna
Pillutla contributed equall
Luspatercept stimulates erythropoiesis, increases iron utilization, and redistributes body iron in transfusion-dependent thalassemia
Luspatercept, a ligand-trapping fusion protein, binds select TGF-β superfamily ligands implicated in thalassemic erythropoiesis, promoting late-stage erythroid maturation. Luspatercept reduced transfusion burden in the BELIEVE trial (NCT02604433) of 336 adults with transfusion-dependent thalassemia (TDT). Analysis of biomarkers in BELIEVE offers novel physiological and clinical insights into benefits offered by luspatercept. Transfusion iron loading rates decreased 20% by 1.4 g (~7 blood units; median iron loading rate difference: −0.05 ± 0.07 mg Fe/kg/day, p< .0001) and serum ferritin (s-ferritin) decreased 19.2% by 269.3 ± 963.7 μg/L (p < .0001), indicating reduced macrophage iron. However, liver iron content (LIC) did not decrease but showed statistically nonsignificant increases from 5.3 to 6.7 mg/g dw. Erythropoietin, growth differentiation factor 15, soluble transferrin receptor 1 (sTfR1), and reticulocytes rose by 93%, 59%, 66%, and 112%, respectively; accordingly, erythroferrone increased by 51% and hepcidin decreased by 53% (all p < .0001). Decreased transfusion with luspatercept in patients with TDT was associated with increased erythropoietic markers and decreasing hepcidin. Furthermore, s-ferritin reduction associated with increased erythroid iron incorporation (marked by sTfR1) allowed increased erythrocyte marrow output, consequently reducing transfusion needs and enhancing rerouting of hemolysis (heme) iron and non-transferrin-bound iron to the liver. LIC increased in patients with intact spleens, consistent with iron redistribution given the hepcidin reduction. Thus, erythropoietic and hepcidin changes with luspatercept in TDT lower transfusion dependency and may redistribute iron from macrophages to hepatocytes, necessitating the use of concomitant chelator cover for effective iron management
Ultrasonic Bioreactor as a Platform for Studying Cellular Response
The need for tissue-engineered constructs as replacement tissue continues to grow as the average age of the world’s population increases. However, additional research is required before the efficient production of laboratory-created tissue can be realized. The multitude of parameters that affect cell growth and proliferation is particularly daunting considering that optimized conditions are likely to change as a function of growth. Thus, a generalized research platform is needed in order for quantitative studies to be conducted. In this article, an ultrasonic bioreactor is described for use in studying the response of cells to ultrasonic stimulation. The work is focused on chondrocytes with a long-term view of generating tissue-engineered articular cartilage. Aspects of ultrasound (US) that would negatively affect cells, including temperature and cavitation, are shown to be insignificant for the US protocols used and which cover a wide range of frequencies and pressure amplitudes. The bioreactor is shown to have a positive influence on several factors, including cell proliferation, viability, and gene expression of select chondrocytic markers. Most importantly, we show that a total of 138 unique proteins are differentially expressed on exposure to ultrasonic stimulation, using mass-spectroscopy coupled proteomic analyses. We anticipate that this work will serve as the basis for additional research which will elucidate many of the mechanisms associated with cell response to ultrasonic stimulation
Making sense of the bizarre behaviour of horizons in the McVittie spacetime
The bizarre behaviour of the apparent (black hole and cosmological) horizons
of the McVittie spacetime is discussed using, as an analogy, the
Schwarzschild-de Sitter-Kottler spacetime (which is a special case of McVittie
anyway). For a dust-dominated "background" universe, a black hole cannot exist
at early times because its (apparent) horizon would be larger than the
cosmological(apparent) horizon. A phantom-dominated "background" universe
causes this situation, and the horizon behaviour, to be time-reversed.Comment: 8 pages, 3 figure
Ultrasonic Bioreactor as a Platform for Studying Cellular Response
The need for tissue-engineered constructs as replacement tissue continues to grow as the average age of the world’s population increases. However, additional research is required before the efficient production of laboratory-created tissue can be realized. The multitude of parameters that affect cell growth and proliferation is particularly daunting considering that optimized conditions are likely to change as a function of growth. Thus, a generalized research platform is needed in order for quantitative studies to be conducted. In this article, an ultrasonic bioreactor is described for use in studying the response of cells to ultrasonic stimulation. The work is focused on chondrocytes with a long-term view of generating tissue-engineered articular cartilage. Aspects of ultrasound (US) that would negatively affect cells, including temperature and cavitation, are shown to be insignificant for the US protocols used and which cover a wide range of frequencies and pressure amplitudes. The bioreactor is shown to have a positive influence on several factors, including cell proliferation, viability, and gene expression of select chondrocytic markers. Most importantly, we show that a total of 138 unique proteins are differentially expressed on exposure to ultrasonic stimulation, using mass-spectroscopy coupled proteomic analyses. We anticipate that this work will serve as the basis for additional research which will elucidate many of the mechanisms associated with cell response to ultrasonic stimulation
- …