10 research outputs found
Improving the Privacy and Practicality of Objective Perturbation for Differentially Private Linear Learners
In the arena of privacy-preserving machine learning, differentially private
stochastic gradient descent (DP-SGD) has outstripped the objective perturbation
mechanism in popularity and interest. Though unrivaled in versatility, DP-SGD
requires a non-trivial privacy overhead (for privately tuning the model's
hyperparameters) and a computational complexity which might be extravagant for
simple models such as linear and logistic regression. This paper revamps the
objective perturbation mechanism with tighter privacy analyses and new
computational tools that boost it to perform competitively with DP-SGD on
unconstrained convex generalized linear problems
Differentially Private Learning with Per-Sample Adaptive Clipping
Privacy in AI remains a topic that draws attention from researchers and the
general public in recent years. As one way to implement privacy-preserving AI,
differentially private learning is a framework that enables AI models to use
differential privacy (DP). To achieve DP in the learning process, existing
algorithms typically limit the magnitude of gradients with a constant clipping,
which requires carefully tuned due to its significant impact on model
performance. As a solution to this issue, latest works NSGD and Auto-S
innovatively propose to use normalization instead of clipping to avoid
hyperparameter tuning. However, normalization-based approaches like NSGD and
Auto-S rely on a monotonic weight function, which imposes excessive weight on
small gradient samples and introduces extra deviation to the update. In this
paper, we propose a Differentially Private Per-Sample Adaptive Clipping
(DP-PSAC) algorithm based on a non-monotonic adaptive weight function, which
guarantees privacy without the typical hyperparameter tuning process of using a
constant clipping while significantly reducing the deviation between the update
and true batch-averaged gradient. We provide a rigorous theoretical convergence
analysis and show that with convergence rate at the same order, the proposed
algorithm achieves a lower non-vanishing bound, which is maintained over
training iterations, compared with NSGD/Auto-S. In addition, through extensive
experimental evaluation, we show that DP-PSAC outperforms or matches the
state-of-the-art methods on multiple main-stream vision and language tasks.Comment: To appear in AAAI 2023, Revised acknowledgments and citation
Safeguarding Data in Multimodal AI: A Differentially Private Approach to CLIP Training
The surge in multimodal AI's success has sparked concerns over data privacy
in vision-and-language tasks. While CLIP has revolutionized multimodal learning
through joint training on images and text, its potential to unintentionally
disclose sensitive information necessitates the integration of
privacy-preserving mechanisms. We introduce a differentially private adaptation
of the Contrastive Language-Image Pretraining (CLIP) model that effectively
addresses privacy concerns while retaining accuracy. Our proposed method,
Dp-CLIP, is rigorously evaluated on benchmark datasets encompassing diverse
vision-and-language tasks such as image classification and visual question
answering. We demonstrate that our approach retains performance on par with the
standard non-private CLIP model. Furthermore, we analyze our proposed algorithm
under linear representation settings. We derive the convergence rate of our
algorithm and show a trade-off between utility and privacy when gradients are
clipped per-batch and the loss function does not satisfy smoothness conditions
assumed in the literature for the analysis of DP-SGD
LIPIcs, Volume 251, ITCS 2023, Complete Volume
LIPIcs, Volume 251, ITCS 2023, Complete Volum
Aeronautical engineering: A continuing bibliography with indexes (supplement 272)
This bibliography lists 719 reports, articles, and other documents introduced into the NASA scientific and technical information system in November, 1991. Subject coverage includes: design, construction and testing of aircraft and aircraft engines; aircraft components, equipment, and systems; ground support systems; and theoretical and applied aspects of aerodynamics and general fluid dynamics
Behavior quantification as the missing link between fields: Tools for digital psychiatry and their role in the future of neurobiology
The great behavioral heterogeneity observed between individuals with the same
psychiatric disorder and even within one individual over time complicates both
clinical practice and biomedical research. However, modern technologies are an
exciting opportunity to improve behavioral characterization. Existing
psychiatry methods that are qualitative or unscalable, such as patient surveys
or clinical interviews, can now be collected at a greater capacity and analyzed
to produce new quantitative measures. Furthermore, recent capabilities for
continuous collection of passive sensor streams, such as phone GPS or
smartwatch accelerometer, open avenues of novel questioning that were
previously entirely unrealistic. Their temporally dense nature enables a
cohesive study of real-time neural and behavioral signals.
To develop comprehensive neurobiological models of psychiatric disease, it
will be critical to first develop strong methods for behavioral quantification.
There is huge potential in what can theoretically be captured by current
technologies, but this in itself presents a large computational challenge --
one that will necessitate new data processing tools, new machine learning
techniques, and ultimately a shift in how interdisciplinary work is conducted.
In my thesis, I detail research projects that take different perspectives on
digital psychiatry, subsequently tying ideas together with a concluding
discussion on the future of the field. I also provide software infrastructure
where relevant, with extensive documentation.
Major contributions include scientific arguments and proof of concept results
for daily free-form audio journals as an underappreciated psychiatry research
datatype, as well as novel stability theorems and pilot empirical success for a
proposed multi-area recurrent neural network architecture.Comment: PhD thesis cop