9 research outputs found
Locally Differentially Private Document Generation Using Zero Shot Prompting
Numerous studies have highlighted the privacy risks associated with
pretrained large language models. In contrast, our research offers a unique
perspective by demonstrating that pretrained large language models can
effectively contribute to privacy preservation. We propose a locally
differentially private mechanism called DP-Prompt, which leverages the power of
pretrained large language models and zero-shot prompting to counter author
de-anonymization attacks while minimizing the impact on downstream utility.
When DP-Prompt is used with a powerful language model like ChatGPT (gpt-3.5),
we observe a notable reduction in the success rate of de-anonymization attacks,
showing that it surpasses existing approaches by a considerable margin despite
its simpler design. For instance, in the case of the IMDB dataset, DP-Prompt
(with ChatGPT) perfectly recovers the clean sentiment F1 score while achieving
a 46\% reduction in author identification F1 score against static attackers and
a 26\% reduction against adaptive attackers. We conduct extensive experiments
across six open-source large language models, ranging up to 7 billion
parameters, to analyze various effects of the privacy-utility tradeoff.Comment: Accepted at EMNLP 2023 (Findings
Language Agnostic Code Embeddings
Recently, code language models have achieved notable advancements in
addressing a diverse array of essential code comprehension and generation
tasks. Yet, the field lacks a comprehensive deep dive and understanding of the
code embeddings of multilingual code models. In this paper, we present a
comprehensive study on multilingual code embeddings, focusing on the
cross-lingual capabilities of these embeddings across different programming
languages. Through probing experiments, we demonstrate that code embeddings
comprise two distinct components: one deeply tied to the nuances and syntax of
a specific language, and the other remaining agnostic to these details,
primarily focusing on semantics. Further, we show that when we isolate and
eliminate this language-specific component, we witness significant improvements
in downstream code retrieval tasks, leading to an absolute increase of up to
+17 in the Mean Reciprocal Rank (MRR)
Rieoptax: Riemannian Optimization in JAX
We present Rieoptax, an open source Python library for Riemannian
optimization in JAX. We show that many differential geometric primitives, such
as Riemannian exponential and logarithm maps, are usually faster in Rieoptax
than existing frameworks in Python, both on CPU and GPU. We support various
range of basic and advanced stochastic optimization solvers like Riemannian
stochastic gradient, stochastic variance reduction, and adaptive gradient
methods. A distinguishing feature of the proposed toolbox is that we also
support differentially private optimization on Riemannian manifolds
geomstats/challenge-iclr-2021: Published algorithms (final version)
GitHub repository for the ICLR Computational Geometry & Topology Challenge 202
ICLR 2022 Challenge for Computational Geometry & Topology: Design and Results
International audienceThis paper presents the computational challenge on differential geometry and topology that was hosted within the ICLR 2022 workshop “Geometric and Topo- logical Representation Learning”. The competition asked participants to provide implementations of machine learning algorithms on manifolds that would respect the API of the open-source software Geomstats (manifold part) and Scikit-Learn (machine learning part) or PyTorch. The challenge attracted seven teams in its two month duration. This paper describes the design of the challenge and summarizes its main findings
ICLR 2021 Challenge for Computational Geometry & Topology: Design and Results
This paper presents the computational challenge on differential geometry and topology that happened within the ICLR 2021 workshop "Geometric and Topological Representation Learning". The competition asked participants to provide creative contributions to the fields of computational geometry and topology through the open-source repositories Geomstats and Giotto-TDA. The challenge attracted 16 teams in its two month duration. This paper describes the design of the challenge and summarizes its main findings
ICLR 2021 Challenge for Computational Geometry & Topology: Design and Results
This paper presents the computational challenge on differential geometry and topology that happened within the ICLR 2021 workshop "Geometric and Topological Representation Learning". The competition asked participants to provide creative contributions to the fields of computational geometry and topology through the open-source repositories Geomstats and Giotto-TDA. The challenge attracted 16 teams in its two month duration. This paper describes the design of the challenge and summarizes its main findings
ICLR 2021 Challenge for Computational Geometry & Topology: Design and Results
This paper presents the computational challenge on differential geometry and topology that happened within the ICLR 2021 workshop "Geometric and Topological Representation Learning". The competition asked participants to provide creative contributions to the fields of computational geometry and topology through the open-source repositories Geomstats and Giotto-TDA. The challenge attracted 16 teams in its two month duration. This paper describes the design of the challenge and summarizes its main findings