13 research outputs found
FARE: Provably Fair Representation Learning with Practical Certificates
Fair representation learning (FRL) is a popular class of methods aiming to
produce fair classifiers via data preprocessing. Recent regulatory directives
stress the need for FRL methods that provide practical certificates, i.e.,
provable upper bounds on the unfairness of any downstream classifier trained on
preprocessed data, which directly provides assurance in a practical scenario.
Creating such FRL methods is an important challenge that remains unsolved. In
this work, we address that challenge and introduce FARE (Fairness with
Restricted Encoders), the first FRL method with practical fairness
certificates. FARE is based on our key insight that restricting the
representation space of the encoder enables the derivation of practical
guarantees, while still permitting favorable accuracy-fairness tradeoffs for
suitable instantiations, such as one we propose based on fair trees. To produce
a practical certificate, we develop and apply a statistical procedure that
computes a finite sample high-confidence upper bound on the unfairness of any
downstream classifier trained on FARE embeddings. In our comprehensive
experimental evaluation, we demonstrate that FARE produces practical
certificates that are tight and often even comparable with purely empirical
results obtained by prior methods, which establishes the practical value of our
approach.Comment: ICML 202
Certified Defenses: Why Tighter Relaxations May Hurt Training
Certified defenses based on convex relaxations are an established technique
for training provably robust models. The key component is the choice of
relaxation, varying from simple intervals to tight polyhedra. Paradoxically,
however, training with tighter relaxations can often lead to worse certified
robustness. The poor understanding of this paradox has forced recent
state-of-the-art certified defenses to focus on designing various heuristics in
order to mitigate its effects. In contrast, in this paper we study the
underlying causes and show that tightness alone may not be the determining
factor. Concretely, we identify two key properties of relaxations that impact
training dynamics: continuity and sensitivity. Our extensive experimental
evaluation demonstrates that these two factors, observed alongside tightness,
explain the drop in certified robustness for popular relaxations. Further, we
investigate the possibility of designing and training with relaxations that are
tight, continuous and not sensitive. We believe the insights of this work can
help drive the principled discovery of new and effective certified defense
mechanisms
Data Leakage in Tabular Federated Learning
While federated learning (FL) promises to preserve privacy in distributed
training of deep learning models, recent work in the image and NLP domains
showed that training updates leak private data of participating clients. At the
same time, most high-stakes applications of FL (e.g., legal and financial) use
tabular data. Compared to the NLP and image domains, reconstruction of tabular
data poses several unique challenges: (i) categorical features introduce a
significantly more difficult mixed discrete-continuous optimization problem,
(ii) the mix of categorical and continuous features causes high variance in the
final reconstructions, and (iii) structured data makes it difficult for the
adversary to judge reconstruction quality. In this work, we tackle these
challenges and propose the first comprehensive reconstruction attack on tabular
data, called TabLeak. TabLeak is based on three key ingredients: (i) a softmax
structural prior, implicitly converting the mixed discrete-continuous
optimization problem into an easier fully continuous one, (ii) a way to reduce
the variance of our reconstructions through a pooled ensembling scheme
exploiting the structure of tabular data, and (iii) an entropy measure which
can successfully assess reconstruction quality. Our experimental evaluation
demonstrates the effectiveness of TabLeak, reaching a state-of-the-art on four
popular tabular datasets. For instance, on the Adult dataset, we improve attack
accuracy by 10% compared to the baseline on the practically relevant batch size
of 32 and further obtain non-trivial reconstructions for batch sizes as large
as 128. Our findings are important as they show that performing FL on tabular
data, which often poses high privacy risks, is highly vulnerable
Data Leakage in Federated Averaging
Recent attacks have shown that user data can be recovered from FedSGD
updates, thus breaking privacy. However, these attacks are of limited practical
relevance as federated learning typically uses the FedAvg algorithm. Compared
to FedSGD, recovering data from FedAvg updates is much harder as: (i) the
updates are computed at unobserved intermediate network weights, (ii) a large
number of batches are used, and (iii) labels and network weights vary
simultaneously across client steps. In this work, we propose a new
optimization-based attack which successfully attacks FedAvg by addressing the
above challenges. First, we solve the optimization problem using automatic
differentiation that forces a simulation of the client's update that generates
the unobserved parameters for the recovered labels and inputs to match the
received client update. Second, we address the large number of batches by
relating images from different epochs with a permutation invariant prior.
Third, we recover the labels by estimating the parameters of existing FedSGD
attacks at every FedAvg step. On the popular FEMNIST dataset, we demonstrate
that on average we successfully recover >45% of the client's images from
realistic FedAvg updates computed on 10 local epochs of 10 batches each with 5
images, compared to only <10% using the baseline. Our findings show many
real-world federated learning implementations based on FedAvg are vulnerable
Efficient Certification of Spatial Robustness
Recent work has exposed the vulnerability of computer vision models to vector
field attacks. Due to the widespread usage of such models in safety-critical
applications, it is crucial to quantify their robustness against such spatial
transformations. However, existing work only provides empirical robustness
quantification against vector field deformations via adversarial attacks, which
lack provable guarantees. In this work, we propose novel convex relaxations,
enabling us, for the first time, to provide a certificate of robustness against
vector field transformations. Our relaxations are model-agnostic and can be
leveraged by a wide range of neural network verifiers. Experiments on various
network architectures and different datasets demonstrate the effectiveness and
scalability of our method.Comment: Conference Paper at AAAI 202
Robustness Certification for Point Cloud Models
The use of deep 3D point cloud models in safety-critical applications, such
as autonomous driving, dictates the need to certify the robustness of these
models to real-world transformations. This is technically challenging, as it
requires a scalable verifier tailored to point cloud models that handles a wide
range of semantic 3D transformations. In this work, we address this challenge
and introduce 3DCertify, the first verifier able to certify the robustness of
point cloud models. 3DCertify is based on two key insights: (i) a generic
relaxation based on first-order Taylor approximations, applicable to any
differentiable transformation, and (ii) a precise relaxation for global feature
pooling, which is more complex than pointwise activations (e.g., ReLU or
sigmoid) but commonly employed in point cloud models. We demonstrate the
effectiveness of 3DCertify by performing an extensive evaluation on a wide
range of 3D transformations (e.g., rotation, twisting) for both classification
and part segmentation tasks. For example, we can certify robustness against
rotations by 60{\deg} for 95.7% of point clouds, and our max pool
relaxation increases certification by up to 15.6%.Comment: International Conference on Computer Vision (ICCV) 202
Adversarial Training and Provable Defenses: Bridging the Gap
We present COLT, a new method to train neural networks based on a novel combination of adversarial training and provable defenses. The key idea is to model neural network training as a procedure which includes both, the verifier and the adversary. In every iteration, the verifier aims to certify the network using convex relaxation while the adversary tries to find inputs inside that convex relaxation which cause verification to fail. We experimentally show that this training method, named convex layerwise adversarial training (COLT), is promising and achieves the best of both worlds -- it produces a state-of-the-art neural network with certified robustness of 60.5% and accuracy of 78.4% on the challenging CIFAR-10 dataset with a 2/255 L-infinity perturbation. This significantly improves over the best concurrent results of 54.0% certified robustness and 71.5% accuracy
On the Paradox of Certified Training
Certified defenses based on convex relaxations are an established technique for training provably robust models. The key component is the choice of relaxation, varying from simple intervals to tight polyhedra. Counterintuitively, loose interval-based training often leads to higher certified robustness than what can be achieved with tighter relaxations, which is a well-known but poorly understood paradox. While recent works introduced various improvements aiming to circumvent this issue in practice, the fundamental problem of training models with high certified robustness remains unsolved. In this work, we investigate the underlying reasons behind the paradox and identify two key properties of relaxations, beyond tightness, that impact certified training dynamics: continuity and sensitivity. Our extensive experimental evaluation with a number of popular convex relaxations provides strong evidence that these factors can explain the drop in certified robustness observed for tighter relaxations. We also systematically explore modifications of existing relaxations and discover that improving unfavorable properties is challenging, as such attempts often harm other properties, revealing a complex tradeoff. Our findings represent an important first step towards understanding the intricate optimization challenges involved in certified training