2,418 research outputs found
The Paradox of Noise: An Empirical Study of Noise-Infusion Mechanisms to Improve Generalization, Stability, and Privacy in Federated Learning
In a data-centric era, concerns regarding privacy and ethical data handling
grow as machine learning relies more on personal information. This empirical
study investigates the privacy, generalization, and stability of deep learning
models in the presence of additive noise in federated learning frameworks. Our
main objective is to provide strategies to measure the generalization,
stability, and privacy-preserving capabilities of these models and further
improve them. To this end, five noise infusion mechanisms at varying noise
levels within centralized and federated learning settings are explored. As
model complexity is a key component of the generalization and stability of deep
learning models during training and evaluation, a comparative analysis of three
Convolutional Neural Network (CNN) architectures is provided. The paper
introduces Signal-to-Noise Ratio (SNR) as a quantitative measure of the
trade-off between privacy and training accuracy of noise-infused models, aiming
to find the noise level that yields optimal privacy and accuracy. Moreover, the
Price of Stability and Price of Anarchy are defined in the context of
privacy-preserving deep learning, contributing to the systematic investigation
of the noise infusion strategies to enhance privacy without compromising
performance. Our research sheds light on the delicate balance between these
critical factors, fostering a deeper understanding of the implications of
noise-based regularization in machine learning. By leveraging noise as a tool
for regularization and privacy enhancement, we aim to contribute to the
development of robust, privacy-aware algorithms, ensuring that AI-driven
solutions prioritize both utility and privacy
A Self-Supervised Feature Map Augmentation (FMA) Loss and Combined Augmentations Finetuning to Efficiently Improve the Robustness of CNNs
Deep neural networks are often not robust to semantically-irrelevant changes
in the input. In this work we address the issue of robustness of
state-of-the-art deep convolutional neural networks (CNNs) against commonly
occurring distortions in the input such as photometric changes, or the addition
of blur and noise. These changes in the input are often accounted for during
training in the form of data augmentation. We have two major contributions:
First, we propose a new regularization loss called feature-map augmentation
(FMA) loss which can be used during finetuning to make a model robust to
several distortions in the input. Second, we propose a new combined
augmentations (CA) finetuning strategy, that results in a single model that is
robust to several augmentation types at the same time in a data-efficient
manner. We use the CA strategy to improve an existing state-of-the-art method
called stability training (ST). Using CA, on an image classification task with
distorted images, we achieve an accuracy improvement of on average 8.94% with
FMA and 8.86% with ST absolute on CIFAR-10 and 8.04% with FMA and 8.27% with ST
absolute on ImageNet, compared to 1.98% and 2.12%, respectively, with the well
known data augmentation method, while keeping the clean baseline performance.Comment: Accepted at ACM CSCS 2020 (8 pages, 4 figures
Representability of algebraic topology for biomolecules in machine learning based scoring and virtual screening
This work introduces a number of algebraic topology approaches, such as
multicomponent persistent homology, multi-level persistent homology and
electrostatic persistence for the representation, characterization, and
description of small molecules and biomolecular complexes. Multicomponent
persistent homology retains critical chemical and biological information during
the topological simplification of biomolecular geometric complexity.
Multi-level persistent homology enables a tailored topological description of
inter- and/or intra-molecular interactions of interest. Electrostatic
persistence incorporates partial charge information into topological
invariants. These topological methods are paired with Wasserstein distance to
characterize similarities between molecules and are further integrated with a
variety of machine learning algorithms, including k-nearest neighbors, ensemble
of trees, and deep convolutional neural networks, to manifest their descriptive
and predictive powers for chemical and biological problems. Extensive numerical
experiments involving more than 4,000 protein-ligand complexes from the PDBBind
database and near 100,000 ligands and decoys in the DUD database are performed
to test respectively the scoring power and the virtual screening power of the
proposed topological approaches. It is demonstrated that the present approaches
outperform the modern machine learning based methods in protein-ligand binding
affinity predictions and ligand-decoy discrimination
- …