807 research outputs found
Differential Analysis of Triggers and Benign Features for Black-Box DNN Backdoor Detection
This paper proposes a data-efficient detection method for deep neural
networks against backdoor attacks under a black-box scenario. The proposed
approach is motivated by the intuition that features corresponding to triggers
have a higher influence in determining the backdoored network output than any
other benign features. To quantitatively measure the effects of triggers and
benign features on determining the backdoored network output, we introduce five
metrics. To calculate the five-metric values for a given input, we first
generate several synthetic samples by injecting the input's partial contents
into clean validation samples. Then, the five metrics are computed by using the
output labels of the corresponding synthetic samples. One contribution of this
work is the use of a tiny clean validation dataset. Having the computed five
metrics, five novelty detectors are trained from the validation dataset. A meta
novelty detector fuses the output of the five trained novelty detectors to
generate a meta confidence score. During online testing, our method determines
if online samples are poisoned or not via assessing their meta confidence
scores output by the meta novelty detector. We show the efficacy of our
methodology through a broad range of backdoor attacks, including ablation
studies and comparison to existing approaches. Our methodology is promising
since the proposed five metrics quantify the inherent differences between clean
and poisoned samples. Additionally, our detection method can be incrementally
improved by appending more metrics that may be proposed to address future
advanced attacks.Comment: Published in the IEEE Transactions on Information Forensics and
Securit
Impact of ancestral sequence reconstruction on mechanistic and structural enzymology
Ancestral sequence reconstruction (ASR) provides insight into the changes within a protein sequence across evolution. More specifically, it can illustrate how specific amino acid changes give rise to different phenotypes within a protein family. Over the last few decades it has established itself as a powerful technique for revealing molecular common denominators that govern enzyme function. Here, we describe the strength of ASR in unveiling catalytic mechanisms and emerging phenotypes for a range of different proteins, also highlighting biotechnological applications the methodology can provide.</p
Training Large-Vocabulary Neural Language Models by Private Federated Learning for Resource-Constrained Devices
Federated Learning (FL) is a technique to train models using data distributed
across devices. Differential Privacy (DP) provides a formal privacy guarantee
for sensitive data. Our goal is to train a large neural network language model
(NNLM) on compute-constrained devices while preserving privacy using FL and DP.
However, the DP-noise introduced to the model increases as the model size
grows, which often prevents convergence. We propose Partial Embedding Updates
(PEU), a novel technique to decrease noise by decreasing payload size.
Furthermore, we adopt Low Rank Adaptation (LoRA) and Noise Contrastive
Estimation (NCE) to reduce the memory demands of large models on
compute-constrained devices. This combination of techniques makes it possible
to train large-vocabulary language models while preserving accuracy and
privacy
Distilling Cognitive Backdoor Patterns within an Image
This paper proposes a simple method to distill and detect backdoor patterns
within an image: \emph{Cognitive Distillation} (CD). The idea is to extract the
"minimal essence" from an input image responsible for the model's prediction.
CD optimizes an input mask to extract a small pattern from the input image that
can lead to the same model output (i.e., logits or deep features). The
extracted pattern can help understand the cognitive mechanism of a model on
clean vs. backdoor images and is thus called a \emph{Cognitive Pattern} (CP).
Using CD and the distilled CPs, we uncover an interesting phenomenon of
backdoor attacks: despite the various forms and sizes of trigger patterns used
by different attacks, the CPs of backdoor samples are all surprisingly and
suspiciously small. One thus can leverage the learned mask to detect and remove
backdoor examples from poisoned training datasets. We conduct extensive
experiments to show that CD can robustly detect a wide range of advanced
backdoor attacks. We also show that CD can potentially be applied to help
detect potential biases from face datasets. Code is available at
\url{https://github.com/HanxunH/CognitiveDistillation}.Comment: ICLR202
Recommended from our members
Exploiting Future Word Contexts in Neural Network Language Models for Speech Recognition
Language modelling is a crucial component in a wide range of applications including speech recognition. Language models (LMs) are usually constructed by splitting a sentence into words and computing the probability of a word based on its word history. This sentence probability calculation, making use of conditional probability distributions, assumes that there is little impact from approximations used in the LMs including:
the word history representations; and approaches to handle finite training data. This motivates examining models that make use of additional information from the sentence. In this work future word information, in addition to the history, is used to predict the probability of the current word. For recurrent neural network LMs (RNNLMs) this information can be encapsulated in a bi-directional model. However, if used directly this form
of model is computationally expensive when training on large quantities of data, and can be problematic when used with word lattices. This paper proposes a novel neural network language model structure, the succeeding-word RNNLM, su-RNNLM, to address these issues. Instead of using a recurrent unit to capture the complete future word contexts, a feed-forward unit is used to model a fixed finite number of succeeding words. This is more efficient in training than bi-directional models and can be applied to lattice rescoring. The generated lattices can be used for downstream applications, such as confusion network decoding and keyword search. Experimental results on speech recognition and keyword spotting tasks illustrate the empirical usefulness of future word information, and the flexibility of the proposed model to represent this information
Spartan Daily, December 6, 1962
Volume 50, Issue 51https://scholarworks.sjsu.edu/spartandaily/4372/thumbnail.jp
- …