2,239 research outputs found
Backdoors in Neural Models of Source Code
Deep neural networks are vulnerable to a range of adversaries. A particularly
pernicious class of vulnerabilities are backdoors, where model predictions
diverge in the presence of subtle triggers in inputs. An attacker can implant a
backdoor by poisoning the training data to yield a desired target prediction on
triggered inputs. We study backdoors in the context of deep-learning for source
code. (1) We define a range of backdoor classes for source-code tasks and show
how to poison a dataset to install such backdoors. (2) We adapt and improve
recent algorithms from robust statistics for our setting, showing that
backdoors leave a spectral signature in the learned representation of source
code, thus enabling detection of poisoned data. (3) We conduct a thorough
evaluation on different architectures and languages, showing the ease of
injecting backdoors and our ability to eliminate them
Walling up Backdoors in Intrusion Detection Systems
Interest in poisoning attacks and backdoors recently resurfaced for Deep
Learning (DL) applications. Several successful defense mechanisms have been
recently proposed for Convolutional Neural Networks (CNNs), for example in the
context of autonomous driving. We show that visualization approaches can aid in
identifying a backdoor independent of the used classifier. Surprisingly, we
find that common defense mechanisms fail utterly to remove backdoors in DL for
Intrusion Detection Systems (IDSs). Finally, we devise pruning-based approaches
to remove backdoors for Decision Trees (DTs) and Random Forests (RFs) and
demonstrate their effectiveness for two different network security datasets
Structure and Problem Hardness: Goal Asymmetry and DPLL Proofs in<br> SAT-Based Planning
In Verification and in (optimal) AI Planning, a successful method is to
formulate the application as boolean satisfiability (SAT), and solve it with
state-of-the-art DPLL-based procedures. There is a lack of understanding of why
this works so well. Focussing on the Planning context, we identify a form of
problem structure concerned with the symmetrical or asymmetrical nature of the
cost of achieving the individual planning goals. We quantify this sort of
structure with a simple numeric parameter called AsymRatio, ranging between 0
and 1. We run experiments in 10 benchmark domains from the International
Planning Competitions since 2000; we show that AsymRatio is a good indicator of
SAT solver performance in 8 of these domains. We then examine carefully crafted
synthetic planning domains that allow control of the amount of structure, and
that are clean enough for a rigorous analysis of the combinatorial search
space. The domains are parameterized by size, and by the amount of structure.
The CNFs we examine are unsatisfiable, encoding one planning step less than the
length of the optimal plan. We prove upper and lower bounds on the size of the
best possible DPLL refutations, under different settings of the amount of
structure, as a function of size. We also identify the best possible sets of
branching variables (backdoors). With minimum AsymRatio, we prove exponential
lower bounds, and identify minimal backdoors of size linear in the number of
variables. With maximum AsymRatio, we identify logarithmic DPLL refutations
(and backdoors), showing a doubly exponential gap between the two structural
extreme cases. The reasons for this behavior -- the proof arguments --
illuminate the prototypical patterns of structure causing the empirical
behavior observed in the competition benchmarks
- …