13 research outputs found
Adversarial Illusions in Multi-Modal Embeddings
Multi-modal embeddings encode images, sounds, texts, videos, etc. into a
single embedding space, aligning representations across modalities (e.g.,
associate an image of a dog with a barking sound). We show that multi-modal
embeddings can be vulnerable to an attack we call "adversarial illusions."
Given an image or a sound, an adversary can perturb it so as to make its
embedding close to an arbitrary, adversary-chosen input in another modality.
This enables the adversary to align any image and any sound with any text.
Adversarial illusions exploit proximity in the embedding space and are thus
agnostic to downstream tasks. Using ImageBind embeddings, we demonstrate how
adversarially aligned inputs, generated without knowledge of specific
downstream tasks, mislead image generation, text generation, and zero-shot
classification
(Ab)using Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs
We demonstrate how images and sounds can be used for indirect prompt and
instruction injection in multi-modal LLMs. An attacker generates an adversarial
perturbation corresponding to the prompt and blends it into an image or audio
recording. When the user asks the (unmodified, benign) model about the
perturbed image or audio, the perturbation steers the model to output the
attacker-chosen text and/or make the subsequent dialog follow the attacker's
instruction. We illustrate this attack with several proof-of-concept examples
targeting LLaVa and PandaGPT
Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems
We propose Februus; a new idea to neutralize highly potent and insidious
Trojan attacks on Deep Neural Network (DNN) systems at run-time. In Trojan
attacks, an adversary activates a backdoor crafted in a deep neural network
model using a secret trigger, a Trojan, applied to any input to alter the
model's decision to a target prediction---a target determined by and only known
to the attacker. Februus sanitizes the incoming input by surgically removing
the potential trigger artifacts and restoring the input for the classification
task. Februus enables effective Trojan mitigation by sanitizing inputs with no
loss of performance for sanitized inputs, Trojaned or benign. Our extensive
evaluations on multiple infected models based on four popular datasets across
three contrasting vision applications and trigger types demonstrate the high
efficacy of Februus. We dramatically reduced attack success rates from 100% to
near 0% for all cases (achieving 0% on multiple cases) and evaluated the
generalizability of Februus to defend against complex adaptive attacks;
notably, we realized the first defense against the advanced partial Trojan
attack. To the best of our knowledge, Februus is the first backdoor defense
method for operation at run-time capable of sanitizing Trojaned inputs without
requiring anomaly detection methods, model retraining or costly labeled data.Comment: 16 pages, to appear in the 36th Annual Computer Security Applications
Conference (ACSAC 2020
GROWTH on S190425z: Searching Thousands of Square Degrees to Identify an Optical or Infrared Counterpart to a Binary Neutron Star Merger with the Zwicky Transient Facility and Palomar Gattini-IR
The third observing run by LVC has brought the discovery of many compact binary coalescences. Following the detection of the first binary neutron star merger in this run (LIGO/Virgo S190425z), we performed a dedicated follow-up campaign with the Zwicky Transient Facility (ZTF) and Palomar Gattini-IR telescopes. The initial skymap of this single-detector gravitational wave (GW) trigger spanned most of the sky observable from Palomar Observatory. Covering 8000 deg2 of the initial skymap over the next two nights, corresponding to 46% integrated probability, ZTF system achieved a depth of ≈21 m AB in g- and r-bands. Palomar Gattini-IR covered 2200 square degrees in J-band to a depth of 15.5 mag, including 32% integrated probability based on the initial skymap. The revised skymap issued the following day reduced these numbers to 21% for the ZTF and 19% for Palomar Gattini-IR. We narrowed 338,646 ZTF transient "alerts" over the first two nights of observations to 15 candidate counterparts. Two candidates, ZTF19aarykkb and ZTF19aarzaod, were particularly compelling given that their location, distance, and age were consistent with the GW event, and their early optical light curves were photometrically consistent with that of kilonovae. These two candidates were spectroscopically classified as young core-collapse supernovae. The remaining candidates were ruled out as supernovae. Palomar Gattini-IR did not identify any viable candidates with multiple detections only after merger time. We demonstrate that even with single-detector GW events localized to thousands of square degrees, systematic kilonova discovery is feasible
GROWTH on GW190425: Searching thousands of square degrees to identify an optical or infrared counterpart to a binary neutron star merger with the Zwicky Transient Facility and Palomar Gattini IR
The beginning of the third observing run by the network of gravitational-wave
detectors has brought the discovery of many compact binary coalescences.
Prompted by the detection of the first binary neutron star merger in this run
(GW190425 / LIGO/Virgo S190425z), we performed a dedicated follow-up campaign
with the Zwicky Transient Facility (ZTF) and Palomar Gattini-IR telescopes. As
it was a single gravitational-wave detector discovery, the initial skymap
spanned most of the sky observable from Palomar Observatory, the site of both
instruments. Covering 8000 deg of the inner 99\% of the initial skymap over
the next two nights, corresponding to an integrated probability of 46\%, the
ZTF system achieved a depth of \,21 in - and
-bands. Palomar Gattini-IR covered a total of 2200 square degrees in
-band to a depth of 15.5\,mag, including 32\% of the integrated probability
based on the initial sky map. However, the revised skymap issued the following
day reduced these numbers to 21\% for the Zwicky Transient Facility and 19\%
for Palomar Gattini-IR. Out of the 338,646 ZTF transient "alerts" over the
first two nights of observations, we narrowed this list to 15 candidate
counterparts. Two candidates, ZTF19aarykkb and ZTF19aarzaod were particularly
compelling given that their location, distance, and age were consistent with
the gravitational-wave event, and their early optical lightcurves were
photometrically consistent with that of kilonovae. These two candidates were
spectroscopically classified as young core-collapse supernovae. The remaining
candidates were photometrically or spectroscopically ruled-out as supernovae.
Palomar Gattini-IR identified one fast evolving infrared transient after the
merger, PGIR19bn, which was later spectroscopically classified as an M-dwarf
flare. [abridged