131 research outputs found
Optimal Physical Preprocessing for Example-Based Super-Resolution
In example-based super-resolution, the function relating low-resolution
images to their high-resolution counterparts is learned from a given dataset.
This data-driven approach to solving the inverse problem of increasing image
resolution has been implemented with deep learning algorithms. In this work, we
explore modifying the imaging hardware in order to collect more informative
low-resolution images for better ultimate high-resolution image reconstruction.
We show that this "physical preprocessing" allows for improved image
reconstruction with deep learning in Fourier ptychographic microscopy.
Fourier ptychographic microscopy is a technique allowing for both high
resolution and high field-of-view at the cost of temporal resolution. In
Fourier ptychographic microscopy, variable illumination patterns are used to
collect multiple low-resolution images. These low-resolution images are then
computationally combined to create an image with resolution exceeding that of
any single image from the microscope. We use deep learning to jointly optimize
the illumination pattern with the post-processing reconstruction algorithm for
a given sample type, allowing for single-shot imaging with both high resolution
and high field-of-view. We demonstrate, with simulated data, that the joint
optimization yields improved image reconstruction as compared with sole
optimization of the post-processing reconstruction algorithm
SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks
Despite efforts to align large language models (LLMs) with human values,
widely-used LLMs such as GPT, Llama, Claude, and PaLM are susceptible to
jailbreaking attacks, wherein an adversary fools a targeted LLM into generating
objectionable content. To address this vulnerability, we propose SmoothLLM, the
first algorithm designed to mitigate jailbreaking attacks on LLMs. Based on our
finding that adversarially-generated prompts are brittle to character-level
changes, our defense first randomly perturbs multiple copies of a given input
prompt, and then aggregates the corresponding predictions to detect adversarial
inputs. SmoothLLM reduces the attack success rate on numerous popular LLMs to
below one percentage point, avoids unnecessary conservatism, and admits
provable guarantees on attack mitigation. Moreover, our defense uses
exponentially fewer queries than existing attacks and is compatible with any
LLM. Our code is publicly available at the following link:
https://github.com/arobey1/smooth-llm
Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks
Tight estimation of the Lipschitz constant for deep neural networks (DNNs) is
useful in many applications ranging from robustness certification of
classifiers to stability analysis of closed-loop systems with reinforcement
learning controllers. Existing methods in the literature for estimating the
Lipschitz constant suffer from either lack of accuracy or poor scalability. In
this paper, we present a convex optimization framework to compute guaranteed
upper bounds on the Lipschitz constant of DNNs both accurately and efficiently.
Our main idea is to interpret activation functions as gradients of convex
potential functions. Hence, they satisfy certain properties that can be
described by quadratic constraints. This particular description allows us to
pose the Lipschitz constant estimation problem as a semidefinite program (SDP).
The resulting SDP can be adapted to increase either the estimation accuracy (by
capturing the interaction between activation functions of different layers) or
scalability (by decomposition and parallel implementation). We illustrate the
utility of our approach with a variety of experiments on randomly generated
networks and on classifiers trained on the MNIST and Iris datasets. In
particular, we experimentally demonstrate that our Lipschitz bounds are the
most accurate compared to those in the literature. We also study the impact of
adversarial training methods on the Lipschitz bounds of the resulting
classifiers and show that our bounds can be used to efficiently provide
robustness guarantees
Probable Domain Generalization via Quantile Risk Minimization
Domain generalization (DG) seeks predictors which perform well on unseen test
distributions by leveraging data drawn from multiple related training
distributions or domains. To achieve this, DG is commonly formulated as an
average- or worst-case problem over the set of possible domains. However,
predictors that perform well on average lack robustness while predictors that
perform well in the worst case tend to be overly-conservative. To address this,
we propose a new probabilistic framework for DG where the goal is to learn
predictors that perform well with high probability. Our key idea is that
distribution shifts seen during training should inform us of probable shifts at
test time, which we realize by explicitly relating training and test domains as
draws from the same underlying meta-distribution. To achieve probable DG, we
propose a new optimization problem called Quantile Risk Minimization (QRM). By
minimizing the -quantile of predictor's risk distribution over domains,
QRM seeks predictors that perform well with probability . To solve QRM
in practice, we propose the Empirical QRM (EQRM) algorithm and provide: (i) a
generalization bound for EQRM; and (ii) the conditions under which EQRM
recovers the causal predictor as . In our experiments, we
introduce a more holistic quantile-focused evaluation protocol for DG and
demonstrate that EQRM outperforms state-of-the-art baselines on datasets from
WILDS and DomainBed.Comment: NeurIPS 2022 camera-ready (+ minor corrections
- …