16 research outputs found
Augmented L1 and Nuclear-Norm Models with a Globally Linearly Convergent Algorithm
This paper studies the long-existing idea of adding a nice smooth function to
"smooth" a non-differentiable objective function in the context of sparse
optimization, in particular, the minimization of
, where is a vector, as well as the
minimization of , where is a matrix and
and are the nuclear and Frobenius norms of ,
respectively. We show that they can efficiently recover sparse vectors and
low-rank matrices. In particular, they enjoy exact and stable recovery
guarantees similar to those known for minimizing and under
the conditions on the sensing operator such as its null-space property,
restricted isometry property, spherical section property, or RIPless property.
To recover a (nearly) sparse vector , minimizing
returns (nearly) the same solution as minimizing
almost whenever . The same relation also
holds between minimizing and minimizing
for recovering a (nearly) low-rank matrix , if . Furthermore, we show that the linearized Bregman algorithm for
minimizing subject to enjoys global
linear convergence as long as a nonzero solution exists, and we give an
explicit rate of convergence. The convergence property does not require a
solution solution or any properties on . To our knowledge, this is the best
known global convergence result for first-order sparse optimization algorithms.Comment: arXiv admin note: text overlap with arXiv:1207.5326 by other author
From Low-Distortion Norm Embeddings to Explicit Uncertainty Relations and Efficient Information Locking
The existence of quantum uncertainty relations is the essential reason that
some classically impossible cryptographic primitives become possible when
quantum communication is allowed. One direct operational manifestation of these
uncertainty relations is a purely quantum effect referred to as information
locking. A locking scheme can be viewed as a cryptographic protocol in which a
uniformly random n-bit message is encoded in a quantum system using a classical
key of size much smaller than n. Without the key, no measurement of this
quantum state can extract more than a negligible amount of information about
the message, in which case the message is said to be "locked". Furthermore,
knowing the key, it is possible to recover, that is "unlock", the message. In
this paper, we make the following contributions by exploiting a connection
between uncertainty relations and low-distortion embeddings of L2 into L1. We
introduce the notion of metric uncertainty relations and connect it to
low-distortion embeddings of L2 into L1. A metric uncertainty relation also
implies an entropic uncertainty relation. We prove that random bases satisfy
uncertainty relations with a stronger definition and better parameters than
previously known. Our proof is also considerably simpler than earlier proofs.
We apply this result to show the existence of locking schemes with key size
independent of the message length. We give efficient constructions of metric
uncertainty relations. The bases defining these metric uncertainty relations
are computable by quantum circuits of almost linear size. This leads to the
first explicit construction of a strong information locking scheme. Moreover,
we present a locking scheme that is close to being implementable with current
technology. We apply our metric uncertainty relations to exhibit communication
protocols that perform quantum equality testing.Comment: 60 pages, 5 figures. v4: published versio
Overcoming the convex barrier for simplex input
Recent progress in neural network verification has challenged the notion of a convex
barrier, that is, an inherent weakness in the convex relaxation of the output of
a neural network. Specifically, there now exists a tight relaxation for verifying
the robustness of a neural network to `∞ input perturbations, as well as efficient
primal and dual solvers for the relaxation. Buoyed by this success, we consider
the problem of developing similar techniques for verifying robustness to input
perturbations within the probability simplex. We prove a somewhat surprising
result that, in this case, not only can one design a tight relaxation that overcomes
the convex barrier, but the size of the relaxation remains linear in the number of
neurons, thereby leading to simpler and more efficient algorithms. We establish
the scalability of our overall approach via the specification of `1 robustness for
CIFAR-10 and MNIST classification, where our approach improves the state of the
art verified accuracy by up to 14.4%. Furthermore, we establish its accuracy on
a novel and highly challenging task of verifying the robustness of a multi-modal
(text and image) classifier to arbitrary changes in its textual input
Robust optimization for electricity generation
10.1287/ijoc.2020.0956INFORMS Journal on Computing331336-35
Sensitivity to the Sampling Process Emerges From the Principle of Efficiency
Humans can seamlessly infer other people's preferences, based on what they do. Broadly, two types of accounts have been proposed to explain different aspects of this ability. The first account focuses on spatial information: Agents' efficient navigation in space reveals what they like. The second account focuses on statistical information: Uncommon choices reveal stronger preferences. Together, these two lines of research suggest that we have two distinct capacities for inferring preferences. Here we propose that this is not the case, and that spatial-based and statistical-based preference inferences can be explained by the assumption that agents are efficient alone. We show that people's sensitivity to spatial and statistical information when they infer preferences is best predicted by a computational model of the principle of efficiency, and that this model outperforms dual-system models, even when the latter are fit to participant judgments. Our results suggest that, as adults, a unified understanding of agency under the principle of efficiency underlies our ability to infer preferences. Copyright ©2018 Cognitive Science Society, Inc.NSF-STC award (CCF-1231216
Lagrangian decomposition for neural network verification
A fundamental component of neural network verification is the computation of bounds on the values their outputs can take. Previous methods have either used off-the-shelf solvers, discarding the problem structure, or relaxed the problem even further, making the bounds unnecessarily loose. We propose a novel approach based on Lagrangian Decomposition. Our formulation admits an efficient supergradient ascent algorithm, as well as an improved proximal algorithm. Both the algorithms offer three advantages: (i) they yield bounds that are provably at least as tight as previous dual algorithms relying on Lagrangian relaxations; (ii) they are based on operations analogous to forward/backward pass of neural networks layers and are therefore easily parallelizable, amenable to GPU implementation and able to take advantage of the convolutional structure of problems; and (iii) they allow for anytime stopping while still providing valid bounds. Empirically, we show that we obtain bounds comparable with off-the-shelf solvers in a fraction of their running time, and obtain tighter bounds in the same time as previous dual algorithms. This results in an overall speed-up when employing the bounds for formal verification. Code for our algorithms is available at https://github.com/oval-group/decomposition-plnn-bounds