Provably Adversarially Robust Nearest Prototype Classifiers

Nearest prototype classifiers (NPCs) assign to each input point the label of the nearest prototype with respect to a chosen distance metric. A direct advantage of NPCs is that the decisions are interpretable. Previous work could provide lower bounds on the minimal adversarial perturbation in the $\ell_p$-threat model when using the same $\ell_p$-distance for the NPCs. In this paper we provide a complete discussion on the complexity when using $\ell_p$-distances for decision and $\ell_q$-threat models for certification for $p,q \in \{1,2,\infty\}$. In particular we provide scalable algorithms for the \emph{exact} computation of the minimal adversarial perturbation when using $\ell_2$-distance and improved lower bounds in other cases. Using efficient improved lower bounds we train our Provably adversarially robust NPC (PNPC), for MNIST which have better $\ell_2$-robustness guarantees than neural networks. Additionally, we show up to our knowledge the first certification results w.r.t. to the LPIPS perceptual metric which has been argued to be a more realistic threat model for image classification than $\ell_p$-balls. Our PNPC has on CIFAR10 higher certified robust accuracy than the empirical robust accuracy reported in (Laidlaw et al., 2021). The code is available in our repository.Comment: Accepted at ICML 202

Introducing Accountability to Anonymity Networks

Many anonymous communication (AC) networks rely on routing traffic through proxy nodes to obfuscate the originator of the traffic. Without an accountability mechanism, exit proxy nodes risk sanctions by law enforcement if users commit illegal actions through the AC network. We present BackRef, a generic mechanism for AC networks that provides practical repudiation for the proxy nodes by tracing back the selected outbound traffic to the predecessor node (but not in the forward direction) through a cryptographically verifiable chain. It also provides an option for full (or partial) traceability back to the entry node or even to the corresponding user when all intermediate nodes are cooperating. Moreover, to maintain a good balance between anonymity and accountability, the protocol incorporates whitelist directories at exit proxy nodes. BackRef offers improved deployability over the related work, and introduces a novel concept of pseudonymous signatures that may be of independent interest. We exemplify the utility of BackRef by integrating it into the onion routing (OR) protocol, and examine its deployability by considering several system-level aspects. We also present the security definitions for the BackRef system (namely, anonymity, backward traceability, no forward traceability, and no false accusation) and conduct a formal security analysis of the OR protocol with BackRef using ProVerif, an automated cryptographic protocol verifier, establishing the aforementioned security properties against a strong adversarial model

LP Solutions of Vectorial Integer Subset Sums - Cryptanalysis of Galbraith\u27s Binary Matrix LWE

We consider Galbraith\u27s space efficient LWE variant, where the $(m \times n)$-matrix $A$ is binary. In this binary case, solving a vectorial subset sum problem over the integers allows for decryption. We show how to solve this problem using (Integer) Linear Programming. Our attack requires only a fraction of a second for all instances in a regime for $m$ that cannot be attacked by current lattice algorithms. E.g.\ we are able to solve 100 instances of Galbraith\u27s small LWE challenge $(n,m) = (256, 400)$ all in a fraction of a second. We also show under a mild assumption that instances with $m \leq 2n$ can be broken in polynomial time via LP relaxation. Moreover, we develop a method that identifies weak instances for Galbraith\u27s large LWE challenge $(n,m)=(256, 640)$

Well-Distributed Sequences: Number Theory, Optimal Transport, and Potential Theory

The purpose of this dissertation will be to examine various ways of measuring how uniformly distributed a sequence of points on compact manifolds and finite combinatorial graphs can be, providing bounds and novel explicit algorithms to pick extremely uniform points, as well as connecting disparate branches of mathematics such as Number Theory and Optimal Transport. Chapter 1 sets the stage by introducing some of the fundamental ideas and results that will be used consistently throughout the thesis: we develop and establish Weyl\u27s Theorem, the definition of discrepancy, LeVeque\u27s Inequality, the Erdős-Turán Inequality, Koksma-Hlawka Inequality, and Schmidt\u27s Theorem about Irregularities of Distribution. Chapter 2 introduces the Monge-Kantorovich transport problem with special emphasis on the Benamou-Brenier Formula (from 2000) and Peyre\u27s inequality (from 2018). Chapter 3 explores Peyre\u27s Inequality in further depth, considering how specific bounds on the Wasserstein distance between a point measure and the uniform measure may be obtained using it, in particular in terms of the Green\u27s function of the Laplacian on a manifold. We also show how a smoothing procedure can be applied by propagating the heat equation on probability mass in order to get stronger bounds on transport distance using well-known properties of the heat equation. In Chapter 4, we turn to the primary question of the thesis: how to select points on a space which are as uniformly distributed as possible. We consider various diverse approaches one might attempt: an ergodic approach iterating functions with good mixing properties; a dyadic approach introduced in a 1975 theorem of Kakutani on proportional splittings on intervals; and a completely novel potential theoretic approach, assigning energy to point configurations and greedily minimizing the total potential arising from pair-wise point interactions. Such energy minimization questions are certainly not new, in the static setting--physicist Thomson posed the question of how to minimize the potential of electrons on a sphere as far back as 1904. However, a greedy approach to uniform distribution via energy minimization is novel, particularly through the lens of Wasserstein, and yields provably Wasserstein-optimal point sequences using the Green\u27s function of the Laplacian as our energy function on manifolds of dimension at least 3 (with dimension 2 losing at most a square root log factor from the optimal bound). We connect this to known results from Graham, Pausinger, and Proinov regarding best possible uniform bounds on the Wasserstein 2-distance of point sequences in the unit interval. We also present many open questions and conjectures on the optimal asymptotic bounds for total energy of point configurations and the growth of the total energy function as points are added, motivated by numerical investigations that display remarkably well-behaved qualities in the dynamical system induced by greedy minimization. In Chapter 5, we consider specific point sequences and bounds on the transport distance from the point measure they generate to the uniform measure. We provide provably optimal rates for the van der Corput sequence, the Kronecker sequence, regular grids and the measures induced by quadratic residues in a field of prime order. We also prove an upper bound for higher degree monomial residues in fields of prime order, and conjecture this to be optimal. In Chapter 6, we consider numerical integration error bounds over Lipschitz functions, asking how closely we can estimate the integral of a function by averaging its values at finitely many points. This is a rather classical question that was answered completely by Bakhalov in 1959 and has since become a standard example (`the easiest case which is perfectly understood\u27). Somewhat surprisingly perhaps, we show that the result is not sharp and improve it in two ways: by refining the function space and by proving that these results can be true uniformly along a subsequence. These bounds refine existing results that were widely considered to be optimal, and we show the intimate connection between transport distance and integration error. Our results are new even for the classical discrete grid. In Chapter 7, we study the case of finite graphs--we show that the fundamental question underlying this thesis can also be meaningfully posed on finite graphs where it leads to a fascinating combinatorial problem. We show that the philosophy introduced in Chapter 4 can be meaningfully adapted and obtain a potential-theoretic algorithm that produces such a sequence on graphs. We show that, using spectral techniques, we are able to obtain empirically strong bounds on the 1-Wasserstein distance between measures on subsets of vertices and the uniform measure, which for graphs of large diameter are much stronger than the trivial diameter bound

Data-Driven Estimation in Equilibrium Using Inverse Optimization

Equilibrium modeling is common in a variety of fields such as game theory and transportation science. The inputs for these models, however, are often difficult to estimate, while their outputs, i.e., the equilibria they are meant to describe, are often directly observable. By combining ideas from inverse optimization with the theory of variational inequalities, we develop an efficient, data-driven technique for estimating the parameters of these models from observed equilibria. We use this technique to estimate the utility functions of players in a game from their observed actions and to estimate the congestion function on a road network from traffic count data. A distinguishing feature of our approach is that it supports both parametric and \emph{nonparametric} estimation by leveraging ideas from statistical learning (kernel methods and regularization operators). In computational experiments involving Nash and Wardrop equilibria in a nonparametric setting, we find that a) we effectively estimate the unknown demand or congestion function, respectively, and b) our proposed regularization technique substantially improves the out-of-sample performance of our estimators.Comment: 36 pages, 5 figures Additional theorems for generalization guarantees and statistical analysis adde

Stochastic Distributed Optimization under Average Second-order Similarity: Algorithms and Analysis

We study finite-sum distributed optimization problems involving a master node and $n-1$ local nodes under the popular $\delta$-similarity and $\mu$-strong convexity conditions. We propose two new algorithms, SVRS and AccSVRS, motivated by previous works. The non-accelerated SVRS method combines the techniques of gradient sliding and variance reduction and achieves a better communication complexity of $\tilde{\mathcal{O}}(n {+} \sqrt{n}\delta/\mu)$ compared to existing non-accelerated algorithms. Applying the framework proposed in Katyusha X, we also develop a directly accelerated version named AccSVRS with the $\tilde{\mathcal{O}}(n {+} n^{3/4}\sqrt{\delta/\mu})$ communication complexity. In contrast to existing results, our complexity bounds are entirely smoothness-free and exhibit superiority in ill-conditioned cases. Furthermore, we establish a nearly matched lower bound to verify the tightness of our AccSVRS method.Comment: Camera-ready version for NeurIPS 202