Towards Fast Computation of Certified Robustness for ReLU Networks

Verifying the robustness property of a general Rectified Linear Unit (ReLU) network is an NP-complete problem [Katz, Barrett, Dill, Julian and Kochenderfer CAV17]. Although finding the exact minimum adversarial distortion is hard, giving a certified lower bound of the minimum distortion is possible. Current available methods of computing such a bound are either time-consuming or delivering low quality bounds that are too loose to be useful. In this paper, we exploit the special structure of ReLU networks and provide two computationally efficient algorithms Fast-Lin and Fast-Lip that are able to certify non-trivial lower bounds of minimum distortions, by bounding the ReLU units with appropriate linear functions Fast-Lin, or by bounding the local Lipschitz constant Fast-Lip. Experiments show that (1) our proposed methods deliver bounds close to (the gap is 2-3X) exact minimum distortion found by Reluplex in small MNIST networks while our algorithms are more than 10,000 times faster; (2) our methods deliver similar quality of bounds (the gap is within 35% and usually around 10%; sometimes our bounds are even better) for larger networks compared to the methods based on solving linear programming problems but our algorithms are 33-14,000 times faster; (3) our method is capable of solving large MNIST and CIFAR networks up to 7 layers with more than 10,000 neurons within tens of seconds on a single CPU core. In addition, we show that, in fact, there is no polynomial time algorithm that can approximately find the minimum $\ell_1$ adversarial distortion of a ReLU network with a $0.99\ln n$ approximation ratio unless $\mathsf{NP}$=$\mathsf{P}$, where $n$ is the number of neurons in the network.Comment: Tsui-Wei Weng and Huan Zhang contributed equall

How Powerful is Adiabatic Quantum Computation?

We analyze the computational power and limitations of the recently proposed 'quantum adiabatic evolution algorithm'.Comment: 12 pages, LaTeX2e, requires fullpage, times, amssymb, and amsmath packages. This article appeared in the proceedings of FOCS'01; original submission date: April 27, 200

Tensor Network States: Optimizations and Applications in Quantum Many-Body Physics and Machine Learning

Tensor network states are ubiquitous in the investigation of quantum many-body (QMB) physics. Their advantage over other state representations is evident from their reduction in the computational complexity required to obtain various quantities of interest, namely observables. Additionally, they provide a natural platform for investigating entanglement properties within a system. In this dissertation, we develop various novel algorithms and optimizations to tensor networks for the investigation of QMB systems, including classical and quantum circuits. Specifically, we study optimizations for the two-dimensional Ising model in a transverse field, we create an algorithm for the $k$-SAT problem, and we study the entanglement properties of random unitary circuits. In addition to these applications, we reinterpret renormalization group principles from QMB physics in the context of machine learning to develop a novel algorithm for the tasks of classification and regression, and then utilize machine learning architectures for the time evolution of operators in QMB systems

Picturing counting reductions with the ZH-calculus

Counting the solutions to Boolean formulae defines the problem #SAT, which is complete for the complexity class #P. We use the ZH-calculus, a universal and complete graphical language for linear maps which naturally encodes counting problems in terms of diagrams, to give graphical reductions from #SAT to several related counting problems. Some of these graphical reductions, like to #2SAT, are substantially simpler than known reductions via the matrix permanent. Additionally, our approach allows us to consider the case of counting solutions modulo an integer on equal footing. Finally, since the ZH-calculus was originally introduced to reason about quantum computing, we show that the problem of evaluating ZH-diagrams in the fragment corresponding to the Clifford+T gateset, is in $FP^{\#P}$. Our results show that graphical calculi represent an intuitive and useful framework for reasoning about counting problems

Quantum Invariants of 3-manifolds and NP vs #P

The computational complexity class #P captures the difficulty of counting the satisfying assignments to a boolean formula. In this work, we use basic tools from quantum computation to give a proof that the SO(3) Witten-Reshetikhin-Turaev (WRT) invariant of 3-manifolds is #P-hard to calculate. We then apply this result to a question about the combinatorics of Heegaard splittings, motivated by analogous work on link diagrams by M. Freedman. We show that, if $\#\text{P}\neq\text{FP}^\text{NP}$, then there exist infinitely many Heegaard splittings which cannot be made logarithmically thin by local WRT-preserving moves, except perhaps via a superpolynomial number of steps. We also outline two extensions of the above results. First, adapting a result of Kuperberg, we show that any presentation-independent approximation of WRT is also #P-hard. Second, we sketch out how all of our results can be translated to the setting of triangulations and Turaev-Viro invariants.Comment: 22 pages, 5 figure

Nonuniversal entanglement level statistics in projection-driven quantum circuits and glassy dynamics in classical computation circuits

In this thesis, I describe research results on three topics : (i) a phase transition in the area-law regime of quantum circuits driven by projection measurements; (ii) ultra slow dynamics in two dimensional spin circuits; and (iii) tensor network methods applied to boolean satisfiability problems. (i) Nonuniversal entanglement level statistics in projection-driven quantum circuits; Non-thermalized closed quantum many-body systems have drawn considerable attention, due to their relevance to experimentally controllable quantum systems. In the first part of the thesis, we study the level-spacing statistics in the entanglement spectrum of output states of random universal quantum circuits where, at each time step, qubits are subject to a finite probability of projection onto states of the computational basis. We encounter two phase transitions with increasing projection rate: The first is the volume-to-area law transition observed in quantum circuits with projective measurements; The second separates the pure Poisson level statistics phase at large projective measurement rates from a regime of residual level repulsion in the entanglement spectrum within the area-law phase, characterized by non-universal level spacing statistics that interpolates between the Wigner-Dyson and Poisson distributions. The same behavior is observed in both circuits of random two-qubit unitaries and circuits of universal gates, including the set implemented by Google in its Sycamore circuits. (ii) Ultra-slow dynamics in a translationally invariant spin model for multiplication and factorization; Slow relaxation of glassy systems in the absence of disorder remains one of the most intriguing problems in condensed matter physics. In the second part of the thesis we investigate slow relaxation in a classical model of short-range interacting Ising spins on a translationally invariant two-dimensional lattice that mimics a reversible circuit that, depending on the choice of boundary conditions, either multiplies or factorizes integers. We prove that, for open boundary conditions, the model exhibits no finite-temperature phase transition. Yet we find that it displays glassy dynamics with astronomically slow relaxation times, numerically consistent with a double exponential dependence on the inverse temperature. The slowness of the dynamics arises due to errors that occur during thermal annealing that cost little energy but flip an extensive number of spins. We argue that the energy barrier that needs to be overcome in order to heal such defects scales linearly with the correlation length, which diverges exponentially with inverse temperature, thus yielding the double exponential behavior of the relaxation time. (iii) Reversible circuit embedding on tensor networks for Boolean satisfiability; Finally, in the third part of the thesis we present an embedding of Boolean satisfiability (SAT) problems on a two-dimensional tensor network. The embedding uses reversible circuits encoded into the tensor network whose trace counts the number of solutions of the satisfiability problem. We specifically present the formulation of #2SAT, #3SAT, and #3XORSAT formulas into planar tensor networks. We use a compression-decimation algorithm introduced by us to propagate constraints in the network before coarse-graining the boundary tensors. Iterations of these two steps gradually collapse the network while slowing down the growth of bond dimensions. For the case of #3XORSAT, we show numerically that this procedure recognizes, at least partially, the simplicity of XOR constraints for which it achieves subexponential time to solution. For a #P-complete subset of #2SAT we find that our algorithm scales with size in the same way as state-of-the-art #SAT counters, albeit with a larger prefactor. We find that the compression step performs less efficiently for #3SAT than for #2SAT

Provably efficient machine learning for quantum many-body problems

Classical machine learning (ML) provides a potentially powerful approach to solving challenging quantum many-body problems in physics and chemistry. However, the advantages of ML over more traditional methods have not been firmly established. In this work, we prove that classical ML algorithms can efficiently predict ground state properties of gapped Hamiltonians in finite spatial dimensions, after learning from data obtained by measuring other Hamiltonians in the same quantum phase of matter. In contrast, under widely accepted complexity theory assumptions, classical algorithms that do not learn from data cannot achieve the same guarantee. We also prove that classical ML algorithms can efficiently classify a wide range of quantum phases of matter. Our arguments are based on the concept of a classical shadow, a succinct classical description of a many-body quantum state that can be constructed in feasible quantum experiments and be used to predict many properties of the state. Extensive numerical experiments corroborate our theoretical results in a variety of scenarios, including Rydberg atom systems, 2D random Heisenberg models, symmetry-protected topological phases, and topologically ordered phases.Comment: 10 pages, 12 figures + 57 page appendi