Learning to Convolve: A Generalized Weight-Tying Approach

Recent work (Cohen & Welling, 2016) has shown that generalizations of convolutions, based on group theory, provide powerful inductive biases for learning. In these generalizations, filters are not only translated but can also be rotated, flipped, etc. However, coming up with exact models of how to rotate a 3 x 3 filter on a square pixel-grid is difficult. In this paper, we learn how to transform filters for use in the group convolution, focussing on roto-translation. For this, we learn a filter basis and all rotated versions of that filter basis. Filters are then encoded by a set of rotation invariant coefficients. To rotate a filter, we switch the basis. We demonstrate we can produce feature maps with low sensitivity to input rotations, while achieving high performance on MNIST and CIFAR-10.Comment: Accepted to ICML 201

Constructing Hamiltonian quantum theories from path integrals in a diffeomorphism invariant context

Osterwalder and Schrader introduced a procedure to obtain a (Lorentzian) Hamiltonian quantum theory starting from a measure on the space of (Euclidean) histories of a scalar quantum field. In this paper, we extend that construction to more general theories which do not refer to any background, space-time metric (and in which the space of histories does not admit a natural linear structure). Examples include certain gauge theories, topological field theories and relativistic gravitational theories. The treatment is self-contained in the sense that an a priori knowledge of the Osterwalder-Schrader theorem is not assumed.Comment: Plain Latex, 25 p., references added, abstract and title changed (originally :``Osterwalder Schrader Reconstruction and Diffeomorphism Invariance''), introduction extended, one appendix with illustrative model added, accepted by Class. Quantum Gra

Generalization of Bloch's theorem for arbitrary boundary conditions: Interfaces and topological surface band structure

We describe a method for exactly diagonalizing clean $D$-dimensional lattice systems of independent fermions subject to arbitrary boundary conditions in one direction, as well as systems composed of two bulks meeting at a planar interface. Our method builds on the generalized Bloch theorem [A. Alase et al., Phys. Rev. B 96, 195133 (2017)] and the fact that the bulk-boundary separation of the Schrodinger equation is compatible with a partial Fourier transform operation. Bulk equations may display unusual features because they are relative eigenvalue problems for non-Hermitian, bulk-projected Hamiltonians. Nonetheless, they admit a rich symmetry analysis that can simplify considerably the structure of energy eigenstates, often allowing a solution in fully analytical form. We illustrate our extension of the generalized Bloch theorem to multicomponent systems by determining the exact Andreev bound states for a simple SNS junction. We then analyze the Creutz ladder model, by way of a conceptual bridge from one to higher dimensions. Upon introducing a new Gaussian duality transformation that maps the Creutz ladder to a system of two Majorana chains, we show how the model provides a first example of a short-range chiral topological insulator hosting topological zero modes with a power-law profile. Additional applications include the complete analytical diagonalization of graphene ribbons with both zigzag-bearded and armchair boundary conditions, and the analytical determination of the edge modes in a chiral $p+ip$ two-dimensional topological superconductor. Lastly, we revisit the phenomenon of Majorana flat bands and anomalous bulk-boundary correspondence in a two-band gapless $s$-wave topological superconductor. We analyze the equilibrium Josephson response of the system, showing how the presence of Majorana flat bands implies a substantial enhancement in the $4\pi$-periodic supercurrent.Comment: 20+9 pages, 10 figure

Vacuum orbit and spontaneous symmetry breaking in hyperbolic sigma models

We present a detailed study of quantized noncompact, nonlinear SO(1,N) sigma-models in arbitrary space-time dimensions D \geq 2, with the focus on issues of spontaneous symmetry breaking of boost and rotation elements of the symmetry group. The models are defined on a lattice both in terms of a transfer matrix and by an appropriately gauge-fixed Euclidean functional integral. The main results in all dimensions \geq 2 are: (i) On a finite lattice the systems have infinitely many nonnormalizable ground states transforming irreducibly under a nontrivial representation of SO(1,N); (ii) the SO(1,N) symmetry is spontaneously broken. For D =2 this shows that the systems evade the Mermin-Wagner theorem. In this case in addition: (iii) Ward identities for the Noether currents are derived to verify numerically the absence of explicit symmetry breaking; (iv) numerical results are presented for the two-point functions of the spin field and the Noether current as well as a new order parameter; (v) in a large N saddle-point analysis the dynamically generated squared mass is found to be negative and of order 1/(V \ln V) in the volume, the 0-component of the spin field diverges as \sqrt{\ln V}, while SO(1,N) invariant quantities remain finite.Comment: 60 pages, 12 Figures, AMS-Latex; v2: results on vacuum orbit and spontaneous symmetry breaking extended to all dimension

Thermal Quantum Fields without Cut-offs in 1+1 Space-time Dimensions

We construct interacting quantum fields in 1+1 dimensional Minkowski space, representing neutral scalar bosons at positive temperature. Our work is based on prior work by Klein and Landau and Hoegh-KrohnComment: 48 page

Sketching for Large-Scale Learning of Mixture Models

Learning parameters from voluminous data can be prohibitive in terms of memory and computational requirements. We propose a "compressive learning" framework where we estimate model parameters from a sketch of the training data. This sketch is a collection of generalized moments of the underlying probability distribution of the data. It can be computed in a single pass on the training set, and is easily computable on streams or distributed datasets. The proposed framework shares similarities with compressive sensing, which aims at drastically reducing the dimension of high-dimensional signals while preserving the ability to reconstruct them. To perform the estimation task, we derive an iterative algorithm analogous to sparse reconstruction algorithms in the context of linear inverse problems. We exemplify our framework with the compressive estimation of a Gaussian Mixture Model (GMM), providing heuristics on the choice of the sketching procedure and theoretical guarantees of reconstruction. We experimentally show on synthetic data that the proposed algorithm yields results comparable to the classical Expectation-Maximization (EM) technique while requiring significantly less memory and fewer computations when the number of database elements is large. We further demonstrate the potential of the approach on real large-scale data (over 10 8 training samples) for the task of model-based speaker verification. Finally, we draw some connections between the proposed framework and approximate Hilbert space embedding of probability distributions using random features. We show that the proposed sketching operator can be seen as an innovative method to design translation-invariant kernels adapted to the analysis of GMMs. We also use this theoretical framework to derive information preservation guarantees, in the spirit of infinite-dimensional compressive sensing

Discrete Euler-Poincar\'{e} and Lie-Poisson Equations

In this paper, discrete analogues of Euler-Poincar\'{e} and Lie-Poisson reduction theory are developed for systems on finite dimensional Lie groups $G$ with Lagrangians $L:TG \to {\mathbb R}$ that are $G$-invariant. These discrete equations provide ``reduced'' numerical algorithms which manifestly preserve the symplectic structure. The manifold $G \times G$ is used as an approximation of $TG$, and a discrete Langragian ${\mathbb L}:G \times G \to {\mathbb R}$ is construced in such a way that the $G$-invariance property is preserved. Reduction by $G$ results in new ``variational'' principle for the reduced Lagrangian $\ell:G \to {\mathbb R}$, and provides the discrete Euler-Poincar\'{e} (DEP) equations. Reconstruction of these equations recovers the discrete Euler-Lagrange equations developed in \cite{MPS,WM} which are naturally symplectic-momentum algorithms. Furthermore, the solution of the DEP algorithm immediately leads to a discrete Lie-Poisson (DLP) algorithm. It is shown that when $G=\text{SO} (n)$, the DEP and DLP algorithms for a particular choice of the discrete Lagrangian ${\mathbb L}$ are equivalent to the Moser-Veselov scheme for the generalized rigid body. %As an application, a reduced symplectic integrator for two dimensional %hydrodynamics is constructed using the SU$(n)$ approximation to the volume %preserving diffeomorphism group of ${\mathbb T}^2$