A Variational Perspective on Accelerated Methods in Optimization

Accelerated gradient methods play a central role in optimization, achieving optimal rates in many settings. While many generalizations and extensions of Nesterov's original acceleration method have been proposed, it is not yet clear what is the natural scope of the acceleration concept. In this paper, we study accelerated methods from a continuous-time perspective. We show that there is a Lagrangian functional that we call the \emph{Bregman Lagrangian} which generates a large class of accelerated methods in continuous time, including (but not limited to) accelerated gradient descent, its non-Euclidean extension, and accelerated higher-order gradient methods. We show that the continuous-time limit of all of these methods correspond to traveling the same curve in spacetime at different speeds. From this perspective, Nesterov's technique and many of its generalizations can be viewed as a systematic way to go from the continuous-time curves generated by the Bregman Lagrangian to a family of discrete-time accelerated algorithms.Comment: 38 pages. Subsumes an earlier working draft arXiv:1509.0361

Non-classical behavior in multimode and disordered transverse structures in OPO. Use of the Q representation

We employ the Q representation to study the non-classical correlations that are present from below to above-threshold in the degenerate optical parametric oscillator. Our study shows that such correlations are present just above threshold, in the regime in which stripe patterns are formed, but that they also persist further above threshold in the presence of spatially disordered structures

Adam through a Second-Order Lens

Research into optimisation for deep learning is characterised by a tension between the computational efficiency of first-order, gradient-based methods (such as SGD and Adam) and the theoretical efficiency of second-order, curvature-based methods (such as quasi-Newton methods and K-FAC). We seek to combine the benefits of both approaches into a single computationally-efficient algorithm. Noting that second-order methods often depend on stabilising heuristics (such as Levenberg-Marquardt damping), we propose AdamQLR: an optimiser combining damping and learning rate selection techniques from K-FAC (Martens and Grosse, 2015) with the update directions proposed by Adam, inspired by considering Adam through a second-order lens. We evaluate AdamQLR on a range of regression and classification tasks at various scales, achieving competitive generalisation performance vs runtime.Comment: 28 pages, 15 figures, 4 tables. Submitted to ICLR 202

Detecting gravitational waves from highly eccentric compact binaries

In dense stellar regions, highly eccentric binaries of black holes and neutron stars can form through various n-body interactions. Such a binary could emit a significant fraction of its binding energy in a sequence of largely isolated gravitational wave bursts prior to merger. Given expected black hole and neutron star masses, many such systems will emit these repeated bursts at frequencies within the sensitive band of contemporary ground-based gravitational wave detectors. Unfortunately, existing gravitational wave searches are ill-suited to detect these signals. In this work, we adapt a "power stacking" method to the detection of gravitational wave signals from highly eccentric binaries. We implement this method as an extension of the Q-transform, a projection onto a multiresolution basis of windowed complex exponentials that has previously been used to analyze data from the network of LIGO/Virgo detectors. Our method searches for excess power over an ensemble of time-frequency tiles. We characterize the performance of our method using Monte Carlo experiments with signals injected in simulated detector noise. Our results indicate that the power stacking method achieves substantially better sensitivity to eccentric binary signals than existing localized burst searches.Comment: 17 pages, 20 figure

Head-on collisions of boson stars

We study head-on collisions of boson stars in three dimensions. We consider evolutions of two boson stars which may differ in their phase or have opposite frequencies but are otherwise identical. Our studies show that these phase differences result in different late time behavior and gravitational wave output

Shape Animation with Combined Captured and Simulated Dynamics

We present a novel volumetric animation generation framework to create new types of animations from raw 3D surface or point cloud sequence of captured real performances. The framework considers as input time incoherent 3D observations of a moving shape, and is thus particularly suitable for the output of performance capture platforms. In our system, a suitable virtual representation of the actor is built from real captures that allows seamless combination and simulation with virtual external forces and objects, in which the original captured actor can be reshaped, disassembled or reassembled from user-specified virtual physics. Instead of using the dominant surface-based geometric representation of the capture, which is less suitable for volumetric effects, our pipeline exploits Centroidal Voronoi tessellation decompositions as unified volumetric representation of the real captured actor, which we show can be used seamlessly as a building block for all processing stages, from capture and tracking to virtual physic simulation. The representation makes no human specific assumption and can be used to capture and re-simulate the actor with props or other moving scenery elements. We demonstrate the potential of this pipeline for virtual reanimation of a real captured event with various unprecedented volumetric visual effects, such as volumetric distortion, erosion, morphing, gravity pull, or collisions