A Tutorial on Sparse Gaussian Processes and Variational Inference

Gaussian processes (GPs) provide a framework for Bayesian inference that can offer principled uncertainty estimates for a large range of problems. For example, if we consider regression problems with Gaussian likelihoods, a GP model enjoys a posterior in closed form. However, identifying the posterior GP scales cubically with the number of training examples and requires to store all examples in memory. In order to overcome these obstacles, sparse GPs have been proposed that approximate the true posterior GP with pseudo-training examples. Importantly, the number of pseudo-training examples is user-defined and enables control over computational and memory complexity. In the general case, sparse GPs do not enjoy closed-form solutions and one has to resort to approximate inference. In this context, a convenient choice for approximate inference is variational inference (VI), where the problem of Bayesian inference is cast as an optimization problem -- namely, to maximize a lower bound of the log marginal likelihood. This paves the way for a powerful and versatile framework, where pseudo-training examples are treated as optimization arguments of the approximate posterior that are jointly identified together with hyperparameters of the generative model (i.e. prior and likelihood). The framework can naturally handle a wide scope of supervised learning problems, ranging from regression with heteroscedastic and non-Gaussian likelihoods to classification problems with discrete labels, but also multilabel problems. The purpose of this tutorial is to provide access to the basic matter for readers without prior knowledge in both GPs and VI. A proper exposition to the subject enables also access to more recent advances (like importance-weighted VI as well as interdomain, multioutput and deep GPs) that can serve as an inspiration for new research ideas

Mathematical Imaging and Surface Processing

Within the last decade image and geometry processing have become increasingly rigorous with solid foundations in mathematics. Both areas are research fields at the intersection of different mathematical disciplines, ranging from geometry and calculus of variations to PDE analysis and numerical analysis. The workshop brought together scientists from all these areas and a fruitful interplay took place. There was a lively exchange of ideas between geometry and image processing applications areas, characterized in a number of ways in this workshop. For example, optimal transport, first applied in computer vision is now used to define a distance measure between 3d shapes, spectral analysis as a tool in image processing can be applied in surface classification and matching, and so on. We have also seen the use of Riemannian geometry as a powerful tool to improve the analysis of multivalued images. This volume collects the abstracts for all the presentations covering this wide spectrum of tools and application domains

Productive and efficient computational science through domain-specific abstractions

In an ideal world, scientific applications are computationally efficient, maintainable and composable and allow scientists to work very productively. We argue that these goals are achievable for a specific application field by choosing suitable domain-specific abstractions that encapsulate domain knowledge with a high degree of expressiveness. This thesis demonstrates the design and composition of domain-specific abstractions by abstracting the stages a scientist goes through in formulating a problem of numerically solving a partial differential equation. Domain knowledge is used to transform this problem into a different, lower level representation and decompose it into parts which can be solved using existing tools. A system for the portable solution of partial differential equations using the finite element method on unstructured meshes is formulated, in which contributions from different scientific communities are composed to solve sophisticated problems. The concrete implementations of these domain-specific abstractions are Firedrake and PyOP2. Firedrake allows scientists to describe variational forms and discretisations for linear and non-linear finite element problems symbolically, in a notation very close to their mathematical models. PyOP2 abstracts the performance-portable parallel execution of local computations over the mesh on a range of hardware architectures, targeting multi-core CPUs, GPUs and accelerators. Thereby, a separation of concerns is achieved, in which Firedrake encapsulates domain knowledge about the finite element method separately from its efficient parallel execution in PyOP2, which in turn is completely agnostic to the higher abstraction layer. As a consequence of the composability of those abstractions, optimised implementations for different hardware architectures can be automatically generated without any changes to a single high-level source. Performance matches or exceeds what is realistically attainable by hand-written code. Firedrake and PyOP2 are combined to form a tool chain that is demonstrated to be competitive with or faster than available alternatives on a wide range of different finite element problems.Open Acces

Graph Signal Processing: Overview, Challenges and Applications

Research in Graph Signal Processing (GSP) aims to develop tools for processing data defined on irregular graph domains. In this paper we first provide an overview of core ideas in GSP and their connection to conventional digital signal processing. We then summarize recent developments in developing basic GSP tools, including methods for sampling, filtering or graph learning. Next, we review progress in several application areas using GSP, including processing and analysis of sensor network data, biological data, and applications to image processing and machine learning. We finish by providing a brief historical perspective to highlight how concepts recently developed in GSP build on top of prior research in other areas.Comment: To appear, Proceedings of the IEE

CUPOLETS: Chaotic unstable periodic orbits theory and applications

Recent theoretical work suggests that periodic orbits of chaotic systems are a rich source of qualitative information about the dynamical system. The presence of unstable periodic orbits located densely on the attractor is a typical characteristic of chaotic systems. This abundance of unstable periodic orbits can be utilized in a wide variety of theoretical and practical applications [19]. In particular, chaotic communication techniques and methods of controlling chaos depend on this property of chaotic attractors [12, 13]. In the first part of this thesis, a control scheme for stabilizing the unstable periodic orbits of chaotic systems is presented and the properties of these orbits are investigated. The technique allows for creation of thousands of periodic orbits. These approximated chaotic unstable periodic orbits are called cupolets (C&barbelow;haotic U&barbelow;nstable P&barbelow;eriodic O&barbelow;rbit- lets). We show that these orbits can be passed through a phase transformation to a compact cupolet state that possesses a wavelet-like structure and can be used to construct adaptive bases. The cupolet transformation can be regarded as an alternative to Fourier and wavelet transformations. In fact, this new framework provides a continuum between Fourier and wavelet transformations and can be used in variety of applications such as data and music compression, as well as image and video processing. The key point in this method is that all of these different dynamical behaviors are easily accessible via small controls. This technique is implemented in order to produce cupolets which are essentially approximate periodic orbits of the chaotic system. The orbits are produced with small perturbations which in turn suggests that these orbits might not be very far away from true periodic orbits. The controls can be considered as external numerical errors that happen at some points along the computer generated orbits. This raises the question of shadowability of these orbits. It is very interesting to know if there exists a true orbit of the system with a slightly different initial condition that stays close to the computer generated orbit. This true orbit, if it exists, is called a shadow and the computer generated orbit is then said to be shadowable by a true orbit. We will present two general purpose shadowing theorems for periodic and nonperiodic orbits of ordinary differential equations. The theorems provide a way to establish the existence of true periodic and non-periodic orbits near the approximated ones. Both theorems are suitable for computations and the shadowing distances, i.e., the distance between the true orbits and approximated orbits are given by quantities computable form the vector field of the differential equation

Quantum computational chemistry methods for early-stage quantum computers

One of the first practical applications of quantum computers is expected to be molecular modelling. Performing this task would profoundly affect areas such as chemistry, materials science and drug synthesis. Modelling of molecules, which are classically intractable, can be achieved with just over $30$ qubits, whereas state of the art quantum computers already have more than $50$ qubits. The Variational Quantum Eigensolver (VQE) algorithm and VQE based protocols, are promising candidates to enable this task on emerging Noisy Intermediate-Scale Quantum (NISQ) computers. These protocols require short quantum circuits and short coherence times, and are particularly resilient to quantum errors. Nevertheless, there is still a significant gap between the accuracy and the coherence times of current NISQ computers, and the hardware requirements of VQE protocols to simulate practically interesting molecules. In this thesis, I present my contribution to narrowing this gap by developing VQE protocols for molecular modelling that are less demanding on quantum hardware. The VQE relies on the Rayleigh-Ritz variational principle to estimate the eigenvalues of a Hamiltonian operator, by minimizing its expectation value with respect to a trial quantum state, prepared by an ansatz. A major challenge for the practical realisation of VQE protocols on NISQ computers is to construct an ansatz that: (1) can accurately approximate the eigenstates of the Hamiltonian; (2) is easy to optimize; and (3) can be implemented by a shallow circuit, within the capabilities of a NISQ computer. The most widely used, unitary coupled cluster (UCC), type of ans\"atze mathematically correspond to a product of unitary evolutions of fermionic excitation operators. Owing to their fermionic structure, UCC ans\"atze preserve the symmetries of electronic wavefunctions, and thus are accurate and easy to optimize. Nevertheless, UCC ans\"atze are implemented by high depth circuits, which severely limit the size of the molecules that can be reliably simulated on NISQ computers. In this thesis, I begin by constructing efficient quantum circuits to perform evolutions of fermionic excitation operators. The circuits are optimized in the number of two-qubit entangling gates, which are the current bottleneck of NISQ computers. Compared to the standard circuits used to implement evolutions of fermionic excitation operators, the circuits derived in this thesis reduce the number of two-qubit entangling gates by more than $70\%$ on average. As an intermediate result, I also derive efficient circuits to perform evolutions of qubit excitation operators (excitation operators that account for qubit, rather than fermionic commutation relations). Even with the fermionic-excitation-evolution circuits derived here, UCC ans\"atze still require very long circuits, with a particularly large number of two-qubit entangling gates. In this thesis, I consider the use of alternative VQE ans\"atze, based on evolutions of qubit excitation operators. Due to not accounting for fermionic anticommutation, evolutions of qubit excitation operators can be performed by circuits that require asymptotically fewer two-qubit entangling gates. Furthermore, qubit excitation operators preserve many of the physical properties of fermionic excitation operators. Performing a number of classical numerical VQE simulations for small molecules, I show that qubit-excitation-based ans\"atze can approximate molecular electronic wavefunctions almost as accurately as fermionic-excitation-based ans\"atze. Hence, I argue that evolutions of qubit excitation operators are more suitable to construct molecular ans\"atze than evolutions of fermionic excitation operators, especially in the era of NISQ computers. Motivated by the advantage of qubit-excitation-based ans\"atze, I introduce the qubit-excitation-based adaptive variational quantum eigensolver (QEB-ADAPT-VQE). The QEB-ADAPT-VQE belongs to a family of ADAPT-VQE protocols for molecular modelling that grow a problem-tailored ansatz by iteratively appending unitary operators sampled from a predefined finite-size pool of operators. The operator at each iteration is sampled based on an ansatz-growing strategy, which aims to achieve the lowest estimate for the Hamiltonian expectation value at each iteration. In this way, ADAPT-VQE protocols construct shallow-circuit, few-parameter ans\"atze tailored specifically to the molecular systems of interest. In the case of the QEB-ADAPT-VQE, the operator pool is defined by a set of evolutions of single and double qubit excitation operators. I benchmark the performance of the QEB-ADAPT-VQE, by performing classical numerical simulations. I demonstrate that it can construct ans\"atze that are several orders of magnitude more accurate, and require significantly shallower circuits, than standard UCC ans\"atze. I also compare the QEB-ADAPT-VQE against the original fermionic-ADAPT-VQE, which utilizes a pool of fermionic excitation evolutions, and the qubit-ADAPT-VQE, which utilizes a pool of Pauli-string evolutions. I demonstrate that, in terms of circuit efficiency and convergence speed, the QEB-ADAPT-VQE systematically outperforms the qubit-ADAPT-VQE, which to my knowledge was the previous most circuit-efficient, scalable VQE protocol for molecular modeling. The QEB-ADAPT-VQE protocol, therefore represents a significant improvement in the field of VQE protocols for molecular modelling and brings us closer to achieving practical quantum advantage. Lastly, I outline a modified version of the QEB-ADAPT-VQE, the excited-QEB-ADAPT-VQE, designed to estimate energies of excited molecular states. The excited-QEB-ADAPT-VQE is more robust to initial simulation conditions, at the expense of increased computational complexity.I acknowledge the funding I received from the Engineering and Physical Sciences Research Council, and Hitachi Cambridge Laborator