Quantum machine learning: a classical perspective

Recently, increased computational power and data availability, as well as algorithmic advances, have led machine learning techniques to impressive results in regression, classification, data-generation and reinforcement learning tasks. Despite these successes, the proximity to the physical limits of chip fabrication alongside the increasing size of datasets are motivating a growing number of researchers to explore the possibility of harnessing the power of quantum computation to speed-up classical machine learning algorithms. Here we review the literature in quantum machine learning and discuss perspectives for a mixed readership of classical machine learning and quantum computation experts. Particular emphasis will be placed on clarifying the limitations of quantum algorithms, how they compare with their best classical counterparts and why quantum resources are expected to provide advantages for learning problems. Learning in the presence of noise and certain computationally hard problems in machine learning are identified as promising directions for the field. Practical questions, like how to upload classical data into quantum form, will also be addressed.Comment: v3 33 pages; typos corrected and references adde

Learning Multiple Visual Tasks while Discovering their Structure

Multi-task learning is a natural approach for computer vision applications that require the simultaneous solution of several distinct but related problems, e.g. object detection, classification, tracking of multiple agents, or denoising, to name a few. The key idea is that exploring task relatedness (structure) can lead to improved performances. In this paper, we propose and study a novel sparse, non-parametric approach exploiting the theory of Reproducing Kernel Hilbert Spaces for vector-valued functions. We develop a suitable regularization framework which can be formulated as a convex optimization problem, and is provably solvable using an alternating minimization approach. Empirical tests show that the proposed method compares favorably to state of the art techniques and further allows to recover interpretable structures, a problem of interest in its own right.Comment: 19 pages, 3 figures, 3 table

Variational Data Assimilation via Sparse Regularization

This paper studies the role of sparse regularization in a properly chosen basis for variational data assimilation (VDA) problems. Specifically, it focuses on data assimilation of noisy and down-sampled observations while the state variable of interest exhibits sparsity in the real or transformed domain. We show that in the presence of sparsity, the $\ell_{1}$-norm regularization produces more accurate and stable solutions than the classic data assimilation methods. To motivate further developments of the proposed methodology, assimilation experiments are conducted in the wavelet and spectral domain using the linear advection-diffusion equation

MLP-BASED SOURCE SEPARATION FOR MLP-LIKE NONLINEAR MIXTURES

In this paper, the nonlinear blind source separation problem is addressed by using a multilayer perceptron (MLP) as separating system, which is justified in the universal approximation property of MLP networks. An adaptive learning algorithm for a perceptron with two hidden-layers is presented. The algorithm minimizes the mutual information between the outputs of the MLP. The performance of the proposed method is illustrated by some experiments. 1. INTRODUCTION. Blind Source Separation (BSS) is a fundamental problem in signal processing. It consists of retrieving unobserved sources s1(t),..., sN (t), assumed to be statistically independent (which is phisically plausible when the source

Statistical unfolding of elementary particle spectra: Empirical Bayes estimation and bias-corrected uncertainty quantification

We consider the high energy physics unfolding problem where the goal is to estimate the spectrum of elementary particles given observations distorted by the limited resolution of a particle detector. This important statistical inverse problem arising in data analysis at the Large Hadron Collider at CERN consists in estimating the intensity function of an indirectly observed Poisson point process. Unfolding typically proceeds in two steps: one first produces a regularized point estimate of the unknown intensity and then uses the variability of this estimator to form frequentist confidence intervals that quantify the uncertainty of the solution. In this paper, we propose forming the point estimate using empirical Bayes estimation which enables a data-driven choice of the regularization strength through marginal maximum likelihood estimation. Observing that neither Bayesian credible intervals nor standard bootstrap confidence intervals succeed in achieving good frequentist coverage in this problem due to the inherent bias of the regularized point estimate, we introduce an iteratively bias-corrected bootstrap technique for constructing improved confidence intervals. We show using simulations that this enables us to achieve nearly nominal frequentist coverage with only a modest increase in interval length. The proposed methodology is applied to unfolding the $Z$ boson invariant mass spectrum as measured in the CMS experiment at the Large Hadron Collider.Comment: Published at http://dx.doi.org/10.1214/15-AOAS857 in the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org). arXiv admin note: substantial text overlap with arXiv:1401.827

Graph Tikhonov Regularization and Interpolation via Random Spanning Forests

Novel Monte Carlo estimators are proposed to solve both the Tikhonov regularization (TR) and the interpolation problems on graphs. These estimators are based on random spanning forests (RSF), the theoretical properties of which enable to analyze the estimators' theoretical mean and variance. We also show how to perform hyperparameter tuning for these RSF-based estimators. TR is a component in many well-known algorithms, and we show how the proposed estimators can be easily adapted to avoid expensive intermediate steps in generalized semi-supervised learning, label propagation, Newton's method and iteratively reweighted least squares. In the experiments, we illustrate the proposed methods on several problems and provide observations on their run time

Avoiding local minima in variational quantum eigensolvers with the natural gradient optimizer

We compare the BFGS optimizer, ADAM and Natural Gradient Descent (NatGrad) in the context of Variational Quantum Eigensolvers (VQEs). We systematically analyze their performance on the QAOA ansatz for the Transverse Field Ising Model (TFIM) as well as on overparametrized circuits with the ability to break the symmetry of the Hamiltonian. The BFGS algorithm is frequently unable to find a global minimum for systems beyond about 20 spins and ADAM easily gets trapped in local minima. On the other hand, NatGrad shows stable performance on all considered system sizes, albeit at a significantly higher cost per epoch. In sharp contrast to most classical gradient based learning, the performance of all optimizers is found to decrease upon seemingly benign overparametrization of the ansatz class, with BFGS and ADAM failing more often and more severely than NatGrad. Additional tests for the Heisenberg XXZ model corroborate the accuracy problems of BFGS in high dimensions, but they reveal some shortcomings of NatGrad as well. Our results suggest that great care needs to be taken in the choice of gradient based optimizers and the parametrization for VQEs.Comment: 16 pages, 6 figures, 1 tabl

On the Statistical Approximation of Conditional Expectation Operators

Diese Dissertation erörtert die datengetriebene Approximation des sogenannten conditional expectation operators, welcher den Erwartungswert einer reellwertigen Transformation einer Zufallsvariablen bedingt auf eine zweite Zufallsvariable beschreibt. Sie stellt dieses klassische numerische Problem in einem neuen theoretischen Zusammenhang dar und beleuchtet es mit verschiedenen ausgewählten Methoden der modernen statistischen Lerntheorie. Es werden sowohl ein bekannter parametrischer Projektionsansatz aus dem numerischen Bereich als auch ein nichtparametrisches Modell auf Basis eines reproducing kernel Hilbert space untersucht. Die Untersuchungen dieser Arbeit werden motiviert duch den speziellen Fall, in dem der conditional expectation operator die Übergangswahrscheinlichkeiten eines Markovprozesses beschreibt. In diesem Kontext sind die Spektraleigenschaften des resultierenden Markov transition operators von großem praktischen Interesse zur datenbasierten Untersuchung von komplexer Dynamik. Die oben genannten vorgestellten Schätzer werden in diesem Szenario in der Praxis verwendet. Es werden diverse neue Konvergenz- und Approximationsresultate sowohl für stochastisch unabhängige als auch abhängige Daten gezeigt. Als Werkzeuge für diese Ergebnisse dienen Konzepte aus den Theorien inverser Probleme, schwach abhängiger stochastischer Prozesse, der St ̈orung von Spektraleigenschaften und der Konzentration von Wahrscheinlichkeitsmaßen. Zur theoretischen Rechtfertigung des nichtparametrischen Modells wird das Schätzen von kernel autocovariance operators von stationären Zeitreihen untersucht. Diese Betrachtung kann zusätzlich vielfältig in anderen Zusammenhängen genutzt werden, was anhand von neuen Ergebnissen zur Konsistenz von kernelbasierter Hauptkomponentenanalyse mit schwach abhängigen Daten demonstriert wird. Diese Dissertation ist theoretischer Natur und dient nicht zur unmittelbaren Umsetzung von neuen numerischen Methoden. Sie stellt jedoch den direkten Zusammenhang von bekannten Ansätzen in diesem Feld zu relevanten statistischen Arbeiten der letzten Jahre her, welche sowohl stärkere theoretische Ergebnisse als auch effizientere praktische Schätzer für dieses Problem in der Zukunft möglich machen.This dissertation discusses the data-driven approximation of the so-called conditional expectation operator, which describes the expected value of a real-valued transformation of a random variable conditioned on a second random variable. It presents this classical numerical problem in a new theoretical context and examines it using various selected methods from modern statistical learning theory. Both a well-known parametric projection approach from the numerical domain and a nonparametric model based on a reproducing kernel Hilbert space are investigated. The investigations of this work are motivated by the special case in which the conditional expectation operator describes the transition probabilities of a Markov process. In this context, the spectral properties of the resulting Markov transition operator are of great practical interest for the data-based study of complex dynamics. The presented estimators are used in practice in this scenario. Various new convergence and approximation results are shown for both stochastically independent and dependent data. Concepts from the theories of inverse problems, weakly dependent stochastic processes, spectral perturbation, and concentration of measure serve as tools for these results. For the theoretical justification of the nonparametric model, the estimation of kernel autocovariance operators of stationary time series is investigated. This consideration can additionally be used in a variety of ways in other contexts, which is demonstrated in terms of new results on the consistency of kernel-based principal component analysis with weakly dependent data. This dissertation is theoretical in nature and does not serve to directly implement new numerical methods. It does, however, provide a direct link from known approaches in this field to relevant statistical work from recent years, which will make both stronger theoretical results and more efficient practical estimators for this problem possible in the future