Large-Scale Gaussian Processes via Alternating Projection

Gaussian process (GP) hyperparameter optimization requires repeatedly solving linear systems with $n \times n$ kernel matrices. To address the prohibitive $\mathcal{O}(n^3)$ time complexity, recent work has employed fast iterative numerical methods, like conjugate gradients (CG). However, as datasets increase in magnitude, the corresponding kernel matrices become increasingly ill-conditioned and still require $\mathcal{O}(n^2)$ space without partitioning. Thus, while CG increases the size of datasets GPs can be trained on, modern datasets reach scales beyond its applicability. In this work, we propose an iterative method which only accesses subblocks of the kernel matrix, effectively enabling \emph{mini-batching}. Our algorithm, based on alternating projection, has $\mathcal{O}(n)$ per-iteration time and space complexity, solving many of the practical challenges of scaling GPs to very large datasets. Theoretically, we prove our method enjoys linear convergence and empirically we demonstrate its robustness to ill-conditioning. On large-scale benchmark datasets up to four million datapoints our approach accelerates training by a factor of 2$\times$ to 27$\times$ compared to CG

Reducing the Variance of Gaussian Process Hyperparameter Optimization with Preconditioning

Gaussian processes remain popular as a flexible and expressive model class, but the computational cost of kernel hyperparameter optimization stands as a major limiting factor to their scaling and broader adoption. Recent work has made great strides combining stochastic estimation with iterative numerical techniques, essentially boiling down GP inference to the cost of (many) matrix-vector multiplies. Preconditioning -- a highly effective step for any iterative method involving matrix-vector multiplication -- can be used to accelerate convergence and thus reduce bias in hyperparameter optimization. Here, we prove that preconditioning has an additional benefit that has been previously unexplored. It not only reduces the bias of the $\log$-marginal likelihood estimator and its derivatives, but it also simultaneously can reduce variance at essentially negligible cost. We leverage this result to derive sample-efficient algorithms for GP hyperparameter optimization requiring as few as $\mathcal{O}(\log(\varepsilon^{-1}))$ instead of $\mathcal{O}(\varepsilon^{-2})$ samples to achieve error $\varepsilon$. Our theoretical results enable provably efficient and scalable optimization of kernel hyperparameters, which we validate empirically on a set of large-scale benchmark problems. There, variance reduction via preconditioning results in an order of magnitude speedup in hyperparameter optimization of exact GPs

Fabrication of Artificial Graphene in a GaAs Quantum Heterostructure

The unusual electronic properties of graphene, which are a direct consequence of its two-dimensional (2D) honeycomb lattice, have attracted a great deal of attention in recent years. Creation of artificial lattices that recreate graphene's honeycomb topology, known as artificial graphene, can facilitate the investigation of graphene-like phenomena, such as the existence of massless Dirac fermions, in a tunable system. In this work, we present the fabrication of artificial graphene in an ultra-high quality GaAs/AlGaAs quantum well, with lattice period as small as 50 nm, the smallest reported so far for this type of system. Electron-beam lithography is used to define an etch mask with honeycomb geometry on the surface of the sample, and different methodologies are compared and discussed. An optimized anisotropic reactive ion etching process is developed to transfer the pattern into the AlGaAs layer and create the artificial graphene. The achievement of such high-resolution artificial graphene should allow the observation for the first time of massless Dirac fermions in an engineered semiconductor.Comment: 13 pages text, 8 figures, plus reference