25 research outputs found
Sharp Calibrated Gaussian Processes
While Gaussian processes are a mainstay for various engineering and
scientific applications, the uncertainty estimates don't satisfy frequentist
guarantees and can be miscalibrated in practice. State-of-the-art approaches
for designing calibrated models rely on inflating the Gaussian process
posterior variance, which yields confidence intervals that are potentially too
coarse. To remedy this, we present a calibration approach that generates
predictive quantiles using a computation inspired by the vanilla Gaussian
process posterior variance but using a different set of hyperparameters chosen
to satisfy an empirical calibration constraint. This results in a calibration
approach that is considerably more flexible than existing approaches, which we
optimize to yield tight predictive quantiles. Our approach is shown to yield a
calibrated model under reasonable assumptions. Furthermore, it outperforms
existing approaches in sharpness when employed for calibrated regression
CoLA: Exploiting Compositional Structure for Automatic and Efficient Numerical Linear Algebra
Many areas of machine learning and science involve large linear algebra
problems, such as eigendecompositions, solving linear systems, computing matrix
exponentials, and trace estimation. The matrices involved often have Kronecker,
convolutional, block diagonal, sum, or product structure. In this paper, we
propose a simple but general framework for large-scale linear algebra problems
in machine learning, named CoLA (Compositional Linear Algebra). By combining a
linear operator abstraction with compositional dispatch rules, CoLA
automatically constructs memory and runtime efficient numerical algorithms.
Moreover, CoLA provides memory efficient automatic differentiation, low
precision computation, and GPU acceleration in both JAX and PyTorch, while also
accommodating new objects, operations, and rules in downstream packages via
multiple dispatch. CoLA can accelerate many algebraic operations, while making
it easy to prototype matrix structures and algorithms, providing an appealing
drop-in tool for virtually any computational effort that requires linear
algebra. We showcase its efficacy across a broad range of applications,
including partial differential equations, Gaussian processes, equivariant model
construction, and unsupervised learning.Comment: Code available at https://github.com/wilson-labs/col