2 research outputs found
Scalable Variational Gaussian Process Regression Networks
Gaussian process regression networks (GPRN) are powerful Bayesian models for
multi-output regression, but their inference is intractable. To address this
issue, existing methods use a fully factorized structure (or a mixture of such
structures) over all the outputs and latent functions for posterior
approximation, which, however, can miss the strong posterior dependencies among
the latent variables and hurt the inference quality. In addition, the updates
of the variational parameters are inefficient and can be prohibitively
expensive for a large number of outputs. To overcome these limitations, we
propose a scalable variational inference algorithm for GPRN, which not only
captures the abundant posterior dependencies but also is much more efficient
for massive outputs. We tensorize the output space and introduce
tensor/matrix-normal variational posteriors to capture the posterior
correlations and to reduce the parameters. We jointly optimize all the
parameters and exploit the inherent Kronecker product structure in the
variational model evidence lower bound to accelerate the computation. We
demonstrate the advantages of our method in several real-world applications
Multi-Fidelity High-Order Gaussian Processes for Physical Simulation
The key task of physical simulation is to solve partial differential
equations (PDEs) on discretized domains, which is known to be costly. In
particular, high-fidelity solutions are much more expensive than low-fidelity
ones. To reduce the cost, we consider novel Gaussian process (GP) models that
leverage simulation examples of different fidelities to predict
high-dimensional PDE solution outputs. Existing GP methods are either not
scalable to high-dimensional outputs or lack effective strategies to integrate
multi-fidelity examples. To address these issues, we propose Multi-Fidelity
High-Order Gaussian Process (MFHoGP) that can capture complex correlations both
between the outputs and between the fidelities to enhance solution estimation,
and scale to large numbers of outputs. Based on a novel nonlinear
coregionalization model, MFHoGP propagates bases throughout fidelities to fuse
information, and places a deep matrix GP prior over the basis weights to
capture the (nonlinear) relationships across the fidelities. To improve
inference efficiency and quality, we use bases decomposition to largely reduce
the model parameters, and layer-wise matrix Gaussian posteriors to capture the
posterior dependency and to simplify the computation. Our stochastic
variational learning algorithm successfully handles millions of outputs without
extra sparse approximations. We show the advantages of our method in several
typical applications