Error Bounds for Learning with Vector-Valued Random Features

This paper provides a comprehensive error analysis of learning with vector-valued random features (RF). The theory is developed for RF ridge regression in a fully general infinite-dimensional input-output setting, but nonetheless applies to and improves existing finite-dimensional analyses. In contrast to comparable work in the literature, the approach proposed here relies on a direct analysis of the underlying risk functional and completely avoids the explicit RF ridge regression solution formula in terms of random matrices. This removes the need for concentration results in random matrix theory or their generalizations to random operators. The main results established in this paper include strong consistency of vector-valued RF estimators under model misspecification and minimax optimal convergence rates in the well-specified setting. The parameter complexity (number of random features) and sample complexity (number of labeled data) required to achieve such rates are comparable with Monte Carlo intuition and free from logarithmic factors.Comment: 25 pages, 1 tabl

The Random Feature Model for Input-Output Maps between Banach Spaces

Well known to the machine learning community, the random feature model, originally introduced by Rahimi and Recht in 2008, is a parametric approximation to kernel interpolation or regression methods. It is typically used to approximate functions mapping a finite-dimensional input space to the real line. In this paper, we instead propose a methodology for use of the random feature model as a data-driven surrogate for operators that map an input Banach space to an output Banach space. Although the methodology is quite general, we consider operators defined by partial differential equations (PDEs); here, the inputs and outputs are themselves functions, with the input parameters being functions required to specify the problem, such as initial data or coefficients, and the outputs being solutions of the problem. Upon discretization, the model inherits several desirable attributes from this infinite-dimensional, function space viewpoint, including mesh-invariant approximation error with respect to the true PDE solution map and the capability to be trained at one mesh resolution and then deployed at different mesh resolutions. We view the random feature model as a non-intrusive data-driven emulator, provide a mathematical framework for its interpretation, and demonstrate its ability to efficiently and accurately approximate the nonlinear parameter-to-solution maps of two prototypical PDEs arising in physical science and engineering applications: viscous Burgers' equation and a variable coefficient elliptic equation

The Random Feature Model for Input-Output Maps between Banach Spaces

Well known to the machine learning community, the random feature model is a parametric approximation to kernel interpolation or regression methods. It is typically used to approximate functions mapping a finite-dimensional input space to the real line. In this paper, we instead propose a methodology for use of the random feature model as a data-driven surrogate for operators that map an input Banach space to an output Banach space. Although the methodology is quite general, we consider operators defined by partial differential equations (PDEs); here, the inputs and outputs are themselves functions, with the input parameters being functions required to specify the problem, such as initial data or coefficients, and the outputs being solutions of the problem. Upon discretization, the model inherits several desirable attributes from this infinite-dimensional viewpoint, including mesh-invariant approximation error with respect to the true PDE solution map and the capability to be trained at one mesh resolution and then deployed at different mesh resolutions. We view the random feature model as a non-intrusive data-driven emulator, provide a mathematical framework for its interpretation, and demonstrate its ability to efficiently and accurately approximate the nonlinear parameter-to-solution maps of two prototypical PDEs arising in physical science and engineering applications: viscous Burgers' equation and a variable coefficient elliptic equation.Comment: To appear in SIAM Journal on Scientific Computing; 32 pages, 9 figure

On partial differential equations modified with fractional operators and integral transformations: Nonlocal and nonlinear PDF models

We explore nonlocal and pseudo-differential operators in the setting of partial differential equations (PDE). The two primary PDE in this work are the generalized heat equation and the nonlocal Burgers' type advection-diffusion equation. These nonlocal and nonlinear models arise in complex physical systems including material phase transition and fluid flow

Reduced order framework for optimal control of nonlinear partial differential equations: ROM-based optimal flow control

A variety of partial differential equations (PDE) can govern the spatial and time evolution of fluid flows; however, direct numerical simulation (DNS) of the Euler or Navier-Stokes equation or other traditional computational fluid dynamics (CFD) models can be computationally expensive and intractable. An alternative is to use model order reduction techniques, e.g., reduced order models (ROM) via proper orthogonal decomposition (POD) or dynamic mode decomposition (DMD), to reduce the dimensionality of these nonlinear dynamical systems while still retaining the essential physics. The objective of this work is to design a reduced order numerical framework for effective simulation and control of complex flow phenomena. To build our computational method with this philosophy, we first simulate the 1D Burgers' equation ut + uux ? ?uxx = f(x, t), a well-known PDE modeling nonlinear advection-diffusion flow physics and shock waves, as a full order high resolution benchmark. We then apply canonical reduction approaches incorporating Fourier and POD modes with a Galerkin projection to approximate the solution to the posed initial boundary value problem. The control objective is simple: we seek the optimal (pointwise) input into the system that forces the spatial evolution of the PDE solution to converge to a preselected target state uT(x) at some final time T > 0. To implement an iterative control loop, we parametrize the unknown control function as a truncated Fourier series defined via a set of finite parameters. The performance of the POD ROM is compared to that of the Fourier ROM and full order model for six numerical experiments

Convergence Rates for Learning Linear Operators from Noisy Data

This paper studies the learning of linear operators between infinite-dimensional Hilbert spaces. The training data comprises pairs of random input vectors in a Hilbert space and their noisy images under an unknown self-adjoint linear operator. Assuming that the operator is diagonalizable in a known basis, this work solves the equivalent inverse problem of estimating the operator's eigenvalues given the data. Adopting a Bayesian approach, the theoretical analysis establishes posterior contraction rates in the infinite data limit with Gaussian priors that are not directly linked to the forward map of the inverse problem. The main results also include learning-theoretic generalization error guarantees for a wide range of distribution shifts. These convergence rates quantify the effects of data smoothness and true eigenvalue decay or growth, for compact or unbounded operators, respectively, on sample complexity. Numerical evidence supports the theory in diagonal and non-diagonal settings.Comment: To appear in SIAM/ASA Journal on Uncertainty Quantification (JUQ); 34 pages, 5 figures, 2 table