315 research outputs found
A Comprehensive Survey on Functional Approximation
The theory of functional approximation has numerous applications in sciences and industry. This thesis focuses on the possible approaches to approximate a continuous function on a compact subset of R2 using a variety of constructions. The results are presented from the following four general topics: polynomials, Fourier series, wavelets, and neural networks. Approximation with polynomials on subsets of R leads to the discussion of the Stone-Weierstrass theorem. Convergence of Fourier series is characterized on the unit circle. Wavelets are introduced following the Fourier transform, and their construction as well as ability to approximate functions in L2(R) is discussed. At the end, the universal approximation theorem for artificial neural networks is presented, and the function representation and approximation with single- and multilayer neural networks on R2 is constructed
Compressive Fourier collocation methods for high-dimensional diffusion equations with periodic boundary conditions
High-dimensional Partial Differential Equations (PDEs) are a popular
mathematical modelling tool, with applications ranging from finance to
computational chemistry. However, standard numerical techniques for solving
these PDEs are typically affected by the curse of dimensionality. In this work,
we tackle this challenge while focusing on stationary diffusion equations
defined over a high-dimensional domain with periodic boundary conditions.
Inspired by recent progress in sparse function approximation in high
dimensions, we propose a new method called compressive Fourier collocation.
Combining ideas from compressive sensing and spectral collocation, our method
replaces the use of structured collocation grids with Monte Carlo sampling and
employs sparse recovery techniques, such as orthogonal matching pursuit and
minimization, to approximate the Fourier coefficients of the PDE
solution. We conduct a rigorous theoretical analysis showing that the
approximation error of the proposed method is comparable with the best -term
approximation (with respect to the Fourier basis) to the solution. Using the
recently introduced framework of random sampling in bounded Riesz systems, our
analysis shows that the compressive Fourier collocation method mitigates the
curse of dimensionality with respect to the number of collocation points under
sufficient conditions on the regularity of the diffusion coefficient. We also
present numerical experiments that illustrate the accuracy and stability of the
method for the approximation of sparse and compressible solutions.Comment: 33 pages, 9 figure
Applications of classical approximation theory to periodic basis function networks and computational harmonic analysis
In this paper, we describe a novel approach to classical approximation theory of periodic univariate and multivariate functions by trigonometric polynomials.
While classical wisdom holds that such approximation is too sensitive to the lack of smoothness of the target functions at isolated points, our constructions show how to overcome this problem. We describe applications to approximation by periodic basis function networks, and indicate further research in the direction of Jacobi expansion and approximation on the Euclidean sphere. While the paper is mainly intended to be a survey of our recent research in these directions, several results are proved for the
first time here
Optimal approximation of -functions using shallow complex-valued neural networks
We prove a quantitative result for the approximation of functions of
regularity (in the sense of real variables) defined on the complex cube
using shallow
complex-valued neural networks. Precisely, we consider neural networks with a
single hidden layer and neurons, i.e., networks of the form and show that one
can approximate every function in
using a function of that form with error of the order as , provided that the activation function is smooth but not polyharmonic on some non-empty open set.
Furthermore, we show that the selection of the weights and is continuous with respect to and
prove that the derived rate of approximation is optimal under this continuity
assumption. We also discuss the optimality of the result for a possibly
discontinuous choice of the weights
Error estimates for DeepOnets: A deep learning framework in infinite dimensions
DeepOnets have recently been proposed as a framework for learning nonlinear
operators mapping between infinite dimensional Banach spaces. We analyze
DeepOnets and prove estimates on the resulting approximation and generalization
errors. In particular, we extend the universal approximation property of
DeepOnets to include measurable mappings in non-compact spaces. By a
decomposition of the error into encoding, approximation and reconstruction
errors, we prove both lower and upper bounds on the total error, relating it to
the spectral decay properties of the covariance operators, associated with the
underlying measures. We derive almost optimal error bounds with very general
affine reconstructors and with random sensor locations as well as bounds on the
generalization error, using covering number arguments. We illustrate our
general framework with four prototypical examples of nonlinear operators,
namely those arising in a nonlinear forced ODE, an elliptic PDE with variable
coefficients and nonlinear parabolic and hyperbolic PDEs. In all these
examples, we prove that DeepOnets break the curse of dimensionality, thus
demonstrating the efficient approximation of infinite-dimensional operators
with this machine learning framework
Approximation Theory and Related Applications
In recent years, we have seen a growing interest in various aspects of approximation theory. This happened due to the increasing complexity of mathematical models that require computer calculations and the development of the theoretical foundations of the approximation theory. Approximation theory has broad and important applications in many areas of mathematics, including functional analysis, differential equations, dynamical systems theory, mathematical physics, control theory, probability theory and mathematical statistics, and others. Approximation theory is also of great practical importance, as approximate methods and estimation of approximation errors are used in physics, economics, chemistry, signal theory, neural networks and many other areas. This book presents the works published in the Special Issue "Approximation Theory and Related Applications". The research of the world’s leading scientists presented in this book reflect new trends in approximation theory and related topics
Recommended from our members
Representational Capabilities of Feed-forward and Sequential Neural Architectures
Despite the widespread empirical success of deep neural networks over the past decade, a comprehensive understanding of their mathematical properties remains elusive, which limits the abilities of practitioners to train neural networks in a principled manner. This dissertation provides a representational characterization of a variety of neural network architectures, including fully-connected feed-forward networks and sequential models like transformers.
The representational capabilities of neural networks are most famously characterized by the universal approximation theorem, which states that sufficiently large neural networks can closely approximate any well-behaved target function. However, the universal approximation theorem applies exclusively to two-layer neural networks of unbounded size and fails to capture the comparative strengths and weaknesses of different architectures.
The thesis addresses these limitations by quantifying the representational consequences of random features, weight regularization, and model depth on feed-forward architectures. It further investigates and contrasts the expressive powers of transformers and other sequential neural architectures. Taken together, these results apply a wide range of theoretical tools—including approximation theory, discrete dynamical systems, and communication complexity—to prove rigorous separations between different neural architectures and scaling regimes
Neural and spectral operator surrogates: unified construction and expression rate bounds
Approximation rates are analyzed for deep surrogates of maps between
infinite-dimensional function spaces, arising e.g. as data-to-solution maps of
linear and nonlinear partial differential equations. Specifically, we study
approximation rates for Deep Neural Operator and Generalized Polynomial Chaos
(gpc) Operator surrogates for nonlinear, holomorphic maps between
infinite-dimensional, separable Hilbert spaces. Operator in- and outputs from
function spaces are assumed to be parametrized by stable, affine representation
systems. Admissible representation systems comprise orthonormal bases, Riesz
bases or suitable tight frames of the spaces under consideration. Algebraic
expression rate bounds are established for both, deep neural and spectral
operator surrogates acting in scales of separable Hilbert spaces containing
domain and range of the map to be expressed, with finite Sobolev or Besov
regularity. We illustrate the abstract concepts by expression rate bounds for
the coefficient-to-solution map for a linear elliptic PDE on the torus
Localized linear polynomial operators and quadrature formulas on the sphere
The purpose of this paper is to construct universal, auto--adaptive,
localized, linear, polynomial (-valued) operators based on scattered data on
the (hyper--)sphere \SS^q (). The approximation and localization
properties of our operators are studied theoretically in deterministic as well
as probabilistic settings. Numerical experiments are presented to demonstrate
their superiority over traditional least squares and discrete Fourier
projection polynomial approximations. An essential ingredient in our
construction is the construction of quadrature formulas based on scattered
data, exact for integrating spherical polynomials of (moderately) high degree.
Our formulas are based on scattered sites; i.e., in contrast to such well known
formulas as Driscoll--Healy formulas, we need not choose the location of the
sites in any particular manner. While the previous attempts to construct such
formulas have yielded formulas exact for spherical polynomials of degree at
most 18, we are able to construct formulas exact for spherical polynomials of
degree 178.Comment: 24 pages 2 figures, accepted for publication in SIAM J. Numer. Ana
- …