Tight error bounds for rank-1 lattice sampling in spaces of hybrid mixed smoothness

We consider the approximate recovery of multivariate periodic functions from a discrete set of function values taken on a rank-$s$ integration lattice. The main result is the fact that any (non-)linear reconstruction algorithm taking function values on a rank-$s$ lattice of size $M$ has a dimension-independent lower bound of $2^{-(\alpha+1)/2} M^{-\alpha/2}$ when considering the optimal worst-case error with respect to function spaces of (hybrid) mixed smoothness $\alpha>0$ on the $d$-torus. We complement this lower bound with upper bounds that coincide up to logarithmic terms. These upper bounds are obtained by a detailed analysis of a rank-1 lattice sampling strategy, where the rank-1 lattices are constructed by a component-by-component (CBC) method. This improves on earlier results obtained in [25] and [27]. The lattice (group) structure allows for an efficient approximation of the underlying function from its sampled values using a single one-dimensional fast Fourier transform. This is one reason why these algorithms keep attracting significant interest. We compare our results to recent (almost) optimal methods based upon samples on sparse grids

Hyperbolic Cross Approximation

Hyperbolic cross approximation is a special type of multivariate approximation. Recently, driven by applications in engineering, biology, medicine and other areas of science new challenging problems have appeared. The common feature of these problems is high dimensions. We present here a survey on classical methods developed in multivariate approximation theory, which are known to work very well for moderate dimensions and which have potential for applications in really high dimensions. The theory of hyperbolic cross approximation and related theory of functions with mixed smoothness are under detailed study for more than 50 years. It is now well understood that this theory is important both for theoretical study and for practical applications. It is also understood that both theoretical analysis and construction of practical algorithms are very difficult problems. This explains why many fundamental problems in this area are still unsolved. Only a few survey papers and monographs on the topic are published. This and recently discovered deep connections between the hyperbolic cross approximation (and related sparse grids) and other areas of mathematics such as probability, discrepancy, and numerical integration motivated us to write this survey. We try to put emphases on the development of ideas and methods rather than list all the known results in the area. We formulate many problems, which, to our knowledge, are open problems. We also include some very recent results on the topic, which sometimes highlight new interesting directions of research. We hope that this survey will stimulate further active research in this fascinating and challenging area of approximation theory and numerical analysis.Comment: 185 pages, 24 figure

Constructing spatial discretizations for sparse multivariate trigonometric polynomials that allow for a fast discrete Fourier transform

The paper discusses the construction of high dimensional spatial discretizations for arbitrary multivariate trigonometric polynomials, where the frequency support of the trigonometric polynomial is known. We suggest a construction based on the union of several rank-1 lattices as sampling scheme and call such schemes multiple rank-1 lattices. This approach automatically makes available a fast discrete Fourier transform (FFT) on the data. The key objective of the construction of spatial discretizations is the unique reconstruction of the trigonometric polynomial using the sampling values at the sampling nodes. We develop construction methods for multiple rank-1 lattices that allow for this unique reconstruction and for estimates of the number $M$ of distinct sampling nodes within the resulting spatial discretizations. Assuming that the multivariate trigonometric polynomial under consideration is a linear combination of $T$ trigonometric monomials, the oversampling factor $M/T$ is independent of the spatial dimension and, roughly speaking, with high probability only logarithmic in $T$, which is much better than the oversampling factor that is expected when using one single rank-1 lattice. The newly developed approaches for the construction of spatial discretizations are probabilistic methods. The arithmetic complexity of these algorithms depend only linearly on the spatial dimension and, with high probability, only linearly on $T$ up to some logarithmic factors. Furthermore, we analyze the computational complexities of the resulting FFT algorithms in detail and obtain upper bounds in $\mathcal{O}\left(M\log M\right)$, where the constants depend only linearly on the spatial dimension. With high probability, we construct spatial discretizations where $M/T\le C\log T$ holds, which implies that the complexity of the corresponding FFT converts to $\mathcal{O}\left(T\log^2 T\right)$

A Deterministic Algorithm for Constructing Multiple Rank-1 Lattices of Near-Optimal Size

In this paper we present the first known deterministic algorithm for the construction of multiple rank-1 lattices for the approximation of periodic functions of many variables. The algorithm works by converting a potentially large reconstructing single rank-1 lattice for some $d$-dimensional frequency set $I \subset [N]^d$ into a collection of much smaller rank-1 lattices which allow for accurate and efficient reconstruction of trigonometric polynomials with coefficients in $I$ (and, therefore, for the approximation of multivariate periodic functions). The total number of sampling points in the resulting multiple rank-1 lattices is theoretically shown to be less than $\mathcal{O}\left( |I| \log^{ 2 }(N |I|) \right)$ with constants independent of $d$, and by performing one-dimensional fast Fourier transforms on samples of trigonometric polynomials with Fourier support in $I$ at these points, we obtain exact reconstruction of all Fourier coefficients in fewer than $\mathcal{O}\left(d\,|I|\log^4 (N|I|)\right)$ total operations. Additionally, we present a second multiple rank-1 lattice construction algorithm which constructs lattices with even fewer sampling points at the cost of only being able to reconstruct exact trigonometric polynomials rather than having additional theoretical approximation. Both algorithms are tested numerically and surpass the theoretical bounds. Notably, we observe that the oversampling factors #samples$/|I|$ appear to grow only logarithmically in $|I|$ for the first algorithm and appear near-optimally bounded by four in the second algorithm

Approximation of multivariate periodic functions based on sampling along multiple rank-1 lattices

In this work, we consider the approximate reconstruction of high-dimensional periodic functions based on sampling values. As sampling schemes, we utilize so-called reconstructing multiple rank-1 lattices, which combine several preferable properties such as easy constructability, the existence of high-dimensional fast Fourier transform algorithms, high reliability, and low oversampling factors. Especially, we show error estimates for functions from Sobolev Hilbert spaces of generalized mixed smoothness. For instance, when measuring the sampling error in the $L_2$-norm, we show sampling error estimates where the exponent of the main part reaches those of the optimal sampling rate except for an offset of $1/2+\varepsilon$, i.e., the exponent is almost a factor of two better up to the mentioned offset compared to single rank-1 lattice sampling. Various numerical tests in medium and high dimensions demonstrate the high performance and confirm the obtained theoretical results of multiple rank-1 lattice sampling

Index Set Fourier Series Features for Approximating Multi-dimensional Periodic Kernels

Periodicity is often studied in timeseries modelling with autoregressive methods but is less popular in the kernel literature, particularly for higher dimensional problems such as in textures, crystallography, and quantum mechanics. Large datasets often make modelling periodicity untenable for otherwise powerful non-parametric methods like Gaussian Processes (GPs) which typically incur an $\mathcal{O}(N^3)$ computational burden and, consequently, are unable to scale to larger datasets. To this end we introduce a method termed \emph{Index Set Fourier Series Features} to tractably exploit multivariate Fourier series and efficiently decompose periodic kernels on higher-dimensional data into a series of basis functions. We show that our approximation produces significantly less predictive error than alternative approaches such as those based on random Fourier features and achieves better generalisation on regression problems with periodic data

Efficient multivariate approximation on the cube

We combine a periodization strategy for weighted $L_{2}$-integrands with efficient approximation methods in order to approximate multivariate non-periodic functions on the high-dimensional cube $\left[-\frac{1}{2},\frac{1}{2}\right]^{d}$. Our concept allows to determine conditions on the $d$-variate torus-to-cube transformations ${\psi:\left[-\frac{1}{2},\frac{1}{2}\right]^{d}\to\left[-\frac{1}{2},\frac{1}{2}\right]^{d}}$ such that a non-periodic function is transformed into a smooth function in the Sobolev space $\mathcal H^{m}(\mathbb{T}^{d})$ when applying $\psi$. We adapt some $L_{\infty}(\mathbb{T}^{d})$- and $L_{2}(\mathbb{T}^{d})$-approximation error estimates for single rank-$1$ lattice approximation methods and adjust algorithms for the fast evaluation and fast reconstruction of multivariate trigonometric polynomials on the torus in order to apply these methods to the non-periodic setting. We illustrate the theoretical findings by means of numerical tests in up to $d=5$ dimensions.Comment: arXiv admin note: substantial text overlap with arXiv:1805.0910

Multiple Lattice Rules for Multivariate L∞L_\infty Approximation in the Worst-Case Setting

We develop a general framework for estimating the $L_\infty(\mathbb{T}^d)$ error for the approximation of multivariate periodic functions belonging to specific reproducing kernel Hilbert spaces (RHKS) using approximants that are trigonometric polynomials computed from sampling values. The used sampling schemes are suitable sets of rank-1 lattices that can be constructed in an extremely efficient way. Furthermore, the structure of the sampling schemes allows for fast Fourier transform (FFT) algorithms. We present and discuss one FFT algorithm and analyze the worst case $L_\infty$ error for this specific approach. Using this general result we work out very weak requirements on the RHKS that allow for a simple upper bound on the sampling numbers in terms of approximation numbers, where the approximation error is measured in the $L_\infty$ norm. Tremendous advantages of this estimate are its pre-asymptotic validity as well as its simplicity and its specification in full detail. It turns out, that approximation numbers and sampling numbers differ at most slightly. The occurring multiplicative gap does not depend on the spatial dimension $d$ and depends at most logarithmically on the number of used linear information or sampling values, respectively. Moreover, we illustrate the capability of the new sampling method with the aid of specific highly popular source spaces, which yields that the suggested algorithm is nearly optimal from different points of view. For instance, we improve tractability results for the $L_\infty$ approximation for sampling methods and we achieve almost optimal sampling rates for functions of dominating mixed smoothness. Great advantages of the suggested sampling method are the constructive methods for determining sampling sets that guarantee the shown error bounds, the simplicity and extreme efficiency of all necessary algorithms

Transformed rank-1 lattices for high-dimensional approximation

This paper describes an extension of Fourier approximation methods for multivariate functions defined on the torus $\mathbb{T}^d$ to functions in a weighted Hilbert space $L_{2}(\mathbb{R}^d, \omega)$ via a multivariate change of variables $\psi:\left(-\frac{1}{2},\frac{1}{2}\right)^d\to\mathbb{R}^d$. We establish sufficient conditions on $\psi$ and $\omega$ such that the composition of a function in such a weighted Hilbert space with $\psi$ yields a function in the Sobolev space $H_{\mathrm{mix}}^{m}(\mathbb{T}^d)$ of functions on the torus with mixed smoothness of natural order $m \in \mathbb{N}_{0}$. In this approach we adapt algorithms for the evaluation and reconstruction of multivariate trigonometric polynomials on the torus $\mathbb{T}^d$ based on single and multiple reconstructing rank-$1$ lattices. Since in applications it may be difficult to choose a related function space, we make use of dimension incremental construction methods for sparse frequency sets. Various numerical tests confirm obtained theoretical results for the transformed methods

Fast Cross-validation in Harmonic Approximation

Finding a good regularization parameter for Tikhonov regularization problems is a though yet often asked question. One approach is to use leave-one-out cross-validation scores to indicate the goodness of fit. This utilizes only the noisy function values but, on the downside, comes with a high computational cost. In this paper we present a general approach to shift the main computations from the function in question to the node distribution and, making use of FFT and FFT-like algorithms, even reduce this cost tremendously to the cost of the Tikhonov regularization problem itself. We apply this technique in different settings on the torus, the unit interval, and the two-dimensional sphere. Given that the sampling points satisfy a quadrature rule our algorithm computes the cross-validations scores in floating-point precision. In the cases of arbitrarily scattered nodes we propose an approximating algorithm with the same complexity. Numerical experiments indicate the applicability of our algorithms