Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spaces

In this paper, we study regression problems over a separable Hilbert space with the square loss, covering non-parametric regression over a reproducing kernel Hilbert space. We investigate a class of spectral-regularized algorithms, including ridge regression, principal component analysis, and gradient methods. We prove optimal, high-probability convergence results in terms of variants of norms for the studied algorithms, considering a capacity assumption on the hypothesis space and a general source condition on the target function. Consequently, we obtain almost sure convergence results with optimal rates. Our results improve and generalize previous results, filling a theoretical gap for the non-attainable cases

Stability Estimates and Structural Spectral Properties of Saddle Point Problems

For a general class of saddle point problems sharp estimates for Babu\v{s}ka's inf-sup stability constants are derived in terms of the constants in Brezzi's theory. In the finite-dimensional Hermitian case more detailed spectral properties of preconditioned saddle point matrices are presented, which are helpful for the convergence analysis of common Krylov subspace methods. The theoretical results are applied to two model problems from optimal control with time-periodic state equations. Numerical experiments with the preconditioned minimal residual method are reported

Rates of convergence for empirical spectral measures: a soft approach

Understanding the limiting behavior of eigenvalues of random matrices is the central problem of random matrix theory. Classical limit results are known for many models, and there has been significant recent progress in obtaining more quantitative, non-asymptotic results. In this paper, we describe a systematic approach to bounding rates of convergence and proving tail inequalities for the empirical spectral measures of a wide variety of random matrix ensembles. We illustrate the approach by proving asymptotically almost sure rates of convergence of the empirical spectral measure in the following ensembles: Wigner matrices, Wishart matrices, Haar-distributed matrices from the compact classical groups, powers of Haar matrices, randomized sums and random compressions of Hermitian matrices, a random matrix model for the Hamiltonians of quantum spin glasses, and finally the complex Ginibre ensemble. Many of the results appeared previously and are being collected and described here as illustrations of the general method; however, some details (particularly in the Wigner and Wishart cases) are new. Our approach makes use of techniques from probability in Banach spaces, in particular concentration of measure and bounds for suprema of stochastic processes, in combination with more classical tools from matrix analysis, approximation theory, and Fourier analysis. It is highly flexible, as evidenced by the broad list of examples. It is moreover based largely on "soft" methods, and involves little hard analysis

Domain decomposition methods for systems of conservation laws: Spectral collocation approximations

Hyperbolic systems of conversation laws are considered which are discretized in space by spectral collocation methods and advanced in time by finite difference schemes. At any time-level a domain deposition method based on an iteration by subdomain procedure was introduced yielding at each step a sequence of independent subproblems (one for each subdomain) that can be solved simultaneously. The method is set for a general nonlinear problem in several space variables. The convergence analysis, however, is carried out only for a linear one-dimensional system with continuous solutions. A precise form of the error reduction factor at each iteration is derived. Although the method is applied here to the case of spectral collocation approximation only, the idea is fairly general and can be used in a different context as well. For instance, its application to space discretization by finite differences is straight forward

Can linear collocation ever beat quadratic?

Computational approaches are becoming increasingly important in neuroscience, where complex, nonlinear systems modelling neural activity across multiple spatial and temporal scales are the norm. This paper considers collocation techniques for solving neural field models, which typically take the form of a partial integro-dfferential equation. In particular, we investigate and compare the convergence properties of linear and quadratic collocation on both regular grids and more general meshes not fixed to the regular Cartesian grid points. For regular grids we perform a comparative analysis against more standard techniques, in which the convolution integral is computed either by using Fourier based methods or via the trapezoidal rule. Perhaps surprisingly, we find that on regular, periodic meshes, linear collocation displays better convergence properties than quadratic collocation, and is in fact comparable with the spectral convergence displayed by both the Fourier based and trapezoidal techniques. However, for more general meshes we obtain superior convergence of the convolution integral using higher order methods, as expected

Quantile spectral processes: Asymptotic analysis and inference

Quantile- and copula-related spectral concepts recently have been considered by various authors. Those spectra, in their most general form, provide a full characterization of the copulas associated with the pairs $(X_t,X_{t-k})$ in a process $(X_t)_{t\in\mathbb{Z}}$, and account for important dynamic features, such as changes in the conditional shape (skewness, kurtosis), time-irreversibility, or dependence in the extremes that their traditional counterparts cannot capture. Despite various proposals for estimation strategies, only quite incomplete asymptotic distributional results are available so far for the proposed estimators, which constitutes an important obstacle for their practical application. In this paper, we provide a detailed asymptotic analysis of a class of smoothed rank-based cross-periodograms associated with the copula spectral density kernels introduced in Dette et al. [Bernoulli 21 (2015) 781-831]. We show that, for a very general class of (possibly nonlinear) processes, properly scaled and centered smoothed versions of those cross-periodograms, indexed by couples of quantile levels, converge weakly, as stochastic processes, to Gaussian processes. A first application of those results is the construction of asymptotic confidence intervals for copula spectral density kernels. The same convergence results also provide asymptotic distributions (under serially dependent observations) for a new class of rank-based spectral methods involving the Fourier transforms of rank-based serial statistics such as the Spearman, Blomqvist or Gini autocovariance coefficients.Comment: Published at http://dx.doi.org/10.3150/15-BEJ711 in the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm

The Vanishing Moment Method for Fully Nonlinear Second Order Partial Differential Equations: Formulation, Theory, and Numerical Analysis

The vanishing moment method was introduced by the authors in [37] as a reliable methodology for computing viscosity solutions of fully nonlinear second order partial differential equations (PDEs), in particular, using Galerkin-type numerical methods such as finite element methods, spectral methods, and discontinuous Galerkin methods, a task which has not been practicable in the past. The crux of the vanishing moment method is the simple idea of approximating a fully nonlinear second order PDE by a family (parametrized by a small parameter \vepsi) of quasilinear higher order (in particular, fourth order) PDEs. The primary objectives of this book are to present a detailed convergent analysis for the method in the radial symmetric case and to carry out a comprehensive finite element numerical analysis for the vanishing moment equations (i.e., the regularized fourth order PDEs). Abstract methodological and convergence analysis frameworks of conforming finite element methods and mixed finite element methods are first developed for fully nonlinear second order PDEs in general settings. The abstract frameworks are then applied to three prototypical nonlinear equations, namely, the Monge-Amp\`ere equation, the equation of prescribed Gauss curvature, and the infinity-Laplacian equation. Numerical experiments are also presented for each problem to validate the theoretical error estimate results and to gauge the efficiency of the proposed numerical methods and the vanishing moment methodology.Comment: 141 pages, 16 figure

Explicit exponential Runge-Kutta methods for semilinear integro-differential equations

The aim of this paper is to construct and analyze explicit exponential Runge-Kutta methods for the temporal discretization of linear and semilinear integro-differential equations. By expanding the errors of the numerical method in terms of the solution, we derive order conditions that form the basis of our error bounds for integro-differential equations. The order conditions are further used for constructing numerical methods. The convergence analysis is performed in a Hilbert space setting, where the smoothing effect of the resolvent family is heavily used. For the linear case, we derive the order conditions for general order $p$ and prove convergence of order $p$, whenever these conditions are satisfied. In the semilinear case, we consider in addition spatial discretization by a spectral Galerkin method, and we require locally Lipschitz continuous nonlinearities. We derive the order conditions for orders one and two, construct methods satisfying these conditions and prove their convergence. Finally, some numerical experiments illustrating our theoretical results are given