A combined first and second order variational approach for image reconstruction

In this paper we study a variational problem in the space of functions of bounded Hessian. Our model constitutes a straightforward higher-order extension of the well known ROF functional (total variation minimisation) to which we add a non-smooth second order regulariser. It combines convex functions of the total variation and the total variation of the first derivatives. In what follows, we prove existence and uniqueness of minimisers of the combined model and present the numerical solution of the corresponding discretised problem by employing the split Bregman method. The paper is furnished with applications of our model to image denoising, deblurring as well as image inpainting. The obtained numerical results are compared with results obtained from total generalised variation (TGV), infimal convolution and Euler's elastica, three other state of the art higher-order models. The numerical discussion confirms that the proposed higher-order model competes with models of its kind in avoiding the creation of undesirable artifacts and blocky-like structures in the reconstructed images -- a known disadvantage of the ROF model -- while being simple and efficiently numerically solvable.Comment: 34 pages, 89 figure

Optimal Uniform Convergence Rates for Sieve Nonparametric Instrumental Variables Regression

We study the problem of nonparametric regression when the regressor is endogenous, which is an important nonparametric instrumental variables (NPIV) regression in econometrics and a difficult ill-posed inverse problem with unknown operator in statistics. We first establish a general upper bound on the sup-norm (uniform) convergence rate of a sieve estimator, allowing for endogenous regressors and weakly dependent data. This result leads to the optimal sup-norm convergence rates for spline and wavelet least squares regression estimators under weakly dependent data and heavy-tailed error terms. This upper bound also yields the sup-norm convergence rates for sieve NPIV estimators under i.i.d. data: the rates coincide with the known optimal $L^2$-norm rates for severely ill-posed problems, and are power of $\log(n)$ slower than the optimal $L^2$-norm rates for mildly ill-posed problems. We then establish the minimax risk lower bound in sup-norm loss, which coincides with our upper bounds on sup-norm rates for the spline and wavelet sieve NPIV estimators. This sup-norm rate optimality provides another justification for the wide application of sieve NPIV estimators. Useful results on weakly-dependent random matrices are also provided

A mathematical framework for inverse wave problems in heterogeneous media

This paper provides a theoretical foundation for some common formulations of inverse problems in wave propagation, based on hyperbolic systems of linear integro-differential equations with bounded and measurable coefficients. The coefficients of these time-dependent partial differential equations respresent parametrically the spatially varying mechanical properties of materials. Rocks, manufactured materials, and other wave propagation environments often exhibit spatial heterogeneity in mechanical properties at a wide variety of scales, and coefficient functions representing these properties must mimic this heterogeneity. We show how to choose domains (classes of nonsmooth coefficient functions) and data definitions (traces of weak solutions) so that optimization formulations of inverse wave problems satisfy some of the prerequisites for application of Newton's method and its relatives. These results follow from the properties of a class of abstract first-order evolution systems, of which various physical wave systems appear as concrete instances. Finite speed of propagation for linear waves with bounded, measurable mechanical parameter fields is one of the by-products of this theory

Distal and non-distal NIP theories

We study one way in which stable phenomena can exist in an NIP theory. We start by defining a notion of 'pure instability' that we call 'distality' in which no such phenomenon occurs. O-minimal theories and the p-adics for example are distal. Next, we try to understand what happens when distality fails. Given a type p over a sufficiently saturated model, we extract, in some sense, the stable part of p and define a notion of stable-independence which is implied by non-forking and has bounded weight. As an application, we show that the expansion of a model by traces of externally definable sets from some adequate indiscernible sequence eliminates quantifiers

On the stability of solitary water waves with a point vortex

This paper investigates the stability of traveling wave solutions to the free boundary Euler equations with a submerged point vortex. We prove that sufficiently small-amplitude waves with small enough vortex strength are conditionally orbitally stable. In the process of obtaining this result, we develop a quite general stability/instability theory for bound state solutions of a large class of infinite-dimensional Hamiltonian systems in the presence of symmetry. This is in the spirit of the seminal work of Grillakis, Shatah, and Strauss, but with hypotheses that are relaxed in a number of ways necessary for the point vortex system, and for other hydrodynamical applications more broadly. In particular, we are able to allow the Poisson map to have merely dense range, as opposed to being surjective, and to be state-dependent. As a second application of the general theory, we consider a family of nonlinear dispersive PDEs that includes the generalized KdV and Benjamin--Ono equations. The stability/instability of solitary waves for these systems has been studied extensively, notably by Bona, Souganidis, and Strauss, who used a modification of the GSS method. We provide a new, more direct proof of these results that follows as a straightforward consequence of our abstract theory. At the same time, we extend them to fractional order dispersive equations.Comment: 45 page

Optimal approximation of piecewise smooth functions using deep ReLU neural networks

We study the necessary and sufficient complexity of ReLU neural networks---in terms of depth and number of weights---which is required for approximating classifier functions in $L^2$. As a model class, we consider the set $\mathcal{E}^\beta (\mathbb R^d)$ of possibly discontinuous piecewise $C^\beta$ functions $f : [-1/2, 1/2]^d \to \mathbb R$, where the different smooth regions of $f$ are separated by $C^\beta$ hypersurfaces. For dimension $d \geq 2$, regularity $\beta > 0$, and accuracy $\varepsilon > 0$, we construct artificial neural networks with ReLU activation function that approximate functions from $\mathcal{E}^\beta(\mathbb R^d)$ up to $L^2$ error of $\varepsilon$. The constructed networks have a fixed number of layers, depending only on $d$ and $\beta$, and they have $O(\varepsilon^{-2(d-1)/\beta})$ many nonzero weights, which we prove to be optimal. In addition to the optimality in terms of the number of weights, we show that in order to achieve the optimal approximation rate, one needs ReLU networks of a certain depth. Precisely, for piecewise $C^\beta(\mathbb R^d)$ functions, this minimal depth is given---up to a multiplicative constant---by $\beta/d$. Up to a log factor, our constructed networks match this bound. This partly explains the benefits of depth for ReLU networks by showing that deep networks are necessary to achieve efficient approximation of (piecewise) smooth functions. Finally, we analyze approximation in high-dimensional spaces where the function $f$ to be approximated can be factorized into a smooth dimension reducing feature map $\tau$ and classifier function $g$---defined on a low-dimensional feature space---as $f = g \circ \tau$. We show that in this case the approximation rate depends only on the dimension of the feature space and not the input dimension.Comment: Generalized some estimates to $L^p$ norms for $0<p<\infty