20,107 research outputs found
Recovering edges in ill-posed inverse problems: optimality of curvelet frames
We consider a model problem of recovering a function from noisy Radon data. The function to be recovered is assumed smooth apart from a discontinuity along a curve, that is, an edge. We use the continuum white-noise model, with noise level .
Traditional linear methods for solving such inverse problems behave poorly in the presence of edges. Qualitatively, the reconstructions are blurred near the edges; quantitatively, they give in our model mean squared errors (MSEs) that tend to zero with noise level only as as . A recent innovation--nonlinear shrinkage in the wavelet domain--visually improves edge sharpness and improves MSE convergence to . However, as we show here, this rate is not optimal.
In fact, essentially optimal performance is obtained by deploying the recently-introduced tight frames of curvelets in this setting. Curvelets are smooth, highly anisotropic elements ideally suited for detecting and synthesizing curved edges. To deploy them in the Radon setting, we construct a curvelet-based biorthogonal decomposition of the Radon operator and build "curvelet shrinkage" estimators based on thresholding of the noisy curvelet coefficients. In effect, the estimator detects edges at certain locations and orientations in the Radon domain and automatically synthesizes edges at corresponding locations and directions in the original domain.
We prove that the curvelet shrinkage can be tuned so that the estimator will attain, within logarithmic factors, the MSE as noise level . This rate of convergence holds uniformly over a class of functions which are except for discontinuities along curves, and (except for log terms) is the minimax rate for that class. Our approach is an instance of a general strategy which should apply in other inverse problems; we sketch a deconvolution example
Group Invariance, Stability to Deformations, and Complexity of Deep Convolutional Representations
The success of deep convolutional architectures is often attributed in part
to their ability to learn multiscale and invariant representations of natural
signals. However, a precise study of these properties and how they affect
learning guarantees is still missing. In this paper, we consider deep
convolutional representations of signals; we study their invariance to
translations and to more general groups of transformations, their stability to
the action of diffeomorphisms, and their ability to preserve signal
information. This analysis is carried by introducing a multilayer kernel based
on convolutional kernel networks and by studying the geometry induced by the
kernel mapping. We then characterize the corresponding reproducing kernel
Hilbert space (RKHS), showing that it contains a large class of convolutional
neural networks with homogeneous activation functions. This analysis allows us
to separate data representation from learning, and to provide a canonical
measure of model complexity, the RKHS norm, which controls both stability and
generalization of any learned model. In addition to models in the constructed
RKHS, our stability analysis also applies to convolutional networks with
generic activations such as rectified linear units, and we discuss its
relationship with recent generalization bounds based on spectral norms
Stable image reconstruction using total variation minimization
This article presents near-optimal guarantees for accurate and robust image
recovery from under-sampled noisy measurements using total variation
minimization. In particular, we show that from O(slog(N)) nonadaptive linear
measurements, an image can be reconstructed to within the best s-term
approximation of its gradient up to a logarithmic factor, and this factor can
be removed by taking slightly more measurements. Along the way, we prove a
strengthened Sobolev inequality for functions lying in the null space of
suitably incoherent matrices.Comment: 25 page
Empirical Bayes selection of wavelet thresholds
This paper explores a class of empirical Bayes methods for level-dependent
threshold selection in wavelet shrinkage. The prior considered for each wavelet
coefficient is a mixture of an atom of probability at zero and a heavy-tailed
density. The mixing weight, or sparsity parameter, for each level of the
transform is chosen by marginal maximum likelihood. If estimation is carried
out using the posterior median, this is a random thresholding procedure; the
estimation can also be carried out using other thresholding rules with the same
threshold. Details of the calculations needed for implementing the procedure
are included. In practice, the estimates are quick to compute and there is
software available. Simulations on the standard model functions show excellent
performance, and applications to data drawn from various fields of application
are used to explore the practical performance of the approach. By using a
general result on the risk of the corresponding marginal maximum likelihood
approach for a single sequence, overall bounds on the risk of the method are
found subject to membership of the unknown function in one of a wide range of
Besov classes, covering also the case of f of bounded variation. The rates
obtained are optimal for any value of the parameter p in (0,\infty],
simultaneously for a wide range of loss functions, each dominating the L_q norm
of the \sigmath derivative, with \sigma\ge0 and 0<q\le2.Comment: Published at http://dx.doi.org/10.1214/009053605000000345 in the
Annals of Statistics (http://www.imstat.org/aos/) by the Institute of
Mathematical Statistics (http://www.imstat.org
- …