Optimising Spatial and Tonal Data for PDE-based Inpainting

Some recent methods for lossy signal and image compression store only a few selected pixels and fill in the missing structures by inpainting with a partial differential equation (PDE). Suitable operators include the Laplacian, the biharmonic operator, and edge-enhancing anisotropic diffusion (EED). The quality of such approaches depends substantially on the selection of the data that is kept. Optimising this data in the domain and codomain gives rise to challenging mathematical problems that shall be addressed in our work. In the 1D case, we prove results that provide insights into the difficulty of this problem, and we give evidence that a splitting into spatial and tonal (i.e. function value) optimisation does hardly deteriorate the results. In the 2D setting, we present generic algorithms that achieve a high reconstruction quality even if the specified data is very sparse. To optimise the spatial data, we use a probabilistic sparsification, followed by a nonlocal pixel exchange that avoids getting trapped in bad local optima. After this spatial optimisation we perform a tonal optimisation that modifies the function values in order to reduce the global reconstruction error. For homogeneous diffusion inpainting, this comes down to a least squares problem for which we prove that it has a unique solution. We demonstrate that it can be found efficiently with a gradient descent approach that is accelerated with fast explicit diffusion (FED) cycles. Our framework allows to specify the desired density of the inpainting mask a priori. Moreover, is more generic than other data optimisation approaches for the sparse inpainting problem, since it can also be extended to nonlinear inpainting operators such as EED. This is exploited to achieve reconstructions with state-of-the-art quality. We also give an extensive literature survey on PDE-based image compression methods

Non-equispaced B-spline wavelets

This paper has three main contributions. The first is the construction of wavelet transforms from B-spline scaling functions defined on a grid of non-equispaced knots. The new construction extends the equispaced, biorthogonal, compactly supported Cohen-Daubechies-Feauveau wavelets. The new construction is based on the factorisation of wavelet transforms into lifting steps. The second and third contributions are new insights on how to use these and other wavelets in statistical applications. The second contribution is related to the bias of a wavelet representation. It is investigated how the fine scaling coefficients should be derived from the observations. In the context of equispaced data, it is common practice to simply take the observations as fine scale coefficients. It is argued in this paper that this is not acceptable for non-interpolating wavelets on non-equidistant data. Finally, the third contribution is the study of the variance in a non-orthogonal wavelet transform in a new framework, replacing the numerical condition as a measure for non-orthogonality. By controlling the variances of the reconstruction from the wavelet coefficients, the new framework allows us to design wavelet transforms on irregular point sets with a focus on their use for smoothing or other applications in statistics.Comment: 42 pages, 2 figure

Image interpolation using Shearlet based iterative refinement

This paper proposes an image interpolation algorithm exploiting sparse representation for natural images. It involves three main steps: (a) obtaining an initial estimate of the high resolution image using linear methods like FIR filtering, (b) promoting sparsity in a selected dictionary through iterative thresholding, and (c) extracting high frequency information from the approximation to refine the initial estimate. For the sparse modeling, a shearlet dictionary is chosen to yield a multiscale directional representation. The proposed algorithm is compared to several state-of-the-art methods to assess its objective as well as subjective performance. Compared to the cubic spline interpolation method, an average PSNR gain of around 0.8 dB is observed over a dataset of 200 images

Quadtree Structured Approximation Algorithms

The success of many image restoration algorithms is often due to their ability to sparsely describe the original signal. Many sparse promoting transforms exist, including wavelets, the so called ‘lets’ family of transforms and more recent non-local learned transforms. The first part of this thesis reviews sparse approximation theory, particularly in relation to 2-D piecewise polynomial signals. We also show the connection between this theory and current state of the art algorithms that cover the following image restoration and enhancement applications: denoising, deconvolution, interpolation and multi-view super resolution. In [63], Shukla et al. proposed a compression algorithm, based on a sparse quadtree decomposition model, which could optimally represent piecewise polynomial images. In the second part of this thesis we adapt this model to image restoration by changing the rate-distortion penalty to a description-length penalty. Moreover, one of the major drawbacks of this type of approximation is the computational complexity required to find a suitable subspace for each node of the quadtree. We address this issue by searching for a suitable subspace much more efficiently using the mathematics of updating matrix factorisations. Novel algorithms are developed to tackle the four problems previously mentioned. Simulation results indicate that we beat state of the art results when the original signal is in the model (e.g. depth images) and are competitive for natural images when the degradation is high.Open Acces

Building Adaptive Basis Functions with a Continuous Self-Organizing Map

This paper introduces CSOM, a continuous version of the Self-Organizing Map (SOM). The CSOM network generates maps similar to those created with the original SOM algorithm but, due to the continuous nature of the mapping, CSOM outperforms the SOM on function approximation tasks. CSOM integrates self-organization and smooth prediction into a single process. This is a departure from previous work that required two training phases, one to self-organize a map using the SOM algorithm, and another to learn a smooth approximation of a function. System performance is illustrated with three examples.Office of Naval Research (N00014-95-10409, N00014-95-0657

Nonlinear thresholding of multiresolution decompositions adapted to the presence of discontinuities

International audienceA new nonlinear representation of multiresolution decompositions and new thresholding adapted to the presence of discontinuities are presented and analyzed. They are based on a nonlinear modification of the multiresolution details coming from an initial (linear or nonlinear) scheme and on a data dependent thresholding. Stability results are derived. Numerical advantages are demonstrated on various numerical experiments

Low-latency compression of mocap data using learned spatial decorrelation transform

Due to the growing needs of human motion capture (mocap) in movie, video games, sports, etc., it is highly desired to compress mocap data for efficient storage and transmission. This paper presents two efficient frameworks for compressing human mocap data with low latency. The first framework processes the data in a frame-by-frame manner so that it is ideal for mocap data streaming and time critical applications. The second one is clip-based and provides a flexible tradeoff between latency and compression performance. Since mocap data exhibits some unique spatial characteristics, we propose a very effective transform, namely learned orthogonal transform (LOT), for reducing the spatial redundancy. The LOT problem is formulated as minimizing square error regularized by orthogonality and sparsity and solved via alternating iteration. We also adopt a predictive coding and temporal DCT for temporal decorrelation in the frame- and clip-based frameworks, respectively. Experimental results show that the proposed frameworks can produce higher compression performance at lower computational cost and latency than the state-of-the-art methods.Comment: 15 pages, 9 figure

Exploratory Analysis of Functional Data via Clustering and Optimal Segmentation

We propose in this paper an exploratory analysis algorithm for functional data. The method partitions a set of functions into $K$ clusters and represents each cluster by a simple prototype (e.g., piecewise constant). The total number of segments in the prototypes, $P$, is chosen by the user and optimally distributed among the clusters via two dynamic programming algorithms. The practical relevance of the method is shown on two real world datasets