Task-Driven Dictionary Learning

Modeling data with linear combinations of a few elements from a learned dictionary has been the focus of much recent research in machine learning, neuroscience and signal processing. For signals such as natural images that admit such sparse representations, it is now well established that these models are well suited to restoration tasks. In this context, learning the dictionary amounts to solving a large-scale matrix factorization problem, which can be done efficiently with classical optimization tools. The same approach has also been used for learning features from data for other purposes, e.g., image classification, but tuning the dictionary in a supervised way for these tasks has proven to be more difficult. In this paper, we present a general formulation for supervised dictionary learning adapted to a wide variety of tasks, and present an efficient algorithm for solving the corresponding optimization problem. Experiments on handwritten digit classification, digital art identification, nonlinear inverse image problems, and compressed sensing demonstrate that our approach is effective in large-scale settings, and is well suited to supervised and semi-supervised classification, as well as regression tasks for data that admit sparse representations.Comment: final draft post-refereein

ARMAS: Active Reconstruction of Missing Audio Segments

Digital audio signal reconstruction of a lost or corrupt segment using deep learning algorithms has been explored intensively in recent years. Nevertheless, prior traditional methods with linear interpolation, phase coding and tone insertion techniques are still in vogue. However, we found no research work on reconstructing audio signals with the fusion of dithering, steganography, and machine learning regressors. Therefore, this paper proposes the combination of steganography, halftoning (dithering), and state-of-the-art shallow (RF- Random Forest regression) and deep learning (LSTM- Long Short-Term Memory) methods. The results (including comparing the SPAIN, Autoregressive, deep learning-based, graph-based, and other methods) are evaluated with three different metrics. The observations from the results show that the proposed solution is effective and can enhance the reconstruction of audio signals performed by the side information (e.g., Latent representation and learning for audio inpainting) steganography provides. Moreover, this paper proposes a novel framework for reconstruction from heavily compressed embedded audio data using halftoning (i.e., dithering) and machine learning, which we termed the HCR (halftone-based compression and reconstruction). This work may trigger interest in optimising this approach and/or transferring it to different domains (i.e., image reconstruction). Compared to existing methods, we show improvement in the inpainting performance in terms of signal-to-noise (SNR), the objective difference grade (ODG) and the Hansen's audio quality metric.Comment: 9 pages, 2 Tables, 8 Figure

Perceptual error optimization for Monte Carlo rendering

Realistic image synthesis involves computing high-dimensional light transport integrals which in practice are numerically estimated using Monte Carlo integration. The error of this estimation manifests itself in the image as visually displeasing aliasing or noise. To ameliorate this, we develop a theoretical framework for optimizing screen-space error distribution. Our model is flexible and works for arbitrary target error power spectra. We focus on perceptual error optimization by leveraging models of the human visual system's (HVS) point spread function (PSF) from halftoning literature. This results in a specific optimization problem whose solution distributes the error as visually pleasing blue noise in image space. We develop a set of algorithms that provide a trade-off between quality and speed, showing substantial improvements over prior state of the art. We perform evaluations using both quantitative and perceptual error metrics to support our analysis, and provide extensive supplemental material to help evaluate the perceptual improvements achieved by our methods

Optimising Spatial and Tonal Data for PDE-based Inpainting

Some recent methods for lossy signal and image compression store only a few selected pixels and fill in the missing structures by inpainting with a partial differential equation (PDE). Suitable operators include the Laplacian, the biharmonic operator, and edge-enhancing anisotropic diffusion (EED). The quality of such approaches depends substantially on the selection of the data that is kept. Optimising this data in the domain and codomain gives rise to challenging mathematical problems that shall be addressed in our work. In the 1D case, we prove results that provide insights into the difficulty of this problem, and we give evidence that a splitting into spatial and tonal (i.e. function value) optimisation does hardly deteriorate the results. In the 2D setting, we present generic algorithms that achieve a high reconstruction quality even if the specified data is very sparse. To optimise the spatial data, we use a probabilistic sparsification, followed by a nonlocal pixel exchange that avoids getting trapped in bad local optima. After this spatial optimisation we perform a tonal optimisation that modifies the function values in order to reduce the global reconstruction error. For homogeneous diffusion inpainting, this comes down to a least squares problem for which we prove that it has a unique solution. We demonstrate that it can be found efficiently with a gradient descent approach that is accelerated with fast explicit diffusion (FED) cycles. Our framework allows to specify the desired density of the inpainting mask a priori. Moreover, is more generic than other data optimisation approaches for the sparse inpainting problem, since it can also be extended to nonlinear inpainting operators such as EED. This is exploited to achieve reconstructions with state-of-the-art quality. We also give an extensive literature survey on PDE-based image compression methods

Geometry-Aware Scattering Compensation for 3D Printing

Commercially available full-color 3D printing allows for detailed control of material deposition in a volume, but an exact reproduction of a target surface appearance is hampered by the strong subsurface scattering that causes nontrivial volumetric cross-talk at the print surface. Previous work showed how an iterative optimization scheme based on accumulating absorptive materials at the surface can be used to find a volumetric distribution of print materials that closely approximates a given target appearance. // In this work, we first revisit the assumption that pushing the absorptive materials to the surface results in minimal volumetric cross-talk. We design a full-fledged optimization on a small domain for this task and confirm this previously reported heuristic. Then, we extend the above approach that is critically limited to color reproduction on planar surfaces, to arbitrary 3D shapes. Our proposed method enables high-fidelity color texture reproduction on 3D prints by effectively compensating for internal light scattering within arbitrarily shaped objects. In addition, we propose a content-aware gamut mapping that significantly improves color reproduction for the pathological case of thin geometric features. Using a wide range of sample objects with complex textures and geometries, we demonstrate color reproduction whose fidelity is superior to state-of-the-art drivers for color 3D printers

Modeling and Halftoning for Multichannel Printers: A Spectral Approach

Printing has been has been the major communication medium for many centuries. In the last twenty years, multichannel printing has brought new opportunities and challenges. Beside of extended colour gamut of the multichannel printer, the opportunity was presented to use a multichannel printer for ‘spectral printing’. The aim of spectral printing is typically the same as for colour printing; that is, to match input signal with printing specific ink combinations. In order to control printers so that the combination or mixture of inks results in specific colour or spectra requires a spectral reflectance printer model that estimates reflectance spectra from nominal dot coverage. The printer models have one of the key roles in accurate communication of colour to the printed media. Accordingly, this has been one of the most active research areas in printing. The research direction was toward improvement of the model accuracy, model simplicity and toward minimal resources used by the model in terms of computational power and usage of material. The contribution of the work included in the thesis is also directed toward improvement of the printer models but for the multichannel printing. The thesis is focused primarily on improving existing spectral printer models and developing a new model. In addition, the aim was to develop and implement a multichannel halftoning method which should provide with high image quality. Therefore, the research goals of the thesis were: maximal accuracy of printer models, optimal resource usage and maximal image quality of halftoning and whole spectral reproduction system. Maximal colour accuracy of a model but with the least resources used is achieved by optimizing printer model calibration process. First, estimation of the physical and optical dot gain is performed with newly proposed method and model. Second, a custom training target is estimated using the proposed new method. These two proposed methods and one proposed model were at the same time the means of optimal resource usage, both in computational time and material. The third goal was satisfied with newly proposed halftoning method for multichannel printing. This method also satisfies the goal of optimal computational time but with maintaining high image quality. When applied in spectral reproduction workflow, this halftoning reduces noise induced in an inversion of the printer model. Finally, a case study was conducted on the practical use of multichannel printers and spectral reproduction workflow. In addition to a gamut comparison in colour space, it is shown that otherwise limited reach of spectral printing could potentially be used to simulate spectra and colour of textile fabrics

Prediction of screener-induced moire in digital halftone pattern generation

In the graphic arts, objectionable moire patterns are often observed on films or printed products due to the interaction of various periodic structures of halftone images. A particular type of moire pattern that results from digital halftoning at arbitrary angles and frequencies using a virtual screen function has been studied. A computer program was developed that produces uniform digital halftone patterns using a virtual screen approach and that calculates the corresponding amplitude spectra. It was found that aliasing due to the sampling of the virtual screen causes low frequency components in the amplitude spectrum. Moire patterns with fundamental vector frequencies equal to those of the strong aliased components were observed in halftone images reconstructed on a film recorder. Moire was also observed at frequencies not represented or under represented in the amplitude spectrum. It is shown that this moire effect is due to the additive beating of two or more higher frequency components that differ by the frequency of the observed moire. It is suggested that the non-linearities of the film recording process amplify this effect . The effects on the resulting moire patterns of varying the halftone parameters of dot size, dot shape, screen angle, and screen frequency were examined. In general, the amplitude spectra are complex, indicating many overlapping patterns. Screener induced moire was found to behave in a nearly identical manner to that induced by digital scanning of an existing halftone

Sparse Modeling for Image and Vision Processing

In recent years, a large amount of multi-disciplinary research has been conducted on sparse models and their applications. In statistics and machine learning, the sparsity principle is used to perform model selection---that is, automatically selecting a simple model among a large collection of them. In signal processing, sparse coding consists of representing data with linear combinations of a few dictionary elements. Subsequently, the corresponding tools have been widely adopted by several scientific communities such as neuroscience, bioinformatics, or computer vision. The goal of this monograph is to offer a self-contained view of sparse modeling for visual recognition and image processing. More specifically, we focus on applications where the dictionary is learned and adapted to data, yielding a compact representation that has been successful in various contexts.Comment: 205 pages, to appear in Foundations and Trends in Computer Graphics and Visio