Rethinking PRL: A Multiscale Progressively Residual Learning Network for Inverse Halftoning

Image inverse halftoning is a classic image restoration task, aiming to recover continuous-tone images from halftone images with only bilevel pixels. Because the halftone images lose much of the original image content, inverse halftoning is a classic ill-problem. Although existing inverse halftoning algorithms achieve good performance, their results lose image details and features. Therefore, it is still a challenge to recover high-quality continuous-tone images. In this paper, we propose an end-to-end multiscale progressively residual learning network (MSPRL), which has a UNet architecture and takes multiscale input images. To make full use of different input image information, we design a shallow feature extraction module to capture similar features between images of different scales. We systematically study the performance of different methods and compare them with our proposed method. In addition, we employ different training strategies to optimize the model, which is important for optimizing the training process and improving performance. Extensive experiments demonstrate that our MSPRL model obtains considerable performance gains in detail restoration

Task-Driven Dictionary Learning

Modeling data with linear combinations of a few elements from a learned dictionary has been the focus of much recent research in machine learning, neuroscience and signal processing. For signals such as natural images that admit such sparse representations, it is now well established that these models are well suited to restoration tasks. In this context, learning the dictionary amounts to solving a large-scale matrix factorization problem, which can be done efficiently with classical optimization tools. The same approach has also been used for learning features from data for other purposes, e.g., image classification, but tuning the dictionary in a supervised way for these tasks has proven to be more difficult. In this paper, we present a general formulation for supervised dictionary learning adapted to a wide variety of tasks, and present an efficient algorithm for solving the corresponding optimization problem. Experiments on handwritten digit classification, digital art identification, nonlinear inverse image problems, and compressed sensing demonstrate that our approach is effective in large-scale settings, and is well suited to supervised and semi-supervised classification, as well as regression tasks for data that admit sparse representations.Comment: final draft post-refereein

Dithering by Differences of Convex Functions

Motivated by a recent halftoning method which is based on electrostatic principles, we analyse a halftoning framework where one minimizes a functional consisting of the difference of two convex functions (DC). One of them describes attracting forces caused by the image gray values, the other one enforces repulsion between points. In one dimension, the minimizers of our functional can be computed analytically and have the following desired properties: the points are pairwise distinct, lie within the image frame and can be placed at grid points. In the two-dimensional setting, we prove some useful properties of our functional like its coercivity and suggest to compute a minimizer by a forward-backward splitting algorithm. We show that the sequence produced by such an algorithm converges to a critical point of our functional. Furthermore, we suggest to compute the special sums occurring in each iteration step by a fast summation technique based on the fast Fourier transform at non-equispaced knots which requires only Ο(m log(m)) arithmetic operations for m points. Finally, we present numerical results showing the excellent performance of our DC dithering method

Studies on Imaging System and Machine Learning: 3D Halftoning and Human Facial Landmark Localization

In this dissertation, studies on digital halftoning and human facial landmark localization will be discussed. 3D printing is becoming increasingly popular around the world today. By utilizing 3D printing technology, customized products can be manufactured much more quickly and efficiently with much less cost. However, 3D printing still suffers from low-quality surface reproduction compared with 2D printing. One approach to improve it is to develop an advanced halftoning algorithm for 3D printing. In this presentation, we will describe a novel method to 3D halftoning that can cooperate with 3D printing technology in order to generate a high-quality surface reproduction. In the second part of this report, a new method named direct element swap to create a threshold matrix for halftoning is proposed. This method directly swaps the elements in a threshold matrix to find the best element arrangement by minimizing a designated perceived error metric. Through experimental results, the new method yields halftone quality that is competitive with the conventional level-by-level matrix design method. Besides, by using direct element swap method, for the first time, threshold matrix can be designed through being trained with real images. In the second part of the dissertation, a novel facial landmark detection system is presented. Facial landmark detection plays a critical role in many face analysis tasks. However, it still remains a very challenging problem. The challenges come from the large variations of face appearance caused by different illuminations, different facial expressions, different yaw, pitch and roll angles of heads and different image qualities. To tackle this problem, a novel coarse-to-fine cascaded convolutional neural network system for robust facial landmark detection of faces in the wild is presented. The experiment result shows our method outperforms other state-of-the-art methods on public test datasets. Besides, a frontal and profile landmark localization system is proposed and designed. By using a frontal/profile face classifier, either frontal landmark configuration or profile landmark configuration is employed in the facial landmark prediction based on the input face yaw angle

Radial Basis Functions: Biomedical Applications and Parallelization

Radial basis function (RBF) is a real-valued function whose values depend only on the distances between an interpolation point and a set of user-specified points called centers. RBF interpolation is one of the primary methods to reconstruct functions from multi-dimensional scattered data. Its abilities to generalize arbitrary space dimensions and to provide spectral accuracy have made it particularly popular in different application areas, including but not limited to: finding numerical solutions of partial differential equations (PDEs), image processing, computer vision and graphics, deep learning and neural networks, etc. The present thesis discusses three applications of RBF interpolation in biomedical engineering areas: (1) Calcium dynamics modeling, in which we numerically solve a set of PDEs by using meshless numerical methods and RBF-based interpolation techniques; (2) Image restoration and transformation, where an image is restored from its triangular mesh representation or transformed under translation, rotation, and scaling, etc. from its original form; (3) Porous structure design, in which the RBF interpolation used to reconstruct a 3D volume containing porous structures from a set of regularly or randomly placed points inside a user-provided surface shape. All these three applications have been investigated and their effectiveness has been supported with numerous experimental results. In particular, we innovatively utilize anisotropic distance metrics to define the distance in RBF interpolation and apply them to the aforementioned second and third applications, which show significant improvement in preserving image features or capturing connected porous structures over the isotropic distance-based RBF method. Beside the algorithm designs and their applications in biomedical areas, we also explore several common parallelization techniques (including OpenMP and CUDA-based GPU programming) to accelerate the performance of the present algorithms. In particular, we analyze how parallel programming can help RBF interpolation to speed up the meshless PDE solver as well as image processing. While RBF has been widely used in various science and engineering fields, the current thesis is expected to trigger some more interest from computational scientists or students into this fast-growing area and specifically apply these techniques to biomedical problems such as the ones investigated in the present work

Hardware-accelerated algorithms in visual computing

This thesis presents new parallel algorithms which accelerate computer vision methods by the use of graphics processors (GPUs) and evaluates them with respect to their speed, scalability, and the quality of their results. It covers the fields of homogeneous and anisotropic diffusion processes, diffusion image inpainting, optic flow, and halftoning. In this turn, it compares different solvers for homogeneous diffusion and presents a novel \u27extended\u27 box filter. Moreover, it suggests to use the fast explicit diffusion scheme (FED) as an efficient and flexible solver for nonlinear and in particular for anisotropic parabolic diffusion problems on graphics hardware. For elliptic diffusion-like processes, it recommends to use cascadic FED or Fast Jacobi schemes. The presented optic flow algorithm represents one of the fastest yet very accurate techniques. Finally, it presents a novel halftoning scheme which yields state-of-the-art results for many applications in image processing and computer graphics.Diese Arbeit präsentiert neue parallele Algorithmen zur Beschleunigung von Methoden in der Bildinformatik mittels Grafikprozessoren (GPUs), und evaluiert diese im Hinblick auf Geschwindigkeit, Skalierungsverhalten, und Qualität der Resultate. Sie behandelt dabei die Gebiete der homogenen und anisotropen Diffusionsprozesse, Inpainting (Bildvervollständigung) mittels Diffusion, die Bestimmung des optischen Flusses, sowie Halbtonverfahren. Dabei werden verschiedene Löser für homogene Diffusion verglichen und ein neuer \u27erweiterter\u27 Mittelwertfilter präsentiert. Ferner wird vorgeschlagen, das schnelle explizite Diffusionsschema (FED) als effizienten und flexiblen Löser für parabolische nichtlineare und speziell anisotrope Diffusionsprozesse auf Grafikprozessoren einzusetzen. Für elliptische diffusionsartige Prozesse wird hingegen empfohlen, kaskadierte FED- oder schnelle Jacobi-Verfahren einzusetzen. Der vorgestellte Algorithmus zur Berechnung des optischen Flusses stellt eines der schnellsten und dennoch äußerst genauen Verfahren dar. Schließlich wird ein neues Halbtonverfahren präsentiert, das in vielen Bereichen der Bildverarbeitung und Computergrafik Ergebnisse produziert, die den Stand der Technik repräsentieren

The modelling of natural imperfections and an improved space filling curve halftoning technique.

by Tien-tsin Wong.Thesis (M.Phil.)--Chinese University of Hong Kong, 1994.Includes bibliographical references (leaves 72-79).Chapter 1 --- Introduction --- p.1Chapter 1.1 --- The Modelling of Natural Imperfections --- p.1Chapter 1.2 --- Improved Clustered-dot Space Filling Curve Halftoning Technique --- p.2Chapter 1.3 --- Structure of the Thesis --- p.3Chapter 2 --- The Modelling of Natural Imperfections --- p.4Chapter 2.1 --- Introduction --- p.4Chapter 2.2 --- Related Work --- p.6Chapter 2.2.1 --- Texture Mapping --- p.6Chapter 2.2.2 --- Blinn's Dusty Surfaces --- p.7Chapter 2.2.3 --- Imperfection Rule-based Systems --- p.7Chapter 2.3 --- Natural Surface Imperfections --- p.8Chapter 2.3.1 --- Dust Accumulation --- p.8Chapter 2.3.2 --- Scratching --- p.10Chapter 2.3.3 --- Rusting --- p.10Chapter 2.3.4 --- Mould --- p.11Chapter 2.4 --- New Modelling Framework for Natural Imperfections --- p.13Chapter 2.4.1 --- Calculation of Tendency --- p.13Chapter 2.4.2 --- Generation of Chaotic Pattern --- p.19Chapter 2.5 --- Modelling of Dust Accumulation --- p.21Chapter 2.5.1 --- Predicted Tendency of Dust Accumulation --- p.22Chapter 2.5.2 --- External Factors --- p.24Chapter 2.5.3 --- Generation of Fuzzy Dust Layer --- p.30Chapter 2.5.4 --- Implementation Issues --- p.31Chapter 2.6 --- Modelling of Scratching --- p.31Chapter 2.6.1 --- External Factor --- p.32Chapter 2.6.2 --- Generation of Chaotic Scratch Patterns --- p.35Chapter 2.6.3 --- Implementation Issues --- p.36Chapter 3 --- An Improved Space Filling Curve Halftoning Technique --- p.39Chapter 3.1 --- Introduction --- p.39Chapter 3.2 --- Review on Some Halftoning Techniques --- p.41Chapter 3.2.1 --- Ordered Dither --- p.41Chapter 3.2.2 --- Error Diffusion and Dither with Blue Noise --- p.42Chapter 3.2.3 --- Dot Diffusion --- p.43Chapter 3.2.4 --- Halftoning Along Space Filling Traversal --- p.43Chapter 3.2.5 --- Space Diffusion --- p.46Chapter 3.3 --- Improvements on the Clustered-Dot Space Filling Halftoning Method --- p.47Chapter 3.3.1 --- Selective Precipitation --- p.47Chapter 3.3.2 --- Adaptive Clustering --- p.50Chapter 3.4 --- Comparison With Other Methods --- p.57Chapter 3.4.1 --- Low Resolution Observations --- p.57Chapter 3.4.2 --- High Resolution Printing Results --- p.58Chapter 3.4.3 --- Analytical Comparison --- p.58Chapter 4 --- Conclusion and Future Work --- p.69Chapter 4.1 --- The Modelling of Natural Imperfections --- p.69Chapter 4.2 --- An Improved Space Filling Curve Halftoning Technique --- p.71Bibliography --- p.7