Efficient Halftoning via Deep Reinforcement Learning

Halftoning aims to reproduce a continuous-tone image with pixels whose intensities are constrained to two discrete levels. This technique has been deployed on every printer, and the majority of them adopt fast methods (e.g., ordered dithering, error diffusion) that fail to render structural details, which determine halftone's quality. Other prior methods of pursuing visual pleasure by searching for the optimal halftone solution, on the contrary, suffer from their high computational cost. In this paper, we propose a fast and structure-aware halftoning method via a data-driven approach. Specifically, we formulate halftoning as a reinforcement learning problem, in which each binary pixel's value is regarded as an action chosen by a virtual agent with a shared fully convolutional neural network (CNN) policy. In the offline phase, an effective gradient estimator is utilized to train the agents in producing high-quality halftones in one action step. Then, halftones can be generated online by one fast CNN inference. Besides, we propose a novel anisotropy suppressing loss function, which brings the desirable blue-noise property. Finally, we find that optimizing SSIM could result in holes in flat areas, which can be avoided by weighting the metric with the contone's contrast map. Experiments show that our framework can effectively train a light-weight CNN, which is 15x faster than previous structure-aware methods, to generate blue-noise halftones with satisfactory visual quality. We also present a prototype of deep multitoning to demonstrate the extensibility of our method

Studies on Imaging System and Machine Learning: 3D Halftoning and Human Facial Landmark Localization

In this dissertation, studies on digital halftoning and human facial landmark localization will be discussed. 3D printing is becoming increasingly popular around the world today. By utilizing 3D printing technology, customized products can be manufactured much more quickly and efficiently with much less cost. However, 3D printing still suffers from low-quality surface reproduction compared with 2D printing. One approach to improve it is to develop an advanced halftoning algorithm for 3D printing. In this presentation, we will describe a novel method to 3D halftoning that can cooperate with 3D printing technology in order to generate a high-quality surface reproduction. In the second part of this report, a new method named direct element swap to create a threshold matrix for halftoning is proposed. This method directly swaps the elements in a threshold matrix to find the best element arrangement by minimizing a designated perceived error metric. Through experimental results, the new method yields halftone quality that is competitive with the conventional level-by-level matrix design method. Besides, by using direct element swap method, for the first time, threshold matrix can be designed through being trained with real images. In the second part of the dissertation, a novel facial landmark detection system is presented. Facial landmark detection plays a critical role in many face analysis tasks. However, it still remains a very challenging problem. The challenges come from the large variations of face appearance caused by different illuminations, different facial expressions, different yaw, pitch and roll angles of heads and different image qualities. To tackle this problem, a novel coarse-to-fine cascaded convolutional neural network system for robust facial landmark detection of faces in the wild is presented. The experiment result shows our method outperforms other state-of-the-art methods on public test datasets. Besides, a frontal and profile landmark localization system is proposed and designed. By using a frontal/profile face classifier, either frontal landmark configuration or profile landmark configuration is employed in the facial landmark prediction based on the input face yaw angle

Design of Farthest-Point Masks for Image Halftoning

In an earlier paper, we briefly presented a new halftoning algorithm called farthest-point halftoning. In the present paper, this method is analyzed in detail, and a novel dispersion measure is defined to improve the simplicity and flexibility of the result. This new stochastic screen algorithm is loosely based on Kang's dispersed-dot ordered dither halftone array construction technique used as part of his microcluster halftoning method. Our new halftoning algorithm uses pixelwise measures of dispersion based on one proposed by Kang which is here modified to be more effective. In addition, our method exploits the concept of farthest-point sampling (FPS), introduced as a progressive irregular sampling method by Eldar et al. but uses a more efficient implementation of FPS in the construction of the dot profiles. The technique we propose is compared to other state-of-the-art dither-based halftoning methods in both qualitative and quantitative manners

Perceptual error optimization for Monte Carlo rendering

Realistic image synthesis involves computing high-dimensional light transport integrals which in practice are numerically estimated using Monte Carlo integration. The error of this estimation manifests itself in the image as visually displeasing aliasing or noise. To ameliorate this, we develop a theoretical framework for optimizing screen-space error distribution. Our model is flexible and works for arbitrary target error power spectra. We focus on perceptual error optimization by leveraging models of the human visual system's (HVS) point spread function (PSF) from halftoning literature. This results in a specific optimization problem whose solution distributes the error as visually pleasing blue noise in image space. We develop a set of algorithms that provide a trade-off between quality and speed, showing substantial improvements over prior state of the art. We perform evaluations using both quantitative and perceptual error metrics to support our analysis, and provide extensive supplemental material to help evaluate the perceptual improvements achieved by our methods

Perceptual Error Optimization for {Monte Carlo} Rendering

Realistic image synthesis involves computing high-dimensional light transport integrals which in practice are numerically estimated using Monte Carlo integration. The error of this estimation manifests itself in the image as visually displeasing aliasing or noise. To ameliorate this, we develop a theoretical framework for optimizing screen-space error distribution. Our model is flexible and works for arbitrary target error power spectra. We focus on perceptual error optimization by leveraging models of the human visual system's (HVS) point spread function (PSF) from halftoning literature. This results in a specific optimization problem whose solution distributes the error as visually pleasing blue noise in image space. We develop a set of algorithms that provide a trade-off between quality and speed, showing substantial improvements over prior state of the art. We perform evaluations using both quantitative and perceptual error metrics to support our analysis, and provide extensive supplemental material to help evaluate the perceptual improvements achieved by our methods

Development Of Digital Half toning Techniques For Grayscale Image

In this paper a complete overview of digital halftoning is given. This paper concludes all about haftoning from its definition to different methods which is used to achieve it. Also drawback of each method is given in this paper. How a new method is better than previous one it is also explained theoretically and mathematically. Floyd & Steinberg's error diffusion technique is a well-known approach to digital halftoning. Its main drawback is that it is inherently serial. This paper is briefly discussing about the method and a complete process to achieve error diffusion and thus to achieve halftoning. ‘Any model which is designed for the human visual system (HVS) is an important component of many halftoning algorithms. Also by using the iterative direct binary search (DBS) algorithm, we have compared the halftone texture quality which is provided by the four HVS models’. We design filter for the four different HVS model and tried to compare these models. Using the different HVS model we have calculated the MSE and thus got the best HVS model

A New framework for an electrophotographic printer model

Digital halftoning is a printing technology that creates the illusion of continuous tone images for printing devices such as electrophotographic printers that can only produce a limited number of tone levels. Digital halftoning works because the human visual system has limited spatial resolution which blurs the printed dots of the halftone image, creating the gray sensation of a continuous tone image. Because the printing process is imperfect it introduces distortions to the halftone image. The quality of the printed image depends, among other factors, on the complex interactions between the halftone image, the printer characteristics, the colorant, and the printing substrate. Printer models are used to assist in the development of new types of halftone algorithms that are designed to withstand the effects of printer distortions. For example, model-based halftone algorithms optimize the halftone image through an iterative process that integrates a printer model within the algorithm. The two main goals of a printer model are to provide accurate estimates of the tone and of the spatial characteristics of the printed halftone pattern. Various classes of printer models, from simple tone calibrations, to complex mechanistic models, have been reported in the literature. Existing models have one or more of the following limiting factors: they only predict tone reproduction, they depend on the halftone pattern, they require complex calibrations or complex calculations, they are printer specific, they reproduce unrealistic dot structures, and they are unable to adapt responses to new data. The two research objectives of this dissertation are (1) to introduce a new framework for printer modeling and (2) to demonstrate the feasibility of such a framework in building an electrophotographic printer model. The proposed framework introduces the concept of modeling a printer as a texture transformation machine. The basic premise is that modeling the texture differences between the output printed images and the input images encompasses all printing distortions. The feasibility of the framework was tested with a case study modeling a monotone electrophotographic printer. The printer model was implemented as a bank of feed-forward neural networks, each one specialized in modeling a group of textural features of the printed halftone pattern. The textural features were obtained using a parametric representation of texture developed from a multiresolution decomposition proposed by other researchers. The textural properties of halftone patterns were analyzed and the key texture parameters to be modeled by the bank were identified. Guidelines for the multiresolution texture decomposition and the model operational parameters and operational limits were established. A method for the selection of training sets based on the morphological properties of the halftone patterns was also developed. The model is fast and has the capability to continue to learn with additional training. The model can be easily implemented because it only requires a calibrated scanner. The model was tested with halftone patterns representing a range of spatial characteristics found in halftoning. Results show that the model provides accurate predictions for the tone and the spatial characteristics when modeling halftone patterns individually and it provides close approximations when modeling multiple halftone patterns simultaneously. The success of the model justifies continued research of this new printer model framework