Perceptual error optimization for Monte Carlo rendering

Realistic image synthesis involves computing high-dimensional light transport integrals which in practice are numerically estimated using Monte Carlo integration. The error of this estimation manifests itself in the image as visually displeasing aliasing or noise. To ameliorate this, we develop a theoretical framework for optimizing screen-space error distribution. Our model is flexible and works for arbitrary target error power spectra. We focus on perceptual error optimization by leveraging models of the human visual system's (HVS) point spread function (PSF) from halftoning literature. This results in a specific optimization problem whose solution distributes the error as visually pleasing blue noise in image space. We develop a set of algorithms that provide a trade-off between quality and speed, showing substantial improvements over prior state of the art. We perform evaluations using both quantitative and perceptual error metrics to support our analysis, and provide extensive supplemental material to help evaluate the perceptual improvements achieved by our methods

New methods for digital halftoning and inverse halftoning

Halftoning is the rendition of continuous-tone pictures on bi-level displays. Here we first review some of the halftoning algorithms which have a direct bearing on our paper and then describe some of the more recent advances in the field. Dot diffusion halftoning has the advantage of pixel-level parallelism, unlike the popular error diffusion halftoning method. We first review the dot diffusion algorithm and describe a recent method to improve its image quality by taking advantage of the Human Visual System function. Then we discuss the inverse halftoning problem: The reconstruction of a continuous tone image from its halftone. We briefly review the methods for inverse halftoning, and discuss the advantages of a recent algorithm, namely, the Look Up Table (LUT)Method. This method is extremely fast and achieves image quality comparable to that of the best known methods. It can be applied to any halftoning scheme. We then introduce LUT based halftoning and tree-structured LUT (TLUT)halftoning. We demonstrate how halftone image quality in between that of error diffusion and Direct Binary Search (DBS)can be achieved depending on the size of tree structure in TLUT algorithm while keeping the complexity of the algorithm much lower than that of DBS

Perceptual Error Optimization for {Monte Carlo} Rendering

Realistic image synthesis involves computing high-dimensional light transport integrals which in practice are numerically estimated using Monte Carlo integration. The error of this estimation manifests itself in the image as visually displeasing aliasing or noise. To ameliorate this, we develop a theoretical framework for optimizing screen-space error distribution. Our model is flexible and works for arbitrary target error power spectra. We focus on perceptual error optimization by leveraging models of the human visual system's (HVS) point spread function (PSF) from halftoning literature. This results in a specific optimization problem whose solution distributes the error as visually pleasing blue noise in image space. We develop a set of algorithms that provide a trade-off between quality and speed, showing substantial improvements over prior state of the art. We perform evaluations using both quantitative and perceptual error metrics to support our analysis, and provide extensive supplemental material to help evaluate the perceptual improvements achieved by our methods

Development Of Digital Half toning Techniques For Grayscale Image

In this paper a complete overview of digital halftoning is given. This paper concludes all about haftoning from its definition to different methods which is used to achieve it. Also drawback of each method is given in this paper. How a new method is better than previous one it is also explained theoretically and mathematically. Floyd & Steinberg's error diffusion technique is a well-known approach to digital halftoning. Its main drawback is that it is inherently serial. This paper is briefly discussing about the method and a complete process to achieve error diffusion and thus to achieve halftoning. ‘Any model which is designed for the human visual system (HVS) is an important component of many halftoning algorithms. Also by using the iterative direct binary search (DBS) algorithm, we have compared the halftone texture quality which is provided by the four HVS models’. We design filter for the four different HVS model and tried to compare these models. Using the different HVS model we have calculated the MSE and thus got the best HVS model

Efficient Halftoning via Deep Reinforcement Learning

Halftoning aims to reproduce a continuous-tone image with pixels whose intensities are constrained to two discrete levels. This technique has been deployed on every printer, and the majority of them adopt fast methods (e.g., ordered dithering, error diffusion) that fail to render structural details, which determine halftone's quality. Other prior methods of pursuing visual pleasure by searching for the optimal halftone solution, on the contrary, suffer from their high computational cost. In this paper, we propose a fast and structure-aware halftoning method via a data-driven approach. Specifically, we formulate halftoning as a reinforcement learning problem, in which each binary pixel's value is regarded as an action chosen by a virtual agent with a shared fully convolutional neural network (CNN) policy. In the offline phase, an effective gradient estimator is utilized to train the agents in producing high-quality halftones in one action step. Then, halftones can be generated online by one fast CNN inference. Besides, we propose a novel anisotropy suppressing loss function, which brings the desirable blue-noise property. Finally, we find that optimizing SSIM could result in holes in flat areas, which can be avoided by weighting the metric with the contone's contrast map. Experiments show that our framework can effectively train a light-weight CNN, which is 15x faster than previous structure-aware methods, to generate blue-noise halftones with satisfactory visual quality. We also present a prototype of deep multitoning to demonstrate the extensibility of our method

Image Quality Metrics for Stochastic Rasterization

We develop a simple perceptual image quality metric for images resulting from stochastic rasterization. The new metric is based on the frequency selectivity of cortical cells, using ideas derived from existing perceptual metrics and research of the human visual system. Masking is not taken into account in the metric, since it does not have a significant effect in this specific application. The new metric achieves high correlation with results from HDR-VDP2 while being conceptually simple and accurately reflecting smaller quality differences than the existing metrics. In addition to HDR-VDP2, measurement results are compared against MS-SSIM results. The new metric is applied to a set of images produced with different sampling schemes to provide quantitative information about the relative quality, strengths, and weaknesses of the different sampling schemes. Several purpose-built three-dimensional test scenes are used for this quality analysis in addition to a few widely used natural scenes. The star discrepancy of sampling patterns is found to be correlated to the average perceptual quality, even though discrepancy can not be recommended as the sole method for estimating perceptual quality. A hardware-friendly low-discrepancy sampling scheme achieves generally good results, but the quality difference to simpler per-pixel stratified sampling decreases as the sample count increases. A comprehensive mathematical model of rendering discrete frames from dynamic 3D scenes is provided as background to the quality analysis

Structure-aware halftoning

our result faithfully preserves the texture details as well as the local tone. All images have the same resolution of 445×377. This paper presents an optimization-based halftoning technique that preserves the structure and tone similarities between the original and the halftone images. By optimizing an objective function consisting of both the structure and the tone metrics, the generated halftone images preserve visually sensitive texture details as well as the local tone. It possesses the blue-noise property and does not introduce annoying patterns. Unlike the existing edge-enhancement halftoning, the proposed method does not suffer from the deficiencies of edge detector. Our method is tested on various types of images. In multiple experiments and the user study, our method consistently obtains the best scores among all tested methods.

New methods for digital halftoning and inverse halftoning

Halftoning is the rendition of continuous-tone pictures on bi-level displays. Here we first review some of the halftoning algorithms which have a direct bearing on our paper and then describe some of the more recent advances in the field. Dot diffusion halftoning has the advantage of pixel-level parallelism, unlike the popular error diffusion halftoning method. We first review the dot diffusion algorithm and describe a recent method to improve its image quality by taking advantage of the Human Visual System function. Then we discuss the inverse halftoning problem: The reconstruction of a continuous tone image from its halftone. We briefly review the methods for inverse halftoning, and discuss the advantages of a recent algorithm, namely, the Look Up Table (LUT)Method. This method is extremely fast and achieves image quality comparable to that of the best known methods. It can be applied to any halftoning scheme. We then introduce LUT based halftoning and tree-structured LUT (TLUT)halftoning. We demonstrate how halftone image quality in between that of error diffusion and Direct Binary Search (DBS)can be achieved depending on the size of tree structure in TLUT algorithm while keeping the complexity of the algorithm much lower than that of DBS

Grayscale Digital Halftoning Using Optimization Techniques

In this paper a complete outline of advanced Digital halftoning techniques is given. This paper explains halftoning from its definition, application of halftoning to different useful techniques which are improved to give a high signal to noise ratio image. Apart from signal to noise ratio another parameter which measures the similarity between two images is also shown in this paper. Additionally the drawback of each method and comparison of the SNR, and SSIM of all methods is also shown in this paper. Error diffusion technique using FS, Stucki and JJN filters is an efficient approach to halftoning. Its main drawback is that it undergoes linear distortion. This paper is completely describing the error diffusion method and the improvements made to error diffusion so as to get a well-defined and a visually pleasing halftone image. However an evolutionary algorithms called as particle swarm optimization and Genetic Algorithm are used to create filters for the image block wise and comparing with that of the image and then finally reconstructing the whole image. In these methods of PSO and GA the cost function is formulated based on the SSIM, average minority pixel distance and the string with the best cost function value is sorted using the Evolutionary algorithms. As the human eye acts as a spatial low pass filter the image which is to be halftoned is filtered through any visual model such as a HVS model and then undergoes through the process of evolutionary algorithms