Perceptual error optimization for Monte Carlo rendering

Realistic image synthesis involves computing high-dimensional light transport integrals which in practice are numerically estimated using Monte Carlo integration. The error of this estimation manifests itself in the image as visually displeasing aliasing or noise. To ameliorate this, we develop a theoretical framework for optimizing screen-space error distribution. Our model is flexible and works for arbitrary target error power spectra. We focus on perceptual error optimization by leveraging models of the human visual system's (HVS) point spread function (PSF) from halftoning literature. This results in a specific optimization problem whose solution distributes the error as visually pleasing blue noise in image space. We develop a set of algorithms that provide a trade-off between quality and speed, showing substantial improvements over prior state of the art. We perform evaluations using both quantitative and perceptual error metrics to support our analysis, and provide extensive supplemental material to help evaluate the perceptual improvements achieved by our methods

Perceptual Error Optimization for {Monte Carlo} Rendering

Realistic image synthesis involves computing high-dimensional light transport integrals which in practice are numerically estimated using Monte Carlo integration. The error of this estimation manifests itself in the image as visually displeasing aliasing or noise. To ameliorate this, we develop a theoretical framework for optimizing screen-space error distribution. Our model is flexible and works for arbitrary target error power spectra. We focus on perceptual error optimization by leveraging models of the human visual system's (HVS) point spread function (PSF) from halftoning literature. This results in a specific optimization problem whose solution distributes the error as visually pleasing blue noise in image space. We develop a set of algorithms that provide a trade-off between quality and speed, showing substantial improvements over prior state of the art. We perform evaluations using both quantitative and perceptual error metrics to support our analysis, and provide extensive supplemental material to help evaluate the perceptual improvements achieved by our methods

Dithering by Differences of Convex Functions

Motivated by a recent halftoning method which is based on electrostatic principles, we analyse a halftoning framework where one minimizes a functional consisting of the difference of two convex functions (DC). One of them describes attracting forces caused by the image gray values, the other one enforces repulsion between points. In one dimension, the minimizers of our functional can be computed analytically and have the following desired properties: the points are pairwise distinct, lie within the image frame and can be placed at grid points. In the two-dimensional setting, we prove some useful properties of our functional like its coercivity and suggest to compute a minimizer by a forward-backward splitting algorithm. We show that the sequence produced by such an algorithm converges to a critical point of our functional. Furthermore, we suggest to compute the special sums occurring in each iteration step by a fast summation technique based on the fast Fourier transform at non-equispaced knots which requires only Ο(m log(m)) arithmetic operations for m points. Finally, we present numerical results showing the excellent performance of our DC dithering method

Novel methods in image halftoning

Ankara : Department of Electrical and Electronics Engineering and Institute of Engineering and Science, Bilkent Univ., 1998.Thesis (Master's) -- Bilkent University, 1998.Includes bibliographical references leaves 97-101Halftoning refers to the problem of rendering continuous-tone (contone) images on display and printing devices which are capable of reproducing only a limited number of colors. A new adaptive halftoning method using the adaptive QR- RLS algorithm is developed for error diffusion which is one of the halftoning techniques. Also, a diagonal scanning strategy to exploit the human visual system properties in processing the image is proposed. Simulation results on color images demonstrate the superior quality of the new method compared to the existing methods. Another problem studied in this thesis is inverse halftoning which is the problem of recovering a contone image from a given halftoned image. A novel inverse halftoning method is developed for restoring a contone image from the halftoned image. A set theoretic formulation is used where sets are defined using the prior information about the problem. A new space domain projection is introduced assuming the halftoning is performed ,with error diffusion, and the error diffusion filter kernel is known. The space domain, frequency domain, and space-scale domain projections are used alternately to obtain a feasible solution for the inverse halftoning problem which does not have a unique solution. Simulation results for both grayscale and color images give good results, and demonstrate the effectiveness of the proposed inverse halftoning method.Bozkurt, GözdeM.S

Efficient Halftoning via Deep Reinforcement Learning

Halftoning aims to reproduce a continuous-tone image with pixels whose intensities are constrained to two discrete levels. This technique has been deployed on every printer, and the majority of them adopt fast methods (e.g., ordered dithering, error diffusion) that fail to render structural details, which determine halftone's quality. Other prior methods of pursuing visual pleasure by searching for the optimal halftone solution, on the contrary, suffer from their high computational cost. In this paper, we propose a fast and structure-aware halftoning method via a data-driven approach. Specifically, we formulate halftoning as a reinforcement learning problem, in which each binary pixel's value is regarded as an action chosen by a virtual agent with a shared fully convolutional neural network (CNN) policy. In the offline phase, an effective gradient estimator is utilized to train the agents in producing high-quality halftones in one action step. Then, halftones can be generated online by one fast CNN inference. Besides, we propose a novel anisotropy suppressing loss function, which brings the desirable blue-noise property. Finally, we find that optimizing SSIM could result in holes in flat areas, which can be avoided by weighting the metric with the contone's contrast map. Experiments show that our framework can effectively train a light-weight CNN, which is 15x faster than previous structure-aware methods, to generate blue-noise halftones with satisfactory visual quality. We also present a prototype of deep multitoning to demonstrate the extensibility of our method

Structure-aware halftoning

our result faithfully preserves the texture details as well as the local tone. All images have the same resolution of 445×377. This paper presents an optimization-based halftoning technique that preserves the structure and tone similarities between the original and the halftone images. By optimizing an objective function consisting of both the structure and the tone metrics, the generated halftone images preserve visually sensitive texture details as well as the local tone. It possesses the blue-noise property and does not introduce annoying patterns. Unlike the existing edge-enhancement halftoning, the proposed method does not suffer from the deficiencies of edge detector. Our method is tested on various types of images. In multiple experiments and the user study, our method consistently obtains the best scores among all tested methods.

Interactive Geometry Remeshing

We present a novel technique, both flexible and efficient, for interactive remeshing of irregular geometry. First, the original (arbitrary genus) mesh is substituted by a series of 2D maps in parameter space. Using these maps, our algorithm is then able to take advantage of established signal processing and halftoning tools that offer real-time interaction and intricate control. The user can easily combine these maps to create a control map – a map which controls the sampling density over the surface patch. This map is then sampled at interactive rates allowing the user to easily design a tailored resampling. Once this sampling is complete, a Delaunay triangulation and fast optimization are performed to perfect the final mesh. As a result, our remeshing technique is extremely versatile and general, being able to produce arbitrarily complex meshes with a variety of properties including: uniformity, regularity, semiregularity, curvature sensitive resampling, and feature preservation. We provide a high level of control over the sampling distribution allowing the user to interactively custom design the mesh based on their requirements thereby increasing their productivity in creating a wide variety of meshes