415 research outputs found
Image based surface reflectance remapping for consistent and tool independent material appearence
Physically-based rendering in Computer Graphics requires the knowledge of material properties other than 3D shapes, textures and colors, in order to solve the rendering equation. A number of material models have been developed, since no model is currently able to reproduce the full range of available materials. Although only few material models have been widely adopted in current rendering systems, the lack of standardisation causes several issues in the 3D modelling
workflow, leading to a heavy tool dependency of material appearance. In industry, final decisions about products are often based on a virtual prototype, a crucial step for the production pipeline, usually developed by a collaborations among several
departments, which exchange data. Unfortunately, exchanged data often tends to differ from the original, when imported into a different application. As a result, delivering consistent visual results requires time, labour and computational cost.
This thesis begins with an examination of the current state of the art in material appearance representation and capture, in order to identify a suitable strategy to tackle material appearance consistency. Automatic solutions to this problem are suggested in this work, accounting for the constraints of real-world scenarios, where the only available information is a reference rendering and the renderer used to obtain it, with no access to the implementation of the shaders. In particular, two image-based frameworks are proposed, working under these constraints.
The first one, validated by means of perceptual studies, is aimed to the remapping of BRDF parameters and useful when the parameters used for the reference rendering are available. The second one provides consistent material appearance across different renderers, even when the parameters used for the reference are unknown. It allows the selection of an arbitrary reference rendering tool, and manipulates the output of other renderers in order to be consistent with the reference
Joint Material and Illumination Estimation from Photo Sets in the Wild
Faithful manipulation of shape, material, and illumination in 2D Internet
images would greatly benefit from a reliable factorization of appearance into
material (i.e., diffuse and specular) and illumination (i.e., environment
maps). On the one hand, current methods that produce very high fidelity
results, typically require controlled settings, expensive devices, or
significant manual effort. To the other hand, methods that are automatic and
work on 'in the wild' Internet images, often extract only low-frequency
lighting or diffuse materials. In this work, we propose to make use of a set of
photographs in order to jointly estimate the non-diffuse materials and sharp
lighting in an uncontrolled setting. Our key observation is that seeing
multiple instances of the same material under different illumination (i.e.,
environment), and different materials under the same illumination provide
valuable constraints that can be exploited to yield a high-quality solution
(i.e., specular materials and environment illumination) for all the observed
materials and environments. Similar constraints also arise when observing
multiple materials in a single environment, or a single material across
multiple environments. The core of this approach is an optimization procedure
that uses two neural networks that are trained on synthetic images to predict
good gradients in parametric space given observation of reflected light. We
evaluate our method on a range of synthetic and real examples to generate
high-quality estimates, qualitatively compare our results against
state-of-the-art alternatives via a user study, and demonstrate
photo-consistent image manipulation that is otherwise very challenging to
achieve
FROST-BRDF: A Fast and Robust Optimal Sampling Technique for BRDF Acquisition
Efficient and accurate BRDF acquisition of real world materials is a
challenging research problem that requires sampling millions of incident light
and viewing directions. To accelerate the acquisition process, one needs to
find a minimal set of sampling directions such that the recovery of the full
BRDF is accurate and robust given such samples. In this paper, we formulate
BRDF acquisition as a compressed sensing problem, where the sensing operator is
one that performs sub-sampling of the BRDF signal according to a set of optimal
sample directions. To solve this problem, we propose the Fast and Robust
Optimal Sampling Technique (FROST) for designing a provably optimal
sub-sampling operator that places light-view samples such that the recovery
error is minimized. FROST casts the problem of designing an optimal
sub-sampling operator for compressed sensing into a sparse representation
formulation under the Multiple Measurement Vector (MMV) signal model. The
proposed reformulation is exact, i.e. without any approximations, hence it
converts an intractable combinatorial problem into one that can be solved with
standard optimization techniques. As a result, FROST is accompanied by strong
theoretical guarantees from the field of compressed sensing. We perform a
thorough analysis of FROST-BRDF using a 10-fold cross-validation with publicly
available BRDF datasets and show significant advantages compared to the
state-of-the-art with respect to reconstruction quality. Finally, FROST is
simple, both conceptually and in terms of implementation, it produces
consistent results at each run, and it is at least two orders of magnitude
faster than the prior art.Comment: Submitted to IEEE Transactions on Visualization and Computer Graphics
(IEEE TVCG
Practical acquisition and rendering of diffraction effects in surface reflectance
We propose two novel contributions for measurement based rendering of diffraction effects in surface reflectance of planar homogeneous diffractive materials. As a general solution for commonly manufactured materials, we propose a practical data-driven rendering technique and a measurement approach to efficiently render complex diffraction effects in real-time. Our measurement step simply involves photographing a planar diffractive sam- ple illuminated with an LED flash. Here, we directly record the resultant diffraction pattern on the sample surface due to a narrow band point source illumination. Furthermore, we propose an efficient rendering method that exploits the measurement in conjunction with the Huygens-Fresnel principle to fit relevant diffraction parameters based on a first order approximation. Our proposed data-driven rendering method requires the precomputation of a single diffraction look up table for accurate spectral rendering of com- plex diffraction effects. Secondly, for sharp specular samples, we propose a novel method for practical measurement of the underlying diffraction grating using out-of-focus “bokeh” photography of the specular highlight. We demonstrate how the measured bokeh can be employed as a height field to drive a diffraction shader based on a first order approximation for efficient real-time rendering. Finally, we also drive analytic solutions for a few special cases of diffraction from our measurements and demonstrate realistic rendering results under complex light sources and environments
Metappearance: Meta-Learning for Visual Appearance Reproduction
There currently are two main approaches to reproducing visual appearance
using Machine Learning (ML): The first is training models that generalize over
different instances of a problem, e.g., different images from a dataset. Such
models learn priors over the data corpus and use this knowledge to provide fast
inference with little input, often as a one-shot operation. However, this
generality comes at the cost of fidelity, as such methods often struggle to
achieve the final quality required. The second approach does not train a model
that generalizes across the data, but overfits to a single instance of a
problem, e.g., a flash image of a material. This produces detailed and
high-quality results, but requires time-consuming training and is, as mere
non-linear function fitting, unable to exploit previous experience. Techniques
such as fine-tuning or auto-decoders combine both approaches but are sequential
and rely on per-exemplar optimization. We suggest to combine both techniques
end-to-end using meta-learning: We over-fit onto a single problem instance in
an inner loop, while also learning how to do so efficiently in an outer-loop
that builds intuition over many optimization runs. We demonstrate this concept
to be versatile and efficient, applying it to RGB textures, Bi-directional
Reflectance Distribution Functions (BRDFs), or Spatially-varying BRDFs
(svBRDFs)
A Qualitative and Quantitative Evaluation of 8 Clear Sky Models
We provide a qualitative and quantitative evaluation of 8 clear sky models
used in Computer Graphics. We compare the models with each other as well as
with measurements and with a reference model from the physics community. After
a short summary of the physics of the problem, we present the measurements and
the reference model, and how we "invert" it to get the model parameters. We
then give an overview of each CG model, and detail its scope, its algorithmic
complexity, and its results using the same parameters as in the reference
model. We also compare the models with a perceptual study. Our quantitative
results confirm that the less simplifications and approximations are used to
solve the physical equations, the more accurate are the results. We conclude
with a discussion of the advantages and drawbacks of each model, and how to
further improve their accuracy
Single-shot layered reflectance separation using a polarized light field camera
We present a novel computational photography technique for single shot separation of diffuse/specular reflectance as well as novel angular domain separation of layered reflectance. Our solution consists of a two-way polarized light field (TPLF) camera which simultaneously captures two orthogonal states of polarization. A single photograph of a subject acquired with the TPLF camera under polarized illumination then enables standard separation of diffuse (depolarizing) and polarization preserving specular reflectance using light field sampling. We further demonstrate that the acquired data also enables novel angular separation of layered reflectance including separation of specular reflectance and single scattering in the polarization preserving component, and separation of shallow scattering from deep scattering in the depolarizing component. We apply our approach for efficient acquisition of facial reflectance including diffuse and specular normal maps, and novel separation of photometric normals into layered reflectance normals for layered facial renderings. We demonstrate our proposed single shot layered reflectance separation to be comparable to an existing multi-shot technique that relies on structured lighting while achieving separation results under a variety of illumination conditions
BRDF representation and acquisition
Photorealistic rendering of real world environments is important in a range of different areas; including Visual Special effects, Interior/Exterior Modelling, Architectural Modelling, Cultural Heritage, Computer Games and Automotive Design.
Currently, rendering systems are able to produce photorealistic simulations of the appearance of many real-world materials. In the real world, viewer perception of objects depends on the lighting and object/material/surface characteristics, the way a surface interacts with the light and on how the light is reflected, scattered, absorbed by the surface and the impact these characteristics have on material appearance. In order to re-produce this, it is necessary to understand how materials interact with light. Thus the representation and acquisition of material models has become such an active research area.
This survey of the state-of-the-art of BRDF Representation and Acquisition presents an overview of BRDF (Bidirectional Reflectance Distribution Function) models used to represent surface/material reflection characteristics, and describes current acquisition methods for the capture and rendering of photorealistic materials
- …