62 research outputs found
Acquisition, Modeling, and Augmentation of Reflectance for Synthetic Optical Flow Reference Data
This thesis is concerned with the acquisition, modeling, and augmentation of material reflectance to simulate high-fidelity synthetic data for computer vision tasks.
The topic is covered in three chapters: I commence with exploring the upper limits of reflectance acquisition.
I analyze state-of-the-art BTF reflectance field renderings and show that they can be applied to optical flow performance analysis with closely matching performance to real-world images.
Next, I present two methods for fitting efficient BRDF reflectance models to measured BTF data.
Both methods combined retain all relevant reflectance information as well as the surface normal details on a pixel level.
I further show that the resulting synthesized images are suited for optical flow performance analysis, with a virtually identical performance for all material types.
Finally, I present a novel method for augmenting real-world datasets with physically plausible precipitation effects, including ground surface wetting, water droplets on the windshield, and water spray and mists.
This is achieved by projecting the realworld image data onto a reconstructed virtual scene, manipulating the scene and the surface reflectance, and performing unbiased light transport simulation of the precipitation effects
Relightable Neural Assets
High-fidelity 3D assets with materials composed of fibers (including hair),
complex layered material shaders, or fine scattering geometry are ubiquitous in
high-end realistic rendering applications. Rendering such models is
computationally expensive due to heavy shaders and long scattering paths.
Moreover, implementing the shading and scattering models is non-trivial and has
to be done not only in the 3D content authoring software (which is necessarily
complex), but also in all downstream rendering solutions. For example, web and
mobile viewers for complex 3D assets are desirable, but frequently cannot
support the full shading complexity allowed by the authoring application. Our
goal is to design a neural representation for 3D assets with complex shading
that supports full relightability and full integration into existing renderers.
We provide an end-to-end shading solution at the first intersection of a ray
with the underlying geometry. All shading and scattering is precomputed and
included in the neural asset; no multiple scattering paths need to be traced,
and no complex shading models need to be implemented to render our assets,
beyond a single neural architecture. We combine an MLP decoder with a feature
grid. Shading consists of querying a feature vector, followed by an MLP
evaluation producing the final reflectance value. Our method provides
high-fidelity shading, close to the ground-truth Monte Carlo estimate even at
close-up views. We believe our neural assets could be used in practical
renderers, providing significant speed-ups and simplifying renderer
implementations
Real3D-AD: A Dataset of Point Cloud Anomaly Detection
High-precision point cloud anomaly detection is the gold standard for
identifying the defects of advancing machining and precision manufacturing.
Despite some methodological advances in this area, the scarcity of datasets and
the lack of a systematic benchmark hinder its development. We introduce
Real3D-AD, a challenging high-precision point cloud anomaly detection dataset,
addressing the limitations in the field. With 1,254 high-resolution 3D items
from forty thousand to millions of points for each item, Real3D-AD is the
largest dataset for high-precision 3D industrial anomaly detection to date.
Real3D-AD surpasses existing 3D anomaly detection datasets available regarding
point cloud resolution (0.0010mm-0.0015mm), 360 degree coverage and perfect
prototype. Additionally, we present a comprehensive benchmark for Real3D-AD,
revealing the absence of baseline methods for high-precision point cloud
anomaly detection. To address this, we propose Reg3D-AD, a registration-based
3D anomaly detection method incorporating a novel feature memory bank that
preserves local and global representations. Extensive experiments on the
Real3D-AD dataset highlight the effectiveness of Reg3D-AD. For reproducibility
and accessibility, we provide the Real3D-AD dataset, benchmark source code, and
Reg3D-AD on our website:https://github.com/M-3LAB/Real3D-AD
Neural reflectance transformation imaging
Reflectance transformation imaging (RTI) is a computational photography technique widely used in the cultural heritage and material science domains to characterize relieved surfaces. It basically consists of capturing multiple images from a fixed viewpoint with varying lights. Handling the potentially huge amount of information stored in an RTI acquisition that consists typically of 50\u2013100RGB values per pixel, allowing data exchange, interactive visualization, and material analysis, is not easy. The solution used in practical applications consists of creating \u201crelightable images\u201d by approximating the pixel information with a function of the light direction, encoded with a small number of parameters. This encoding allows the estimation of images relighted from novel, arbitrary lights, with a quality that, however, is not always satisfactory. In this paper, we present NeuralRTI, a framework for pixel-based encoding and relighting of RTI data. Using a simple autoencoder architecture, we show that it is possible to obtain a highly compressed representation that better preserves the original information and provides increased quality of virtual images relighted from novel directions, especially in the case of challenging glossy materials. We also address the problem of validating the relight quality on different surfaces, proposing a specific benchmark, SynthRTI, including image collections synthetically created with physical-based rendering and featuring objects with different materials and geometric complexity. On this dataset and as well on a collection of real acquisitions performed on heterogeneous surfaces, we demonstrate the advantages of the proposed relightable image encoding
Joint Material and Illumination Estimation from Photo Sets in the Wild
Faithful manipulation of shape, material, and illumination in 2D Internet
images would greatly benefit from a reliable factorization of appearance into
material (i.e., diffuse and specular) and illumination (i.e., environment
maps). On the one hand, current methods that produce very high fidelity
results, typically require controlled settings, expensive devices, or
significant manual effort. To the other hand, methods that are automatic and
work on 'in the wild' Internet images, often extract only low-frequency
lighting or diffuse materials. In this work, we propose to make use of a set of
photographs in order to jointly estimate the non-diffuse materials and sharp
lighting in an uncontrolled setting. Our key observation is that seeing
multiple instances of the same material under different illumination (i.e.,
environment), and different materials under the same illumination provide
valuable constraints that can be exploited to yield a high-quality solution
(i.e., specular materials and environment illumination) for all the observed
materials and environments. Similar constraints also arise when observing
multiple materials in a single environment, or a single material across
multiple environments. The core of this approach is an optimization procedure
that uses two neural networks that are trained on synthetic images to predict
good gradients in parametric space given observation of reflected light. We
evaluate our method on a range of synthetic and real examples to generate
high-quality estimates, qualitatively compare our results against
state-of-the-art alternatives via a user study, and demonstrate
photo-consistent image manipulation that is otherwise very challenging to
achieve
GenPluSSS: A Genetic Algorithm Based Plugin for Measured Subsurface Scattering Representation
This paper presents a plugin that adds a representation of homogeneous and
heterogeneous, optically thick, translucent materials on the Blender 3D
modeling tool. The working principle of this plugin is based on a combination
of Genetic Algorithm (GA) and Singular Value Decomposition (SVD)-based
subsurface scattering method (GenSSS). The proposed plugin has been implemented
using Mitsuba renderer, which is an open source rendering software. The
proposed plugin has been validated on measured subsurface scattering data. It's
shown that the proposed plugin visualizes homogeneous and heterogeneous
subsurface scattering effects, accurately, compactly and computationally
efficiently
Modelling and Visualisation of the Optical Properties of Cloth
Cloth and garment visualisations are widely used in fashion and interior design, entertaining, automotive and nautical industry and are indispensable elements of visual communication. Modern appearance models attempt to offer a complete solution for the visualisation of complex cloth properties. In the review part of the chapter, advanced methods that enable visualisation at micron resolution, methods used in three-dimensional (3D) visualisation workflow and methods used for research purposes are presented. Within the review, those methods offering a comprehensive approach and experiments on explicit clothes attributes that present specific optical phenomenon are analysed. The review of appearance models includes surface and image-based models, volumetric and explicit models. Each group is presented with the representative authors’ research group and the application and limitations of the methods. In the final part of the chapter, the visualisation of cloth specularity and porosity with an uneven surface is studied. The study and visualisation was performed using image data obtained with photography. The acquisition of structure information on a large scale namely enables the recording of structure irregularities that are very common on historical textiles, laces and also on artistic and experimental pieces of cloth. The contribution ends with the presentation of cloth visualised with the use of specular and alpha maps, which is the result of the image processing workflow
- …