7,331 research outputs found
Interpretable Transformations with Encoder-Decoder Networks
Deep feature spaces have the capacity to encode complex transformations of
their input data. However, understanding the relative feature-space
relationship between two transformed encoded images is difficult. For instance,
what is the relative feature space relationship between two rotated images?
What is decoded when we interpolate in feature space? Ideally, we want to
disentangle confounding factors, such as pose, appearance, and illumination,
from object identity. Disentangling these is difficult because they interact in
very nonlinear ways. We propose a simple method to construct a deep feature
space, with explicitly disentangled representations of several known
transformations. A person or algorithm can then manipulate the disentangled
representation, for example, to re-render an image with explicit control over
parameterized degrees of freedom. The feature space is constructed using a
transforming encoder-decoder network with a custom feature transform layer,
acting on the hidden representations. We demonstrate the advantages of explicit
disentangling on a variety of datasets and transformations, and as an aid for
traditional tasks, such as classification.Comment: Accepted at ICCV 201
CNN-based Real-time Dense Face Reconstruction with Inverse-rendered Photo-realistic Face Images
With the powerfulness of convolution neural networks (CNN), CNN based face
reconstruction has recently shown promising performance in reconstructing
detailed face shape from 2D face images. The success of CNN-based methods
relies on a large number of labeled data. The state-of-the-art synthesizes such
data using a coarse morphable face model, which however has difficulty to
generate detailed photo-realistic images of faces (with wrinkles). This paper
presents a novel face data generation method. Specifically, we render a large
number of photo-realistic face images with different attributes based on
inverse rendering. Furthermore, we construct a fine-detailed face image dataset
by transferring different scales of details from one image to another. We also
construct a large number of video-type adjacent frame pairs by simulating the
distribution of real video data. With these nicely constructed datasets, we
propose a coarse-to-fine learning framework consisting of three convolutional
networks. The networks are trained for real-time detailed 3D face
reconstruction from monocular video as well as from a single image. Extensive
experimental results demonstrate that our framework can produce high-quality
reconstruction but with much less computation time compared to the
state-of-the-art. Moreover, our method is robust to pose, expression and
lighting due to the diversity of data.Comment: Accepted by IEEE Transactions on Pattern Analysis and Machine
Intelligence, 201
Photochemical synthesis of a “cage” compound in a microreactor: Rigorous comparison with a batch photoreactor
An intramolecular [2 + 2] photocycloaddition is performed in a microphotoreactor (0.81 mL) built by winding FEP tubing around a commercially available Pyrex immersion well in which a medium pressure mercury lamp is inserted. A rigorous comparison with a batch photoreactor (225 mL) is proposed by means of a simple model coupling the reaction kinetics with the mass, momentum and radiative transfer
equations. This serves as a basis to explain why the chemical conversion and the irradiation time are respectively increased and reduced in the microphotoreactor relative to those in the batch photoreactor. Through this simple model reaction, some criteria for transposing photochemical synthesis from a batch photoreactor to a continuous microphotoreactor are defined
Path-tracing Monte Carlo Library for 3D Radiative Transfer in Highly Resolved Cloudy Atmospheres
Interactions between clouds and radiation are at the root of many
difficulties in numerically predicting future weather and climate and in
retrieving the state of the atmosphere from remote sensing observations. The
large range of issues related to these interactions, and in particular to
three-dimensional interactions, motivated the development of accurate radiative
tools able to compute all types of radiative metrics, from monochromatic, local
and directional observables, to integrated energetic quantities. In the
continuity of this community effort, we propose here an open-source library for
general use in Monte Carlo algorithms. This library is devoted to the
acceleration of path-tracing in complex data, typically high-resolution
large-domain grounds and clouds. The main algorithmic advances embedded in the
library are those related to the construction and traversal of hierarchical
grids accelerating the tracing of paths through heterogeneous fields in
null-collision (maximum cross-section) algorithms. We show that with these
hierarchical grids, the computing time is only weakly sensitivive to the
refinement of the volumetric data. The library is tested with a rendering
algorithm that produces synthetic images of cloud radiances. Two other examples
are given as illustrations, that are respectively used to analyse the
transmission of solar radiation under a cloud together with its sensitivity to
an optical parameter, and to assess a parametrization of 3D radiative effects
of clouds.Comment: Submitted to JAMES, revised and submitted again (this is v2
Scene relighting and editing for improved object insertion
Abstract. The goal of this thesis is to develop a scene relighting and object insertion pipeline using Neural Radiance Fields (NeRF) to incorporate one or more objects into an outdoor environment scene. The output is a 3D mesh that embodies decomposed bidirectional reflectance distribution function (BRDF) characteristics, which interact with varying light source positions and strengths. To achieve this objective, the thesis is divided into two sub-tasks.
The first sub-task involves extracting visual information about the outdoor environment from a sparse set of corresponding images. A neural representation is constructed, providing a comprehensive understanding of the constituent elements, such as materials, geometry, illumination, and shadows. The second sub-task involves generating a neural representation of the inserted object using either real-world images or synthetic data.
To accomplish these objectives, the thesis draws on existing literature in computer vision and computer graphics. Different approaches are assessed to identify their advantages and disadvantages, with detailed descriptions of the chosen techniques provided, highlighting their functioning to produce the ultimate outcome.
Overall, this thesis aims to provide a framework for compositing and relighting that is grounded in NeRF and allows for the seamless integration of objects into outdoor environments. The outcome of this work has potential applications in various domains, such as visual effects, gaming, and virtual reality
- …