42 research outputs found
Learning to Become an Expert: Deep Networks Applied To Super-Resolution Microscopy
With super-resolution optical microscopy, it is now possible to observe
molecular interactions in living cells. The obtained images have a very high
spatial precision but their overall quality can vary a lot depending on the
structure of interest and the imaging parameters. Moreover, evaluating this
quality is often difficult for non-expert users. In this work, we tackle the
problem of learning the quality function of super- resolution images from
scores provided by experts. More specifically, we are proposing a system based
on a deep neural network that can provide a quantitative quality measure of a
STED image of neuronal structures given as input. We conduct a user study in
order to evaluate the quality of the predictions of the neural network against
those of a human expert. Results show the potential while highlighting some of
the limits of the proposed approach.Comment: Accepted to the Thirtieth Innovative Applications of Artificial
Intelligence Conference (IAAI), 201
Real-Time Illumination Capture and Rendering on Mobile Devices
We present our efforts to develop methods for rendering 3D objects on mobile devices using real-world dynamic illumination from the user’s environment. To achieve this, we use the front and back cameras on the mobile device to estimate the light distribution in the environment in real time. We then create a dynamic illumination map and render the object at interactive rates in a browser on the device using a web-based graphics API. This project achieves one of the goals of our related work on realistic visualization of virtual objects: to make virtual objects appear to be situated within the scene they are observed in
Virtual Home Staging: Inverse Rendering and Editing an Indoor Panorama under Natural Illumination
We propose a novel inverse rendering method that enables the transformation
of existing indoor panoramas with new indoor furniture layouts under natural
illumination. To achieve this, we captured indoor HDR panoramas along with
real-time outdoor hemispherical HDR photographs. Indoor and outdoor HDR images
were linearly calibrated with measured absolute luminance values for accurate
scene relighting. Our method consists of three key components: (1) panoramic
furniture detection and removal, (2) automatic floor layout design, and (3)
global rendering with scene geometry, new furniture objects, and a real-time
outdoor photograph. We demonstrate the effectiveness of our workflow in
rendering indoor scenes under different outdoor illumination conditions.
Additionally, we contribute a new calibrated HDR (Cali-HDR) dataset that
consists of 137 calibrated indoor panoramas and their associated outdoor
photographs
EMLight: Lighting Estimation via Spherical Distribution Approximation
Illumination estimation from a single image is critical in 3D rendering and
it has been investigated extensively in the computer vision and computer
graphic research community. On the other hand, existing works estimate
illumination by either regressing light parameters or generating illumination
maps that are often hard to optimize or tend to produce inaccurate predictions.
We propose Earth Mover Light (EMLight), an illumination estimation framework
that leverages a regression network and a neural projector for accurate
illumination estimation. We decompose the illumination map into spherical light
distribution, light intensity and the ambient term, and define the illumination
estimation as a parameter regression task for the three illumination
components. Motivated by the Earth Mover distance, we design a novel spherical
mover's loss that guides to regress light distribution parameters accurately by
taking advantage of the subtleties of spherical distribution. Under the
guidance of the predicted spherical distribution, light intensity and ambient
term, the neural projector synthesizes panoramic illumination maps with
realistic light frequency. Extensive experiments show that EMLight achieves
accurate illumination estimation and the generated relighting in 3D object
embedding exhibits superior plausibility and fidelity as compared with
state-of-the-art methods.Comment: Accepted to AAAI 202