968 research outputs found
Joint Material and Illumination Estimation from Photo Sets in the Wild
Faithful manipulation of shape, material, and illumination in 2D Internet
images would greatly benefit from a reliable factorization of appearance into
material (i.e., diffuse and specular) and illumination (i.e., environment
maps). On the one hand, current methods that produce very high fidelity
results, typically require controlled settings, expensive devices, or
significant manual effort. To the other hand, methods that are automatic and
work on 'in the wild' Internet images, often extract only low-frequency
lighting or diffuse materials. In this work, we propose to make use of a set of
photographs in order to jointly estimate the non-diffuse materials and sharp
lighting in an uncontrolled setting. Our key observation is that seeing
multiple instances of the same material under different illumination (i.e.,
environment), and different materials under the same illumination provide
valuable constraints that can be exploited to yield a high-quality solution
(i.e., specular materials and environment illumination) for all the observed
materials and environments. Similar constraints also arise when observing
multiple materials in a single environment, or a single material across
multiple environments. The core of this approach is an optimization procedure
that uses two neural networks that are trained on synthetic images to predict
good gradients in parametric space given observation of reflected light. We
evaluate our method on a range of synthetic and real examples to generate
high-quality estimates, qualitatively compare our results against
state-of-the-art alternatives via a user study, and demonstrate
photo-consistent image manipulation that is otherwise very challenging to
achieve
An L1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition
Identifying sparse salient structures from dense pixels is a longstanding problem in visual computing. Solutions to this problem can benefit both image manipulation and understanding. In this paper, we introduce an image transform based on the L1 norm for piecewise image flattening. This transform can effectively preserve and sharpen salient edges and contours while eliminating insignificant details, producing a nearly piecewise constant image with sparse structures. A variant of this image transform can perform edge-preserving smoothing more effectively than existing state-of-the-art algorithms. We further present a new method for complex scene-level intrinsic image decomposition. Our method relies on the above image transform to suppress surface shading variations, and perform probabilistic reflectance clustering on the flattened image instead of the original input image to achieve higher accuracy. Extensive testing on the Intrinsic-Images-in-the-Wild database indicates our method can perform significantly better than existing techniques both visually and numerically. The obtained intrinsic images have been successfully used in two applications, surface retexturing and 3D object compositing in photographs.postprin
User-assisted intrinsic images
For many computational photography applications, the lighting and
materials in the scene are critical pieces of information. We seek
to obtain intrinsic images, which decompose a photo into the product
of an illumination component that represents lighting effects
and a reflectance component that is the color of the observed material.
This is an under-constrained problem and automatic methods
are challenged by complex natural images. We describe a new
approach that enables users to guide an optimization with simple
indications such as regions of constant reflectance or illumination.
Based on a simple assumption on local reflectance distributions, we
derive a new propagation energy that enables a closed form solution
using linear least-squares. We achieve fast performance by introducing
a novel downsampling that preserves local color distributions.
We demonstrate intrinsic image decomposition on a variety
of images and show applications.National Science Foundation (U.S.) (NSF CAREER award 0447561)Institut national de recherche en informatique et en automatique (France) (Associate Research Team âFlexible Renderingâ)Microsoft Research (New Faculty Fellowship)Alfred P. Sloan Foundation (Research Fellowship)Quanta Computer, Inc. (MIT-Quanta T Party
Alternating gaze in multi-party storytelling
We present a single case study on gaze alternationâ in three-party storytelling. The study makes use of the XML method, a âcombinatorial approachâ (Haugh & Musgrave 2019) involving multimodal CA transcription converted into the XML syntax. We approach gaze alternation via (i) the addressee-status hypothesis, (ii) the texturing hypothesis, and (iii) the acceleration hypothesis. Hypothesis (i) proposes that the storyteller alternatingly looks at the recipients not only when their addressee status is symmetrical but also when their addressee status is asymmetrical. Hypothesis (ii) predicts that gaze alternation âtexturesâ the telling by occurring when the storytelling progresses from one segment to another. Hypothesis (iii) states that gaze alternation accelerates toward
Climax and decelerates in Post-completion sequences. The analyses support the hypotheses. They suggest that alternating gaze works against the danger of exclusion caused by the dyadic structure of conversation. It further partakes in story organization as it occurs at points of transition from one story section to another section. Finally, accelerated gaze alternation constitutes an indexical process drawing the recipientsâ attention to the immediate relevance of stance display (Stivers 2008). We conclude that the three hypotheses warrant further investigation to determine their generalizability across speakers and speech situations
- âŠ