65 research outputs found
Decomposing Single Images for Layered Photo Retouching
Photographers routinely compose multiple manipulated photos of the same scene into a single image, producing a fidelity difficult to achieve using any individual photo. Alternately, 3D artists set up rendering systems to produce layered images to isolate individual aspects of the light transport, which are composed into the final result in post-production. Regrettably, these approaches either take considerable time and effort to capture, or remain limited to synthetic scenes. In this paper, we suggest a method to decompose a single image into multiple layers that approximates effects such as shadow, diffuse illumination, albedo, and specular shading. To this end, we extend the idea of intrinsic images along two axes: first, by complementing shading and reflectance with specularity and occlusion, and second, by introducing directional dependence. We do so by training a convolutional neural network (CNN) with synthetic data. Such decompositions can then be manipulated in any off-the-shelf image manipulation software and composited back. We demonstrate the effectiveness of our decomposition on synthetic (i. e., rendered) and real data (i. e., photographs), and use them for photo manipulations, which are otherwise impossible to perform based on single images. We provide comparisons with state-of-the-art methods and also evaluate the quality of our decompositions via a user study measuring the effectiveness of the resultant photo retouching setup. Supplementary material and code are available for research use at geometry.cs.ucl.ac.uk/projects/2017/layered-retouching
Self-Supervised Intrinsic Image Decomposition
Intrinsic decomposition from a single image is a highly challenging task, due
to its inherent ambiguity and the scarcity of training data. In contrast to
traditional fully supervised learning approaches, in this paper we propose
learning intrinsic image decomposition by explaining the input image. Our
model, the Rendered Intrinsics Network (RIN), joins together an image
decomposition pipeline, which predicts reflectance, shape, and lighting
conditions given a single image, with a recombination function, a learned
shading model used to recompose the original input based off of intrinsic image
predictions. Our network can then use unsupervised reconstruction error as an
additional signal to improve its intermediate representations. This allows
large-scale unlabeled data to be useful during training, and also enables
transferring learned knowledge to images of unseen object categories, lighting
conditions, and shapes. Extensive experiments demonstrate that our method
performs well on both intrinsic image decomposition and knowledge transfer.Comment: NIPS 2017 camera-ready version, project page:
http://rin.csail.mit.edu
Joint Material and Illumination Estimation from Photo Sets in the Wild
Faithful manipulation of shape, material, and illumination in 2D Internet
images would greatly benefit from a reliable factorization of appearance into
material (i.e., diffuse and specular) and illumination (i.e., environment
maps). On the one hand, current methods that produce very high fidelity
results, typically require controlled settings, expensive devices, or
significant manual effort. To the other hand, methods that are automatic and
work on 'in the wild' Internet images, often extract only low-frequency
lighting or diffuse materials. In this work, we propose to make use of a set of
photographs in order to jointly estimate the non-diffuse materials and sharp
lighting in an uncontrolled setting. Our key observation is that seeing
multiple instances of the same material under different illumination (i.e.,
environment), and different materials under the same illumination provide
valuable constraints that can be exploited to yield a high-quality solution
(i.e., specular materials and environment illumination) for all the observed
materials and environments. Similar constraints also arise when observing
multiple materials in a single environment, or a single material across
multiple environments. The core of this approach is an optimization procedure
that uses two neural networks that are trained on synthetic images to predict
good gradients in parametric space given observation of reflected light. We
evaluate our method on a range of synthetic and real examples to generate
high-quality estimates, qualitatively compare our results against
state-of-the-art alternatives via a user study, and demonstrate
photo-consistent image manipulation that is otherwise very challenging to
achieve
ALL-E: Aesthetics-guided Low-light Image Enhancement
Evaluating the performance of low-light image enhancement (LLE) is highly
subjective, thus making integrating human preferences into image enhancement a
necessity. Existing methods fail to consider this and present a series of
potentially valid heuristic criteria for training enhancement models. In this
paper, we propose a new paradigm, i.e., aesthetics-guided low-light image
enhancement (ALL-E), which introduces aesthetic preferences to LLE and
motivates training in a reinforcement learning framework with an aesthetic
reward. Each pixel, functioning as an agent, refines itself by recursive
actions, i.e., its corresponding adjustment curve is estimated sequentially.
Extensive experiments show that integrating aesthetic assessment improves both
subjective experience and objective evaluation. Our results on various
benchmarks demonstrate the superiority of ALL-E over state-of-the-art methods.
Source code and models are in the project page
A Dataset of Multi-Illumination Images in the Wild
Collections of images under a single, uncontrolled illumination have enabled
the rapid advancement of core computer vision tasks like classification,
detection, and segmentation. But even with modern learning techniques, many
inverse problems involving lighting and material understanding remain too
severely ill-posed to be solved with single-illumination datasets. To fill this
gap, we introduce a new multi-illumination dataset of more than 1000 real
scenes, each captured under 25 lighting conditions. We demonstrate the richness
of this dataset by training state-of-the-art models for three challenging
applications: single-image illumination estimation, image relighting, and
mixed-illuminant white balance.Comment: ICCV 201
Real-time Global Illumination Decomposition of Videos
We propose the first approach for the decomposition of a monocular color
video into direct and indirect illumination components in real time. We
retrieve, in separate layers, the contribution made to the scene appearance by
the scene reflectance, the light sources and the reflections from various
coherent scene regions to one another. Existing techniques that invert global
light transport require image capture under multiplexed controlled lighting, or
only enable the decomposition of a single image at slow off-line frame rates.
In contrast, our approach works for regular videos and produces temporally
coherent decomposition layers at real-time frame rates. At the core of our
approach are several sparsity priors that enable the estimation of the
per-pixel direct and indirect illumination layers based on a small set of
jointly estimated base reflectance colors. The resulting variational
decomposition problem uses a new formulation based on sparse and dense sets of
non-linear equations that we solve efficiently using a novel alternating
data-parallel optimization strategy. We evaluate our approach qualitatively and
quantitatively, and show improvements over the state of the art in this field,
in both quality and runtime. In addition, we demonstrate various real-time
appearance editing applications for videos with consistent illumination
6th International Meeting on Retouching of Cultural Heritage, RECH6
RECH Biennial Meeting is one of the largest educational and scientific events in Retouching field, an ideal venue for conservators and scientists to present their research results about retouching.
The main focus will be to promote the exchange of ideas, concepts, terminology, methods, techniques and materials applied during the retouching process in different areas of conservation: mural painting, easel painting, sculpture, graphic documentation, architecture, plasterwork, photography, contemporary art, among others.
This Meeting aims to address retouching by encouraging papers that contribute to a deeper understanding of this final task of the conservation and restoration intervention.
The main theme embraces the concepts of retouching, the criteria and limits in the retouching process, the bad retouching impact on heritage and their technical and scientific developments.This Meeting will discuss real-life approaches on retouching, focusing on practical solutions and on sharing experiencesColomina Subiela, A.; Doménech García, B.; Bailão, A. (2023). 6th International Meeting on Retouching of Cultural Heritage, RECH6. Editorial Universitat Politècnica de València. https://doi.org/10.4995/RECH6.2021.1601
- …