122 research outputs found
Revealing hidden scenes by photon-efficient occlusion-based opportunistic active imaging
The ability to see around corners, i.e., recover details of a hidden scene
from its reflections in the surrounding environment, is of considerable
interest in a wide range of applications. However, the diffuse nature of light
reflected from typical surfaces leads to mixing of spatial information in the
collected light, precluding useful scene reconstruction. Here, we employ a
computational imaging technique that opportunistically exploits the presence of
occluding objects, which obstruct probe-light propagation in the hidden scene,
to undo the mixing and greatly improve scene recovery. Importantly, our
technique obviates the need for the ultrafast time-of-flight measurements
employed by most previous approaches to hidden-scene imaging. Moreover, it does
so in a photon-efficient manner based on an accurate forward model and a
computational algorithm that, together, respect the physics of three-bounce
light propagation and single-photon detection. Using our methodology, we
demonstrate reconstruction of hidden-surface reflectivity patterns in a
meter-scale environment from non-time-resolved measurements. Ultimately, our
technique represents an instance of a rich and promising new imaging modality
with important potential implications for imaging science.Comment: Related theory in arXiv:1711.0629
Eclipse: Disambiguating Illumination and Materials using Unintended Shadows
Decomposing an object's appearance into representations of its materials and
the surrounding illumination is difficult, even when the object's 3D shape is
known beforehand. This problem is ill-conditioned because diffuse materials
severely blur incoming light, and is ill-posed because diffuse materials under
high-frequency lighting can be indistinguishable from shiny materials under
low-frequency lighting. We show that it is possible to recover precise
materials and illumination -- even from diffuse objects -- by exploiting
unintended shadows, like the ones cast onto an object by the photographer who
moves around it. These shadows are a nuisance in most previous inverse
rendering pipelines, but here we exploit them as signals that improve
conditioning and help resolve material-lighting ambiguities. We present a
method based on differentiable Monte Carlo ray tracing that uses images of an
object to jointly recover its spatially-varying materials, the surrounding
illumination environment, and the shapes of the unseen light occluders who
inadvertently cast shadows upon it.Comment: Project page: https://dorverbin.github.io/eclipse
Can Shadows Reveal Biometric Information?
We study the problem of extracting biometric information of individuals by
looking at shadows of objects cast on diffuse surfaces. We show that the
biometric information leakage from shadows can be sufficient for reliable
identity inference under representative scenarios via a maximum likelihood
analysis. We then develop a learning-based method that demonstrates this
phenomenon in real settings, exploiting the subtle cues in the shadows that are
the source of the leakage without requiring any labeled real data. In
particular, our approach relies on building synthetic scenes composed of 3D
face models obtained from a single photograph of each identity. We transfer
what we learn from the synthetic data to the real data using domain adaptation
in a completely unsupervised way. Our model is able to generalize well to the
real domain and is robust to several variations in the scenes. We report high
classification accuracies in an identity classification task that takes place
in a scene with unknown geometry and occluding objects
Occlusion-based computational periscopy with consumer cameras
The ability to form images of scenes hidden from direct view would be advantageous in many applications – from improved motion planning and collision avoidance in autonomous navigation to enhanced danger anticipation for first-responders in search-and-rescue missions. Recent techniques for imaging around corners have mostly relied on time-of-flight measurements of light propagation, necessitating the use of expensive, specialized optical systems. In this work, we demonstrate how to form images of hidden scenes from intensity-only measurements of the light reaching a visible surface from the hidden scene. Our approach exploits the penumbra cast by an opaque occluding object onto a visible surface. Specifically, we present a physical model that relates the measured photograph to the radiosity of the hidden scene and the visibility function due to the opaque occluder. For a given scene–occluder setup, we characterize the parts of the hidden region for which the physical model is well-conditioned for inversion – i.e., the computational field of view (CFOV) of the imaging system. This concept of CFOV is further verified through the Cram´er–Rao bound of the hidden-scene estimation problem. Finally, we present a two-step computational method for recovering the occluder and the scene behind it. We demonstrate the effectiveness of the proposed method using both synthetic and experimentally measured data.Accepted manuscrip
ORCa: Glossy Objects as Radiance Field Cameras
Reflections on glossy objects contain valuable and hidden information about
the surrounding environment. By converting these objects into cameras, we can
unlock exciting applications, including imaging beyond the camera's
field-of-view and from seemingly impossible vantage points, e.g. from
reflections on the human eye. However, this task is challenging because
reflections depend jointly on object geometry, material properties, the 3D
environment, and the observer viewing direction. Our approach converts glossy
objects with unknown geometry into radiance-field cameras to image the world
from the object's perspective. Our key insight is to convert the object surface
into a virtual sensor that captures cast reflections as a 2D projection of the
5D environment radiance field visible to the object. We show that recovering
the environment radiance fields enables depth and radiance estimation from the
object to its surroundings in addition to beyond field-of-view novel-view
synthesis, i.e. rendering of novel views that are only directly-visible to the
glossy object present in the scene, but not the observer. Moreover, using the
radiance field we can image around occluders caused by close-by objects in the
scene. Our method is trained end-to-end on multi-view images of the object and
jointly estimates object geometry, diffuse radiance, and the 5D environment
radiance field.Comment: for more information, see https://ktiwary2.github.io/objectsascam
Computational periscopy with an ordinary digital camera
Computing the amounts of light arriving from different directions enables a diffusely reflecting surface to play the part of a mirror in a periscope—that is, perform non-line-of-sight imaging around an obstruction. Because computational periscopy has so far depended on light-travel distances being proportional to the times of flight, it has mostly been performed with expensive, specialized ultrafast optical systems^1,2,3,4,5,6,7,8,9,10,11,12. Here we introduce a two-dimensional computational periscopy technique that requires only a single photograph captured with an ordinary digital camera. Our technique recovers the position of an opaque object and the scene behind (but not completely obscured by) the object, when both the object and scene are outside the line of sight of the camera, without requiring controlled or time-varying illumination. Such recovery is based on the visible penumbra of the opaque object having a linear dependence on the hidden scene that can be modelled through ray optics. Non-line-of-sight imaging using inexpensive, ubiquitous equipment may have considerable value in monitoring hazardous environments, navigation and detecting hidden adversaries.We thank F. Durand, W. T. Freeman, Y. Ma, J. Rapp, J. H. Shapiro, A. Torralba, F. N. C. Wong and G. W. Wornell for discussions. This work was supported by the Defense Advanced Research Projects Agency (DARPA) REVEAL Program contract number HR0011-16-C-0030. (HR0011-16-C-0030 - Defense Advanced Research Projects Agency (DARPA) REVEAL Program)Accepted manuscrip
Accidental Pinhole and Pinspeck Cameras
We identify and study two types of “accidental” images that can be formed in scenes. The first is an accidental pinhole camera image. The second class of accidental images are “inverse” pinhole camera images, formed by subtracting an image with a small occluder present from a reference image without the occluder. Both types of accidental cameras happen in a variety of different situations. For example, an indoor scene illuminated by natural light, a street with a person walking under the shadow of a building, etc. The images produced by accidental cameras are often mistaken for shadows or interreflections. However, accidental images can reveal information about the scene outside the image, the lighting conditions, or the aperture by which light enters the scene.National Science Foundation (U.S.) (CAREER Award 0747120)United States. Office of Naval Research. Multidisciplinary University Research Initiative (N000141010933)National Science Foundation (U.S.) (CGV 1111415)National Science Foundation (U.S.) (CGV 0964004
Computational Mirrors: Blind Inverse Light Transport by Deep Matrix Factorization
We recover a video of the motion taking place in a hidden scene by observing
changes in indirect illumination in a nearby uncalibrated visible region. We
solve this problem by factoring the observed video into a matrix product
between the unknown hidden scene video and an unknown light transport matrix.
This task is extremely ill-posed, as any non-negative factorization will
satisfy the data. Inspired by recent work on the Deep Image Prior, we
parameterize the factor matrices using randomly initialized convolutional
neural networks trained in a one-off manner, and show that this results in
decompositions that reflect the true motion in the hidden scene.Comment: 14 pages, 5 figures, Advances in Neural Information Processing
Systems 201
- …