469 research outputs found
Shadow Estimation Method for "The Episolar Constraint: Monocular Shape from Shadow Correspondence"
Recovering shadows is an important step for many vision algorithms. Current
approaches that work with time-lapse sequences are limited to simple
thresholding heuristics. We show these approaches only work with very careful
tuning of parameters, and do not work well for long-term time-lapse sequences
taken over the span of many months. We introduce a parameter-free expectation
maximization approach which simultaneously estimates shadows, albedo, surface
normals, and skylight. This approach is more accurate than previous methods,
works over both very short and very long sequences, and is robust to the
effects of nonlinear camera response. Finally, we demonstrate that the shadow
masks derived through this algorithm substantially improve the performance of
sun-based photometric stereo compared to earlier shadow mask estimation
PS-FCN: A Flexible Learning Framework for Photometric Stereo
This paper addresses the problem of photometric stereo for non-Lambertian
surfaces. Existing approaches often adopt simplified reflectance models to make
the problem more tractable, but this greatly hinders their applications on
real-world objects. In this paper, we propose a deep fully convolutional
network, called PS-FCN, that takes an arbitrary number of images of a static
object captured under different light directions with a fixed camera as input,
and predicts a normal map of the object in a fast feed-forward pass. Unlike the
recently proposed learning based method, PS-FCN does not require a pre-defined
set of light directions during training and testing, and can handle multiple
images and light directions in an order-agnostic manner. Although we train
PS-FCN on synthetic data, it can generalize well on real datasets. We further
show that PS-FCN can be easily extended to handle the problem of uncalibrated
photometric stereo.Extensive experiments on public real datasets show that
PS-FCN outperforms existing approaches in calibrated photometric stereo, and
promising results are achieved in uncalibrated scenario, clearly demonstrating
its effectiveness.Comment: ECCV 2018: https://guanyingc.github.io/PS-FC
EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras
Event-based cameras have shown great promise in a variety of situations where
frame based cameras suffer, such as high speed motions and high dynamic range
scenes. However, developing algorithms for event measurements requires a new
class of hand crafted algorithms. Deep learning has shown great success in
providing model free solutions to many problems in the vision community, but
existing networks have been developed with frame based images in mind, and
there does not exist the wealth of labeled data for events as there does for
images for supervised training. To these points, we present EV-FlowNet, a novel
self-supervised deep learning pipeline for optical flow estimation for event
based cameras. In particular, we introduce an image based representation of a
given event stream, which is fed into a self-supervised neural network as the
sole input. The corresponding grayscale images captured from the same camera at
the same time as the events are then used as a supervisory signal to provide a
loss function at training time, given the estimated flow from the network. We
show that the resulting network is able to accurately predict optical flow from
events only in a variety of different scenes, with performance competitive to
image based networks. This method not only allows for accurate estimation of
dense optical flow, but also provides a framework for the transfer of other
self-supervised methods to the event-based domain.Comment: 9 pages, 5 figures, 1 table. Accompanying video:
https://youtu.be/eMHZBSoq0sE. Dataset:
https://daniilidis-group.github.io/mvsec/, Robotics: Science and Systems 201
TAPA-MVS: Textureless-Aware PAtchMatch Multi-View Stereo
One of the most successful approaches in Multi-View Stereo estimates a depth
map and a normal map for each view via PatchMatch-based optimization and fuses
them into a consistent 3D points cloud. This approach relies on
photo-consistency to evaluate the goodness of a depth estimate. It generally
produces very accurate results; however, the reconstructed model often lacks
completeness, especially in correspondence of broad untextured areas where the
photo-consistency metrics are unreliable. Assuming the untextured areas
piecewise planar, in this paper we generate novel PatchMatch hypotheses so to
expand reliable depth estimates in neighboring untextured regions. At the same
time, we modify the photo-consistency measure such to favor standard or novel
PatchMatch depth hypotheses depending on the textureness of the considered
area. We also propose a depth refinement step to filter wrong estimates and to
fill the gaps on both the depth maps and normal maps while preserving the
discontinuities. The effectiveness of our new methods has been tested against
several state of the art algorithms in the publicly available ETH3D dataset
containing a wide variety of high and low-resolution images
Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation
We address the unsupervised learning of several interconnected problems in
low-level vision: single view depth prediction, camera motion estimation,
optical flow, and segmentation of a video into the static scene and moving
regions. Our key insight is that these four fundamental vision problems are
coupled through geometric constraints. Consequently, learning to solve them
together simplifies the problem because the solutions can reinforce each other.
We go beyond previous work by exploiting geometry more explicitly and
segmenting the scene into static and moving regions. To that end, we introduce
Competitive Collaboration, a framework that facilitates the coordinated
training of multiple specialized neural networks to solve complex problems.
Competitive Collaboration works much like expectation-maximization, but with
neural networks that act as both competitors to explain pixels that correspond
to static or moving regions, and as collaborators through a moderator that
assigns pixels to be either static or independently moving. Our novel method
integrates all these problems in a common framework and simultaneously reasons
about the segmentation of the scene into moving objects and the static
background, the camera motion, depth of the static scene structure, and the
optical flow of moving objects. Our model is trained without any supervision
and achieves state-of-the-art performance among joint unsupervised methods on
all sub-problems.Comment: CVPR 201
- …