15 research outputs found
3D Face Reconstruction from Light Field Images: A Model-free Approach
Reconstructing 3D facial geometry from a single RGB image has recently
instigated wide research interest. However, it is still an ill-posed problem
and most methods rely on prior models hence undermining the accuracy of the
recovered 3D faces. In this paper, we exploit the Epipolar Plane Images (EPI)
obtained from light field cameras and learn CNN models that recover horizontal
and vertical 3D facial curves from the respective horizontal and vertical EPIs.
Our 3D face reconstruction network (FaceLFnet) comprises a densely connected
architecture to learn accurate 3D facial curves from low resolution EPIs. To
train the proposed FaceLFnets from scratch, we synthesize photo-realistic light
field images from 3D facial scans. The curve by curve 3D face estimation
approach allows the networks to learn from only 14K images of 80 identities,
which still comprises over 11 Million EPIs/curves. The estimated facial curves
are merged into a single pointcloud to which a surface is fitted to get the
final 3D face. Our method is model-free, requires only a few training samples
to learn FaceLFnet and can reconstruct 3D faces with high accuracy from single
light field images under varying poses, expressions and lighting conditions.
Comparison on the BU-3DFE and BU-4DFE datasets show that our method reduces
reconstruction errors by over 20% compared to recent state of the art
Light Field Saliency Detection with Deep Convolutional Networks
Light field imaging presents an attractive alternative to RGB imaging because
of the recording of the direction of the incoming light. The detection of
salient regions in a light field image benefits from the additional modeling of
angular patterns. For RGB imaging, methods using CNNs have achieved excellent
results on a range of tasks, including saliency detection. However, it is not
trivial to use CNN-based methods for saliency detection on light field images
because these methods are not specifically designed for processing light field
inputs. In addition, current light field datasets are not sufficiently large to
train CNNs. To overcome these issues, we present a new Lytro Illum dataset,
which contains 640 light fields and their corresponding ground-truth saliency
maps. Compared to current light field saliency datasets [1], [2], our new
dataset is larger, of higher quality, contains more variation and more types of
light field inputs. This makes our dataset suitable for training deeper
networks and benchmarking. Furthermore, we propose a novel end-to-end CNN-based
framework for light field saliency detection. Specifically, we propose three
novel MAC (Model Angular Changes) blocks to process light field micro-lens
images. We systematically study the impact of different architecture variants
and compare light field saliency with regular 2D saliency. Our extensive
comparisons indicate that our novel network significantly outperforms
state-of-the-art methods on the proposed dataset and has desired generalization
abilities on other existing datasets.Comment: 14 pages, 14 figure