28 research outputs found
Deep Depth From Focus
Depth from focus (DFF) is one of the classical ill-posed inverse problems in
computer vision. Most approaches recover the depth at each pixel based on the
focal setting which exhibits maximal sharpness. Yet, it is not obvious how to
reliably estimate the sharpness level, particularly in low-textured areas. In
this paper, we propose `Deep Depth From Focus (DDFF)' as the first end-to-end
learning approach to this problem. One of the main challenges we face is the
hunger for data of deep neural networks. In order to obtain a significant
amount of focal stacks with corresponding groundtruth depth, we propose to
leverage a light-field camera with a co-calibrated RGB-D sensor. This allows us
to digitally create focal stacks of varying sizes. Compared to existing
benchmarks our dataset is 25 times larger, enabling the use of machine learning
for this inverse problem. We compare our results with state-of-the-art DFF
methods and we also analyze the effect of several key deep architectural
components. These experiments show that our proposed method `DDFFNet' achieves
state-of-the-art performance in all scenes, reducing depth error by more than
75% compared to the classical DFF methods.Comment: accepted to Asian Conference on Computer Vision (ACCV) 201
Multi-contrast imaging and digital refocusing on a mobile microscope with a domed LED array
We demonstrate the design and application of an add-on device for improving the diagnostic and research capabilities of CellScope--a low-cost, smartphone-based point-of-care microscope. We replace the single LED illumination of the original CellScope with a programmable domed LED array. By leveraging recent advances in computational illumination, this new device enables simultaneous multi-contrast imaging with brightfield, darkfield, and phase imaging modes. Further, we scan through illumination angles to capture lightfield datasets, which can be used to recover 3D intensity and phase images without any hardware changes. This digital refocusing procedure can be used for either 3D imaging or software-only focus correction, reducing the need for precise mechanical focusing during field experiments. All acquisition and processing is performed on the mobile phone and controlled through a smartphone application, making the computational microscope compact and portable. Using multiple samples and different objective magnifications, we demonstrate that the performance of our device is comparable to that of a commercial microscope. This unique device platform extends the field imaging capabilities of CellScope, opening up new clinical and research possibilities
Deep Eyes: Binocular Depth-from-Focus on Focal Stack Pairs
Human visual system relies on both binocular stereo cues and monocular
focusness cues to gain effective 3D perception. In computer vision, the two
problems are traditionally solved in separate tracks. In this paper, we present
a unified learning-based technique that simultaneously uses both types of cues
for depth inference. Specifically, we use a pair of focal stacks as input to
emulate human perception. We first construct a comprehensive focal stack
training dataset synthesized by depth-guided light field rendering. We then
construct three individual networks: a Focus-Net to extract depth from a single
focal stack, a EDoF-Net to obtain the extended depth of field (EDoF) image from
the focal stack, and a Stereo-Net to conduct stereo matching. We show how to
integrate them into a unified BDfF-Net to obtain high-quality depth maps.
Comprehensive experiments show that our approach outperforms the
state-of-the-art in both accuracy and speed and effectively emulates human
vision systems
Unstructured light fields
Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 35-38).We present a system for interactively acquiring and rendering light fields using a hand-held commodity camera. The main challenge we address is assisting a user in achieving good coverage of the 4D domain despite the challenges of hand-held acquisition. We define coverage by bounding reprojection error between viewpoints, which accounts for all 4 dimensions of the light field. We use this criterion together with a recent Simultaneous Localization and Mapping technique to compute a coverage map on the space of viewpoints. We provide users with real-time feedback and direct them toward under-sampled parts of the light field. Our system is lightweight and has allowed us to capture hundreds of light fields. We further present a new rendering algorithm that is tailored to the unstructured yet dense data we capture. Our method can achieve piecewise-bicubic reconstruction using a triangulation of the captured viewpoints and subdivision rules applied to reconstruction weights.by Myers Abraham Davis (Abe Davis).S.M
The Application of Preconditioned Alternating Direction Method of Multipliers in Depth from Focal Stack
Post capture refocusing effect in smartphone cameras is achievable by using
focal stacks. However, the accuracy of this effect is totally dependent on the
combination of the depth layers in the stack. The accuracy of the extended
depth of field effect in this application can be improved significantly by
computing an accurate depth map which has been an open issue for decades. To
tackle this issue, in this paper, a framework is proposed based on
Preconditioned Alternating Direction Method of Multipliers (PADMM) for depth
from the focal stack and synthetic defocus application. In addition to its
ability to provide high structural accuracy and occlusion handling, the
optimization function of the proposed method can, in fact, converge faster and
better than state of the art methods. The evaluation has been done on 21 sets
of focal stacks and the optimization function has been compared against 5 other
methods. Preliminary results indicate that the proposed method has a better
performance in terms of structural accuracy and optimization in comparison to
the current state of the art methods.Comment: 15 pages, 8 figure
The standard plenoptic camera: applications of a geometrical light field model
A thesis submitted to the University of Bedfordshire, in partial fulfilment of the requirements for the degree of Doctor of PhilosophyThe plenoptic camera is an emerging technology in computer vision able to capture
a light field image from a single exposure which allows a computational change of
the perspective view just as the optical focus, known as refocusing. Until now there was
no general method to pinpoint object planes that have been brought to focus or stereo
baselines of perspective views posed by a plenoptic camera.
Previous research has presented simplified ray models to prove the concept of refocusing
and to enhance image and depth map qualities, but lacked promising distance
estimates and an efficient refocusing hardware implementation. In this thesis, a pair of
light rays is treated as a system of linear functions whose solution yields ray intersections
indicating distances to refocused object planes or positions of virtual cameras that project
perspective views. A refocusing image synthesis is derived from the proposed ray model
and further developed to an array of switch-controlled semi-systolic FIR convolution
filters. Their real-time performance is verified through simulation and implementation by
means of an FPGA using VHDL programming.
A series of experiments is carried out with different lenses and focus settings, where
prediction results are compared with those of a real ray simulation tool and processed
light field photographs for which a blur metric has been considered. Predictions accurately
match measurements in light field photographs and signify deviations of less than 0.35 %
in real ray simulation. A benchmark assessment of the proposed refocusing hardware
implementation suggests a computation time speed-up of 99.91 % in comparison with a
state-of-the-art technique.
It is expected that this research supports in the prototyping stage of plenoptic cameras
and microscopes as it helps specifying depth sampling planes, thus localising objects and
provides a power-efficient refocusing hardware design for full-video applications as in
broadcasting or motion picture arts
Consideraciones acerca de la viabilidad de un sensor plenóptico en dispositivos de consumo
Doctorado en Ingeniería IndustrialPassive distance measurement of the objects in an image gives place to interesting applications that have the potential to revolutionize the field of photography. In this thesis a prototype of plenoptic camera for mobile devices was created and studied. This technique has two main disadvantages: the need for modifying the camera module and the loss of resolution. Because of this, the prototype was discarded in order to utilize another technique: depth from focus.
In this technique the capture method consists in taking several images while varying the focus distance. The set of images is called focal-stack. Different focus operators are studied, which give a measure of defocus per pixel and plane of the focal-stack.
The curvelet based focus operator is chosen as the most adequate. It is computationally more intensive than other operators but it is capable of decomposing natural images using few coefficients. In order to make viable its usage in mobile devices a new curvelet transform based on the discrete Radon transform is built. The discrete Radon transform has logarithmic complexity, does not use the Fourier transform and uses only integer sums.
Lastly, different versions of the Radon transform are analyzed with the goal of achieving an even faster transform. These transforms are implemented to be executed on mobile devices.
Additionally, an application of the Radon transform is presented. It consists in the detection of bar-codes that have any orientation in an image.La medida pasiva de distancia a los objetos en una imagen da lugar a interesantes aplicaciones con capacidad para revolucionar la fotografía. En esta tesis se creó y estudió un prototipo de cámara plenóptica para dispositivos móviles. Esta técnica presenta dos inconvenientes: la necesidad de modificar el módulo de cámara y la pérdida de resolución. Por ello, el prototipo fue descartado para utilizar otra técnica: la profundidad a partir del desenfoque.
En esta técnica el método de captura consiste en tomar varias imagenes variando la distancia de enfoque. El conjunto de imágenes se denomina focal-stack. Se estudian distintos operadores de desenfoque, que dan una medida de desenfoque por pixel y por plano del focal-stack. Siendo elegido como óptimo el operador de desenfoque curvelet, que es computacionalmente más intensivo que otros operadores pero es capaz de descomponer imagenes naturales utilizando muy pocos coeficientes.
Para hacer posible su uso en dispositivos móviles se construye una nueva transformada curvelet basada en la transformada discreta de Radon. La transformada discreta de Radon tiene complejidad linearítmica, no utiliza la transformada de Fourier y usa sólo sumas de enteros.
Por último, se analizan distintas versiones de la transformada de Radon con el objetivo de conseguir una transformada aún más rápida y se implementan para ser ejecutadas en dispositivos móviles.
Además se presenta una aplicación de la transformada de Radon consistente en la detección de códigos de barras con cualquier orientación en una imagen
From Calibration to Large-Scale Structure from Motion with Light Fields
Classic pinhole cameras project the multi-dimensional information of the light flowing through a scene onto a single 2D snapshot. This projection limits the information that can be reconstructed from the 2D acquisition. Plenoptic (or light field) cameras, on the other hand, capture a 4D slice of the plenoptic function, termed the “light field”. These cameras provide both spatial and angular information on the light flowing through a scene; multiple views are captured in a single photographic exposure facilitating various applications. This thesis is concerned with the modelling of light field (or plenoptic) cameras and the development of structure from motion pipelines using such cameras. Specifically, we develop a geometric model for a multi-focus plenoptic camera, followed by a complete pipeline for the calibration of the suggested model. Given a calibrated light field camera, we then remap the captured light field to a grid of pinhole images. We use these images to obtain metric 3D reconstruction through a novel framework for structure from motion with light fields. Finally, we suggest a linear and efficient approach for absolute pose estimation for light fields