Depth Fields: Extending Light Field Techniques to Time-of-Flight Imaging

A variety of techniques such as light field, structured illumination, and time-of-flight (TOF) are commonly used for depth acquisition in consumer imaging, robotics and many other applications. Unfortunately, each technique suffers from its individual limitations preventing robust depth sensing. In this paper, we explore the strengths and weaknesses of combining light field and time-of-flight imaging, particularly the feasibility of an on-chip implementation as a single hybrid depth sensor. We refer to this combination as depth field imaging. Depth fields combine light field advantages such as synthetic aperture refocusing with TOF imaging advantages such as high depth resolution and coded signal processing to resolve multipath interference. We show applications including synthesizing virtual apertures for TOF imaging, improved depth mapping through partial and scattering occluders, and single frequency TOF phase unwrapping. Utilizing space, angle, and temporal coding, depth fields can improve depth sensing in the wild and generate new insights into the dimensions of light's plenoptic function.Comment: 9 pages, 8 figures, Accepted to 3DV 201

Learning sparse representations of depth

This paper introduces a new method for learning and inferring sparse representations of depth (disparity) maps. The proposed algorithm relaxes the usual assumption of the stationary noise model in sparse coding. This enables learning from data corrupted with spatially varying noise or uncertainty, typically obtained by laser range scanners or structured light depth cameras. Sparse representations are learned from the Middlebury database disparity maps and then exploited in a two-layer graphical model for inferring depth from stereo, by including a sparsity prior on the learned features. Since they capture higher-order dependencies in the depth structure, these priors can complement smoothness priors commonly used in depth inference based on Markov Random Field (MRF) models. Inference on the proposed graph is achieved using an alternating iterative optimization technique, where the first layer is solved using an existing MRF-based stereo matching algorithm, then held fixed as the second layer is solved using the proposed non-stationary sparse coding algorithm. This leads to a general method for improving solutions of state of the art MRF-based depth estimation algorithms. Our experimental results first show that depth inference using learned representations leads to state of the art denoising of depth maps obtained from laser range scanners and a time of flight camera. Furthermore, we show that adding sparse priors improves the results of two depth estimation methods: the classical graph cut algorithm by Boykov et al. and the more recent algorithm of Woodford et al.Comment: 12 page

FPGA Based Pattern Generation and Synchonization for High Speed Structured Light 3D Camera

Recently, structured light 3D imaging devices have gained a keen attention due to their potential applications to robotics, industrial manufacturing and medical imaging. Most of these applications require high 3D precision yet high speed in image capturing for hard and/or soft real time environments. This paper presents a method of high speed image capturing for structured light 3D imaging sensors with FPGA based structured light pattern generation and projector-camera synchronization. Suggested setup reduces the time for pattern projection and camera triggering to 16msec from 100msec that should be required by conventional methods

AirCode: Unobtrusive Physical Tags for Digital Fabrication

We present AirCode, a technique that allows the user to tag physically fabricated objects with given information. An AirCode tag consists of a group of carefully designed air pockets placed beneath the object surface. These air pockets are easily produced during the fabrication process of the object, without any additional material or postprocessing. Meanwhile, the air pockets affect only the scattering light transport under the surface, and thus are hard to notice to our naked eyes. But, by using a computational imaging method, the tags become detectable. We present a tool that automates the design of air pockets for the user to encode information. AirCode system also allows the user to retrieve the information from captured images via a robust decoding algorithm. We demonstrate our tagging technique with applications for metadata embedding, robotic grasping, as well as conveying object affordances.Comment: ACM UIST 2017 Technical Paper

Spectral imaging in preclinical research and clinical pathology.

Spectral imaging methods are attracting increased interest from researchers and practitioners in basic science, pre-clinical and clinical arenas. A combination of better labeling reagents and better optics creates opportunities to detect and measure multiple parameters at the molecular and cellular level. These tools can provide valuable insights into the basic mechanisms of life, and yield diagnostic and prognostic information for clinical applications. There are many multispectral technologies available, each with its own advantages and limitations. This chapter will present an overview of the rationale for spectral imaging, and discuss the hardware, software and sample labeling strategies that can optimize its usefulness in clinical settings

Video-rate volumetric neuronal imaging using 3D targeted illumination

Fast volumetric microscopy is required to monitor large-scale neural ensembles with high spatio-temporal resolution. Widefield fluorescence microscopy can image large 2D fields of view at high resolution and speed while remaining simple and costeffective. A focal sweep add-on can further extend the capacity of widefield microscopy by enabling extended-depth-of-field (EDOF) imaging, but suffers from an inability to reject out-of-focus fluorescence background. Here, by using a digital micromirror device to target only in-focus sample features, we perform EDOF imaging with greatly enhanced contrast and signal-to-noise ratio, while reducing the light dosage delivered to the sample. Image quality is further improved by the application of a robust deconvolution algorithm. We demonstrate the advantages of our technique for in vivo calcium imaging in the mouse brain.This work was funded by the National Institutes of Health (R21EY026310) and the National Science Foundation (CBET-1508988). The authors wish to thank E. McCarthy and Prof. M.J. Baum for providing mouse brain slices used in this manuscript, and A. I. Mohammed for providing in vivo mouse brain samples in the early stages of this work. (R21EY026310 - National Institutes of Health; CBET-1508988 - National Science Foundation)Published versio