Low Power Depth Estimation of Rigid Objects for Time-of-Flight Imaging

Depth sensing is useful in a variety of applications that range from augmented reality to robotics. Time-of-flight (TOF) cameras are appealing because they obtain dense depth measurements with minimal latency. However, for many battery-powered devices, the illumination source of a TOF camera is power hungry and can limit the battery life of the device. To address this issue, we present an algorithm that lowers the power for depth sensing by reducing the usage of the TOF camera and estimating depth maps using concurrently collected images. Our technique also adaptively controls the TOF camera and enables it when an accurate depth map cannot be estimated. To ensure that the overall system power for depth sensing is reduced, we design our algorithm to run on a low power embedded platform, where it outputs 640x480 depth maps at 30 frames per second. We evaluate our approach on several RGB-D datasets, where it produces depth maps with an overall mean relative error of 0.96% and reduces the usage of the TOF camera by 85%. When used with commercial TOF cameras, we estimate that our algorithm can lower the total power for depth sensing by up to 73%

Photon-Efficient Computational 3D and Reflectivity Imaging with Single-Photon Detectors

Capturing depth and reflectivity images at low light levels from active illumination of a scene has wide-ranging applications. Conventionally, even with single-photon detectors, hundreds of photon detections are needed at each pixel to mitigate Poisson noise. We develop a robust method for estimating depth and reflectivity using on the order of 1 detected photon per pixel averaged over the scene. Our computational imager combines physically accurate single-photon counting statistics with exploitation of the spatial correlations present in real-world reflectivity and 3D structure. Experiments conducted in the presence of strong background light demonstrate that our computational imager is able to accurately recover scene depth and reflectivity, while traditional maximum-likelihood based imaging methods lead to estimates that are highly noisy. Our framework increases photon efficiency 100-fold over traditional processing and also improves, somewhat, upon first-photon imaging under a total acquisition time constraint in raster-scanned operation. Thus our new imager will be useful for rapid, low-power, and noise-tolerant active optical imaging, and its fixed dwell time will facilitate parallelization through use of a detector array.Comment: 11 pages, 8 figure

Photon-limited time of flight depth acquisition : new parametric model and its analysis

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 83-87).As 3-D imaging systems become more popular, the depth estimation which is their core component should be made as accurate as possible at low power levels. In this thesis, we consider the time of flight depth acquisition problem at low photon counts. We first formulate the received light intensity at the photodetector as the convolution of the source intensity and another signal which we call the scene impulse response. We then present a new framework to calculate the scene impulse response given the scene depth. Using that framework, we propose a richer parametric model than the model being used in existing technologies. We use the maximum likelihood algorithm with our proposed model on simulated data to estimate the depth parameters of the scene. The simulation results show a uniformly better performance for our model relative to the conventional model.by Vahid Montazerhodjat.S.M

Photon Counting and Direct ToF Camera Prototype Based on CMOS SPADs

This paper presents a camera prototype for 2D/3D image capture in low illumination conditions based on single-photon avalanche-diode (SPAD) image sensor for direct time-offlight (d-ToF). The imager is a 64×64 array with in-pixel TDC for high frame rate acquisition. Circuit design techniques are combined to ensure successful 3D image capturing under low sensitivity conditions and high level of uncorrelated noise such as dark count and background illumination. Among them an innovative time gated front-end for the SPAD detector, a reverse start-stop scheme and real-time image reconstruction at Ikfps are incorporated by the imager. To the best of our knowledge, this is the first ToF camera based on a SPAD sensor fabricated and proved for 3D image reconstruction in a standard CMOS process without any opto-flavor or high voltage option. It has a depth resolution of 1cm at an illumination power from less than 6nW/mm 2 down to 0.1nW/mm 2 .Office of Naval Research (USA) N000141410355Ministerio de Economía y Competitividad TEC2015-66878-C3- 1-RJunta de Andalucía P12-TIC 233

Computational multi-depth single-photon imaging

We present an imaging framework that is able to accurately reconstruct multiple depths at individual pixels from single-photon observations. Our active imaging method models the single-photon detection statistics from multiple reflectors within a pixel, and it also exploits the fact that a multi-depth profile at each pixel can be expressed as a sparse signal. We interpret the multi-depth reconstruction problem as a sparse deconvolution problem using single-photon observations, create a convex problem through discretization and relaxation, and use a modified iterative shrinkage-thresholding algorithm to efficiently solve for the optimal multi-depth solution. We experimentally demonstrate that the proposed framework is able to accurately reconstruct the depth features of an object that is behind a partially-reflecting scatterer and 4 m away from the imager with root mean-square error of 11 cm, using only 19 signal photon detections per pixel in the presence of moderate background light. In terms of root mean-square error, this is a factor of 4.2 improvement over the conventional method of Gaussian-mixture fitting for multi-depth recovery.This material is based upon work supported in part by a Samsung Scholarship, the US National Science Foundation under Grant No. 1422034, and the MIT Lincoln Laboratory Advanced Concepts Committee. We thank Dheera Venkatraman for his assistance with the experiments. (Samsung Scholarship; 1422034 - US National Science Foundation; MIT Lincoln Laboratory Advanced Concepts Committee)Accepted manuscrip

A Few Photons Among Many: Unmixing Signal and Noise for Photon-Efficient Active Imaging

Conventional LIDAR systems require hundreds or thousands of photon detections to form accurate depth and reflectivity images. Recent photon-efficient computational imaging methods are remarkably effective with only 1.0 to 3.0 detected photons per pixel, but they are not demonstrated at signal-to-background ratio (SBR) below 1.0 because their imaging accuracies degrade significantly in the presence of high background noise. We introduce a new approach to depth and reflectivity estimation that focuses on unmixing contributions from signal and noise sources. At each pixel in an image, short-duration range gates are adaptively determined and applied to remove detections likely to be due to noise. For pixels with too few detections to perform this censoring accurately, we borrow data from neighboring pixels to improve depth estimates, where the neighborhood formation is also adaptive to scene content. Algorithm performance is demonstrated on experimental data at varying levels of noise. Results show improved performance of both reflectivity and depth estimates over state-of-the-art methods, especially at low signal-to-background ratios. In particular, accurate imaging is demonstrated with SBR as low as 0.04. This validation of a photon-efficient, noise-tolerant method demonstrates the viability of rapid, long-range, and low-power LIDAR imaging

Lidar waveform based analysis of depth images constructed using sparse single-photon data

This paper presents a new Bayesian model and algorithm used for depth and intensity profiling using full waveforms from the time-correlated single photon counting (TCSPC) measurement in the limit of very low photon counts. The model proposed represents each Lidar waveform as a combination of a known impulse response, weighted by the target intensity, and an unknown constant background, corrupted by Poisson noise. Prior knowledge about the problem is embedded in a hierarchical model that describes the dependence structure between the model parameters and their constraints. In particular, a gamma Markov random field (MRF) is used to model the joint distribution of the target intensity, and a second MRF is used to model the distribution of the target depth, which are both expected to exhibit significant spatial correlations. An adaptive Markov chain Monte Carlo algorithm is then proposed to compute the Bayesian estimates of interest and perform Bayesian inference. This algorithm is equipped with a stochastic optimization adaptation mechanism that automatically adjusts the parameters of the MRFs by maximum marginal likelihood estimation. Finally, the benefits of the proposed methodology are demonstrated through a serie of experiments using real data

Single-particle-sensitive imaging of freely propagating ultracold atoms

We present a novel imaging system for ultracold quantum gases in expansion. After release from a confining potential, atoms fall through a sheet of resonant excitation laser light and the emitted fluorescence photons are imaged onto an amplified CCD camera using a high numerical aperture optical system. The imaging system reaches an extraordinary dynamic range, not attainable with conventional absorption imaging. We demonstrate single-atom detection for dilute atomic clouds with high efficiency where at the same time dense Bose-Einstein condensates can be imaged without saturation or distortion. The spatial resolution can reach the sampling limit as given by the 8 \mu m pixel size in object space. Pulsed operation of the detector allows for slice images, a first step toward a 3D tomography of the measured object. The scheme can easily be implemented for any atomic species and all optical components are situated outside the vacuum system. As a first application we perform thermometry on rubidium Bose-Einstein condensates created on an atom chip.Comment: 24 pages, 10 figures. v2: as publishe