Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles

Time of flight cameras produce real-time range maps at a relatively low cost using continuous wave amplitude modulation and demodulation. However, they are geared to measure range (or phase) for a single reflected bounce of light and suffer from systematic errors due to multipath interference. We re-purpose the conventional time of flight device for a new goal: to recover per-pixel sparse time profiles expressed as a sequence of impulses. With this modification, we show that we can not only address multipath interference but also enable new applications such as recovering depth of near-transparent surfaces, looking through diffusers and creating time-profile movies of sweeping light. Our key idea is to formulate the forward amplitude modulated light propagation as a convolution with custom codes, record samples by introducing a simple sequence of electronic time delays, and perform sparse deconvolution to recover sequences of Diracs that correspond to multipath returns. Applications to computer vision include ranging of near-transparent objects and subsurface imaging through diffusers. Our low cost prototype may lead to new insights regarding forward and inverse problems in light transport.United States. Defense Advanced Research Projects Agency (DARPA Young Faculty Award)Alfred P. Sloan Foundation (Fellowship)Massachusetts Institute of Technology. Media Laboratory. Camera Culture Grou

Recent advances in transient imaging: A computer graphics and vision perspective

Transient imaging has recently made a huge impact in the computer graphics and computer vision fields. By capturing, reconstructing, or simulating light transport at extreme temporal resolutions, researchers have proposed novel techniques to show movies of light in motion, see around corners, detect objects in highly-scattering media, or infer material properties from a distance, to name a few. The key idea is to leverage the wealth of information in the temporal domain at the pico or nanosecond resolution, information usually lost during the capture-time temporal integration. This paper presents recent advances in this field of transient imaging from a graphics and vision perspective, including capture techniques, analysis, applications and simulation

Recent advances in transient imaging: A computer graphics and vision perspective

Transient imaging has recently made a huge impact in the computer graphics and computer vision fields. By capturing, reconstructing, or simulating light transport at extreme temporal resolutions, researchers have proposed novel techniques to show movies of light in motion, see around corners, detect objects in highly-scattering media, or infer material properties from a distance, to name a few. The key idea is to leverage the wealth of information in the temporal domain at the pico or nanosecond resolution, information usually lost during the capture-time temporal integration. This paper presents recent advances in this field of transient imaging from a graphics and vision perspective, including capture techniques, analysis, applications and simulation

Phasor Imaging: A Generalization of Correlation-Based Time-of-Flight Imaging

In correlation-based time-of-flight (C-ToF) imaging systems, light sources with temporally varying intensities illuminate the scene. Due to global illumination, the temporally varying radiance received at the sensor is a combination of light received along multiple paths. Recovering scene properties (e.g., scene depths) from the received radiance requires separating these contributions, which is challenging due to the complexity of global illumination and the additional temporal dimension of the radiance. We propose phasor imaging, a framework for performing fast inverse light transport analysis using C-ToF sensors. Phasor imaging is based on the idea that by representing light transport quantities as phasors and light transport events as phasor transformations, light transport analysis can be simplified in the temporal frequency domain. We study the effect of temporal illumination frequencies on light transport, and show that for a broad range of scenes, global radiance (multi-path interference) vanishes for frequencies higher than a scene-dependent threshold. We use this observation for developing two novel scene recovery techniques. First, we present Micro ToF imaging, a ToF based shape recovery technique that is robust to errors due to global illumination. Second, we present a technique for separating the direct and global components of radiance. Both techniques require capturing as few as 3−4 images and minimal computations. We demonstrate the validity of the presented techniques via simulations and experiments performed with our hardware prototype

State-of-the-art active optical techniques for three-dimensional surface metrology: a review [Invited]

This paper reviews recent developments of non-contact three-dimensional (3D) surface metrology using an active structured optical probe. We focus primarily on those active non-contact 3D surface measurement techniques that could be applicable to the manufacturing industry. We discuss principles of each technology, and its advantageous characteristics as well as limitations. Towards the end, we discuss our perspectives on the current technological challenges in designing and implementing these methods in practical applications.Purdue Universit

Developing A Toolbox To Probe Reaction Dynamics With Strong Field Ionization And Non-Linear Attosecond Spectroscopy

Electronic motions which happen in 10 to 100 of attoseconds are the heart of all processes in nature. Therefore monitoring and extracting details in this fundamental level will provide new prospect to the areas as information technology, basic energy science, medicine and life sciences. The challenge being, develop a tool to reach such a fast time scale for real time observation in atomic level. In this thesis work we have address this matter using two interesting approaches related to the laser matter interaction: strong field ionization and nonlinear attosecond spectroscopy. The first part is based on the studies related to the strong field ionization probe. Strong field ionization probe was verified to be sensitive to the sign of magnetic quantum number which evident the capability of probing atomic orientation. The next part is based on non-linear attosecond spectroscopy. With the use 1 kHz laser and the loose focusing geometry we were able to produce attosecond pulse trains with a sufficient flux to perform two photon double ionization. Further, we were also able to extract ion-electron coincidence measurements of the double ionization event of XUV-pump-XUV-probe system for the first time. The extended studies will be carried out with the combination of our newly developed 3D detector to this current setup which will facilitate the triple coincidence capabilities

Atomic and Molecular Dynamics Probed by Intense Extreme Ultraviolet Attosecond Pulses

This thesis work was aimed to investigate dynamical processes in atoms and molecules on ultrafast time scales initiated by absorption of light in the extreme ultraviolet (XUV) regime. In particular, photoionization and photodissociation have been studied using pump-probe techniques involving ultrafast laser pulses. Such pulses are generated using either high-order harmonic generation (HHG) or free-electron lasers (FELs).The work of this thesis consists to a large extent in the development and application of a light source, enabling intense XUV attosecond pulses using HHG. In a long focusing geometry, a high-power infrared laser is frequency up-converted so as to generate a comb of high-order harmonics. An important aspect was the study of the spatial and temporal properties of the generated light pulses in order to gain control of their influence on the experiment. Combining theoretical and experimental results, the effect of the dipole phase on properties of high-order harmonics was explored, along with a metrological series of studies on the harmonic wavefront and the properties of the focusing optics used. Further, the HHG light source was employed to investigate photoionization. Individual angular momentum channels involved in the ionization were characterized using two-photon interferometry in combination with angle-resolved photoelectron detection. A method is applied allowing the full determination of channel-resolved amplitudes and phases of the matrix elements describing the single-photon ionization of neon.Finally, the process of photodissociation was investigated using light pulses generated via both HHG and FELs. The dissociation dynamics induced by multiple ionization of organic molecules were studied. Correlation techniques were used to unravel the underlying fragmentation dynamics, and additionally, pump-probe experiments provided insights into the time scales of the (pre-)dissociation dynamics

Resolving Measurement Errors Inherent with Time-of-Flight Range Imaging Cameras

Range imaging cameras measure the distance to objects in the field-of-view (FoV) of the camera, these cameras enable new machine vision applications in robotics, manufacturing, and human computer interaction. Time-of-flight (ToF) range cameras operate by illuminating the scene with amplitude modulated continuous wave (AMCW) light and measuring the phase difference between the emitted and reflected modulation envelope. Currently ToF range cameras suffer from measurement errors that are highly scene dependent, and these errors limit the accuracy of the depth measurement. The major cause of measurement errors is multiple propagation paths from the light source to pixel, known as multi path interference. Multi-path interference typically arises from: inter reflections, lens flare, subsurface scattering, volumetric scattering, and translucent objects. This thesis contributes three novel methods for resolving multi-path interference: coding in time, coding in frequency, and coding in space. Time coding is implemented by replacing the single frequency amplitude modulation with a binary sequence. Fundamental to ToF range cameras is the cross-correlation between the reflected light and a reference signal. The measured cross-correlation depends on the selection of the binary sequence. With selection of an appropriate binary sequence and using sparse deconvolution on the measured cross-correlation the multiple return path lengths and their amplitudes can be recovered. However, the minimal resolvable path length is dependent on the highest frequency in the binary sequence. Frequency coding is implemented by taking multiple measurements at different modulation frequencies. A subset of frequency coding is operating the camera in a mode analogous to stepped frequency continuous wave (SFCW). Frequency coding uses techniques from radar to resolve multiple propagation paths. The minimal resolvable path length is dependent on the camera's modulation bandwidth and the spectrum estimation technique used to recover distance, and it is shown that SFCW can be used to measure depth of objects behind a translucent sheet, while AMCW measurements can not. Path lengths below quarter a wavelength of the highest modulation frequency are difficult to resolve. The use of spatial coding is used to resolve diffuse multi-path interference. The original technique comes from direct and global separation in computer graphics, and it is modified to operate on the complex data produced by a ToF range camera. By illuminating the scene with a pattern the illuminated areas contain the direct return and the scattering (global return). The non-illuminated regions contain the scattering return, assuming the global component is spatially smooth. The direct and global separation with sinusoidal patterns is combining with the sinusoidal modulation signal of ToF range cameras for a closed form solution to multi-path interference in nine frames. With nine raw frames it is possible to implement direct and global separation at video frame rates. The RMSE of a corner is reduced from 0.0952 m to 0.0112 m. Direct and global separation correctly measures the depth of a diffuse corner, and resolves subsurface scattering however fails to resolve specular reflections. Finally the direct and global separation is combined with replacing the illumination and reference signals with a binary sequence. The combination allows for resolving diffuse multi-path interference present in a corner, with the sparse multi-path interference caused mixed pixels between the foreground and background. The corner is correctly measured and the number of mixed pixels is reduced by 90%. With the development of new methods to resolve multi-path interference ToF range cameras can measure scenes with more confidence. ToF range cameras can be built into small form factors as they require a small number of parts: a pixel array, a light source and a lens. The small form factor coupled with accurate range measurements allows ToF range cameras to be embedded in cellphones and consumer electronic devices, enabling wider adoption and advantages over competing range imaging technologies