High-speed object detection with a single-photon time-of-flight image sensor

3D time-of-flight (ToF) imaging is used in a variety of applications such as augmented reality (AR), computer interfaces, robotics and autonomous systems. Single-photon avalanche diodes (SPADs) are one of the enabling technologies providing accurate depth data even over long ranges. By developing SPADs in array format with integrated processing combined with pulsed, flood-type illumination, high-speed 3D capture is possible. However, array sizes tend to be relatively small, limiting the lateral resolution of the resulting depth maps, and, consequently, the information that can be extracted from the image for applications such as object detection. In this paper, we demonstrate that these limitations can be overcome through the use of convolutional neural networks (CNNs) for high-performance object detection. We present outdoor results from a portable SPAD camera system that outputs 16-bin photon timing histograms with 64x32 spatial resolution. The results, obtained with exposure times down to 2 ms (equivalent to 500 FPS) and in signal-to-background (SBR) ratios as low as 0.05, point to the advantages of providing the CNN with full histogram data rather than point clouds alone. Alternatively, a combination of point cloud and active intensity data may be used as input, for a similar level of performance. In either case, the GPU-accelerated processing time is less than 1 ms per frame, leading to an overall latency (image acquisition plus processing) in the millisecond range, making the results relevant for safety-critical computer vision applications which would benefit from faster than human reaction times.Comment: 13 pages, 5 figures, 3 table

Revealing hidden scenes by photon-efficient occlusion-based opportunistic active imaging

The ability to see around corners, i.e., recover details of a hidden scene from its reflections in the surrounding environment, is of considerable interest in a wide range of applications. However, the diffuse nature of light reflected from typical surfaces leads to mixing of spatial information in the collected light, precluding useful scene reconstruction. Here, we employ a computational imaging technique that opportunistically exploits the presence of occluding objects, which obstruct probe-light propagation in the hidden scene, to undo the mixing and greatly improve scene recovery. Importantly, our technique obviates the need for the ultrafast time-of-flight measurements employed by most previous approaches to hidden-scene imaging. Moreover, it does so in a photon-efficient manner based on an accurate forward model and a computational algorithm that, together, respect the physics of three-bounce light propagation and single-photon detection. Using our methodology, we demonstrate reconstruction of hidden-surface reflectivity patterns in a meter-scale environment from non-time-resolved measurements. Ultimately, our technique represents an instance of a rich and promising new imaging modality with important potential implications for imaging science.Comment: Related theory in arXiv:1711.0629

Real-time spatial modeling to detect and track resources on construction sites

For more than 10 years the U.S. construction industry has experienced over 1,000 fatalities annually. Many fatalities may have been prevented had the individuals and equipment involved been more aware of and alert to the physical state of the environment around them. Awareness may be improved by automatic 3D (three-dimensional) sensing and modeling of the job site environment in real-time. Existing 3D modeling approaches based on range scanning techniques are capable of modeling static objects only, and thus cannot model in real-time dynamic objects in an environment comprised of moving humans, equipment, and materials. Emerging prototype 3D video range cameras offer another alternative by facilitating affordable, wide field of view, automated static and dynamic object detection and tracking at frame rates better than 1Hz (real-time). This dissertation presents an imperical work and methodology to rapidly create a spatial model of construction sites and in particular to detect, model, and track the position, dimension, direction, and velocity of static and moving project resources in real-time, based on range data obtained from a three-dimensional video range camera in a static or moving position. Existing construction site 3D modeling approaches based on optical range sensing technologies (laser scanners, rangefinders, etc.) and 3D modeling approaches (dense, sparse, etc.) that offered potential solutions for this research are reviewed. The choice of an emerging sensing tool and preliminary experiments with this prototype sensing technology are discussed. These findings led to the development of a range data processing algorithm based on three-dimensional occupancy grids which is demonstrated in detail. Testing and validation of the proposed algorithms have been conducted to quantify the performance of sensor and algorithm through extensive experimentation involving static and moving objects. Experiments in indoor laboratory and outdoor construction environments have been conducted with construction resources such as humans, equipment, materials, or structures to verify the accuracy of the occupancy grid modeling approach. Results show that modeling objects and measuring their position, dimension, direction, and speed had an accuracy level compatible to the requirements of active safety features for construction. Results demonstrate that video rate 3D data acquisition and analysis of construction environments can support effective detection, tracking, and convex hull modeling of objects. Exploiting rapidly generated three-dimensional models for improved visualization, communications, and process control has inherent value, broad application, and potential impact, e.g. as-built vs. as-planned comparison, condition assessment, maintenance, operations, and construction activities control. In combination with effective management practices, this sensing approach has the potential to assist equipment operators to avoid incidents that result in reduce human injury, death, or collateral damage on construction sites.Civil, Architectural, and Environmental Engineerin

Optical cooling and trapping of highly magnetic atoms: The benefits of a spontaneous spin polarization

From the study of long-range-interacting systems to the simulation of gauge fields, open-shell Lanthanide atoms with their large magnetic moment and narrow optical transitions open novel directions in the field of ultracold quantum gases. As for other atomic species, the magneto-optical trap (MOT) is the working horse of experiments but its operation is challenging, due to the large electronic spin of the atoms. Here we present an experimental study of narrow-line Dysprosium MOTs. We show that the combination of radiation pressure and gravitational forces leads to a spontaneous polarization of the electronic spin. The spin composition is measured using a Stern-Gerlach separation of spin levels, revealing that the gas becomes almost fully spin-polarized for large laser frequency detunings. In this regime, we reach the optimal operation of the MOT, with samples of typically $3\times 10^8$ atoms at a temperature of 15\,$\mu$K. The spin polarization reduces the complexity of the radiative cooling description, which allows for a simple model accounting for our measurements. We also measure the rate of density-dependent atom losses, finding good agreement with a model based on light-induced Van der Waals forces. A minimal two-body loss rate $\beta\sim 2\times10^{-11}\,$cm$^{3}$/s is reached in the spin-polarized regime. Our results constitute a benchmark for the experimental study of ultracold gases of magnetic Lanthanide atoms.Comment: 21 pages, 9 figure

Highly precise AMCW time-of-flight scanning sensor based on digital-parallel demodulation

In this paper, a novel amplitude-modulated continuous wave (AMCW) time-of-flight (ToF) scanning sensor based on digital-parallel demodulation is proposed and demonstrated in the aspect of distance measurement precision. Since digital-parallel demodulation utilizes a high-amplitude demodulation signal with zero-offset, the proposed sensor platform can maintain extremely high demodulation contrast. Meanwhile, as all cross correlated samples are calculated in parallel and in extremely short integration time, the proposed sensor platform can utilize a 2D laser scanning structure with a single photo detector, maintaining a moderate frame rate. This optical structure can increase the received optical SNR and remove the crosstalk of image pixel array. Based on these measurement properties, the proposed AMCW ToF scanning sensor shows highly precise 3D depth measurement performance. In this study, this precise measurement performance is explained in detail. Additionally, the actual measurement performance of the proposed sensor platform is experimentally validated under various conditions

Analysis, Modeling and Dynamic Optimization of 3D Time-of-Flight Imaging Systems

The present thesis is concerned with the optimization of 3D Time-of-Flight (ToF) imaging systems. These novel cameras determine range images by actively illuminating a scene and measuring the time until the backscattered light is detected. Depth maps are constructed from multiple raw images. Usually two of such raw images are acquired simultaneously using special correlating sensors. This thesis covers four main contributions: A physical sensor model is presented which enables the analysis and optimization of the process of raw image acquisition. This model supports the proposal of a new ToF sensor design which employs a logarithmic photo response. Due to asymmetries of the two read-out paths current systems need to acquire the raw images in multiple instances. This allows the correction of systematic errors. The present thesis proposes a method for dynamic calibration and compensation of these asymmetries. It facilitates the computation of two depth maps from a single set of raw images and thus increases the frame rate by a factor of two. Since not all required raw images are captured simultaneously motion artifacts can occur. The present thesis proposes a robust method for detection and correction of such artifacts. All proposed algorithms have a computational complexity which allowsreal-time execution even on systems with limited resources (e.g. embeddedsystems). The algorithms are demonstrated by use of a commercial ToF camera

A Review of Spatter in Laser Powder Bed Fusion Additive Manufacturing: In Situ Detection, Generation, Effects, and Countermeasures

Spatter is an inherent, unpreventable, and undesired phenomenon in laser powder bed fusion (L-PBF) additive manufacturing. Spatter behavior has an intrinsic correlation with the forming quality in L-PBF because it leads to metallurgical defects and the degradation of mechanical properties. This impact becomes more severe in the fabrication of large-sized parts during the multi-laser L-PBF process. Therefore, investigations of spatter generation and countermeasures have become more urgent. Although much research has provided insights into the melt pool, microstructure, and mechanical property, reviews of spatter in L-PBF are still limited. This work reviews the literature on the in situ detection, generation, effects, and countermeasures of spatter in L-PBF. It is expected to pave the way towards a novel generation of highly efficient and intelligent L-PBF systems

NASA Tech Briefs, February 2010

Topics covered include: Insulation-Testing Cryostat With Lifting Mechanism; Optical Testing of Retroreflectors for Cryogenic Applications; Measuring Cyclic Error in Laser Heterodyne Interferometers; Self-Referencing Hartmann Test for Large-Aperture Telescopes; Measuring a Fiber-Optic Delay Line Using a Mode-Locked Laser; Reconfigurable Hardware for Compressing Hyperspectral Image Data; Spatio-Temporal Equalizer for a Receiving-Antenna Feed Array; High-Speed Ring Bus; Nanoionics-Based Switches for Radio-Frequency Applications; Lunar Dust-Tolerant Electrical Connector; Compact, Reliable EEPROM Controller; Quad-Chip Double-Balanced Frequency Tripler; Ka-Band Waveguide Two-Way Hybrid Combiner for MMIC Amplifiers; Radiation-Hardened Solid-State Drive; Use of Nanofibers to Strengthen Hydrogels of Silica, Other Oxides, and Aerogels; Two Concepts for Deployable Trusses; Concentric Nested Toroidal Inflatable Structures; Investigating Dynamics of Eccentricity in Turbomachines; Improved Low-Temperature Performance of Li-Ion Cells Using New Electrolytes; Integrity Monitoring of Mercury Discharge Lamps; White-Light Phase-Conjugate Mirrors as Distortion Correctors; Biasable, Balanced, Fundamental Submillimeter Monolithic Membrane Mixer; ICER-3D Hyperspectral Image Compression Software; and Context Modeler for Wavelet Compression of Spectral Hyperspectral Images

Development and Characterization of Plasma-based Sources for Ambient Desorption/Ionization Mass Spectrometry

Thesis (Ph.D.) - Indiana University, Chemistry, 2011A number of atmospheric-pressure ionization sources for mass spectrometry has recently appeared in the literature to yield a field that is collectively referred to as Ambient Desorption/Ionization-Mass Spectrometry (ADI-MS). These sources include, among others, Desorption ElectroSpray Ionization (DESI), Direct Analysis in Real Time (DART), the Flowing Atmospheric-Pressure Afterglow (FAPA), and the Low-Temperature Plasma (LTP) probe. Collectively, these ADI-MS sources offer numerous advantages over conventional ionization sources, including direct analysis of solid, liquid, and gaseous samples, high ionization efficiency, and soft ionization. Additionally, their ability to analyze samples directly with no pretreatment dramatically reduces analysis times. The ultimate ambient ionization source would be one that is capable of desorbing and ionizing a broad range of analytes (polar, non-polar, small molecules, biopolymers, etc.) while being minimally influenced by matrix effects. While the plasma-based ADI-MS sources, such as FAPA, DART, and LTP, have been shown to be capable of efficiently ionizing a range of small molecules, few investigations have been aimed at understanding desorption and ionization processes or matrix effects that occur with them. At present, the performance and fundamental characteristics of the FAPA source, developed in our research group, and the LTP probe, developed at Purdue University, are being evaluated through optical and mass-spectrometric methods. The FAPA source consists of a direct-current, atmospheric-pressure glow discharge in a pin-to-plate configuration. A hole in the plate allows ionized and excited plasma species to interact directly with a sample, while physically and electrically isolating the discharge from the sample-introduction region. Conversely, the LTP probe is a high-voltage, alternating-current, dielectric-barrier discharge that interacts directly with a sample. While the fundamental processes governing these discharges are quite different, mass spectra produced with both sources are very similar. The major differences are in achievable detection limits and susceptibility to matrix effects, with the FAPA sources routinely performing better. Direct, fundamental comparisons among the FAPA source, the LTP probe, and DART are made

Mass spectrometry imaging for plant biology: A review

Mass spectrometry imaging (MSI) is a developing technique to measure the spatio-temporal distribution of many biomolecules in tissues. Over the preceding decade, MSI has been adopted by plant biologists and applied in a broad range of areas, including primary metabolism, natural products, plant defense, plant responses to abiotic and biotic stress, plant lipids and the developing field of spatial metabolomics. This review covers recent advances in plant-based MSI, general aspects of instrumentation, analytical approaches, sample preparation and the current trends in respective plant research