Photon counting compressive depth mapping

We demonstrate a compressed sensing, photon counting lidar system based on the single-pixel camera. Our technique recovers both depth and intensity maps from a single under-sampled set of incoherent, linear projections of a scene of interest at ultra-low light levels around 0.5 picowatts. Only two-dimensional reconstructions are required to image a three-dimensional scene. We demonstrate intensity imaging and depth mapping at 256 x 256 pixel transverse resolution with acquisition times as short as 3 seconds. We also show novelty filtering, reconstructing only the difference between two instances of a scene. Finally, we acquire 32 x 32 pixel real-time video for three-dimensional object tracking at 14 frames-per-second.Comment: 16 pages, 8 figure

Magnetic-Visual Sensor Fusion-based Dense 3D Reconstruction and Localization for Endoscopic Capsule Robots

Reliable and real-time 3D reconstruction and localization functionality is a crucial prerequisite for the navigation of actively controlled capsule endoscopic robots as an emerging, minimally invasive diagnostic and therapeutic technology for use in the gastrointestinal (GI) tract. In this study, we propose a fully dense, non-rigidly deformable, strictly real-time, intraoperative map fusion approach for actively controlled endoscopic capsule robot applications which combines magnetic and vision-based localization, with non-rigid deformations based frame-to-model map fusion. The performance of the proposed method is demonstrated using four different ex-vivo porcine stomach models. Across different trajectories of varying speed and complexity, and four different endoscopic cameras, the root mean square surface reconstruction errors 1.58 to 2.17 cm.Comment: submitted to IROS 201

Depth Fields: Extending Light Field Techniques to Time-of-Flight Imaging

A variety of techniques such as light field, structured illumination, and time-of-flight (TOF) are commonly used for depth acquisition in consumer imaging, robotics and many other applications. Unfortunately, each technique suffers from its individual limitations preventing robust depth sensing. In this paper, we explore the strengths and weaknesses of combining light field and time-of-flight imaging, particularly the feasibility of an on-chip implementation as a single hybrid depth sensor. We refer to this combination as depth field imaging. Depth fields combine light field advantages such as synthetic aperture refocusing with TOF imaging advantages such as high depth resolution and coded signal processing to resolve multipath interference. We show applications including synthesizing virtual apertures for TOF imaging, improved depth mapping through partial and scattering occluders, and single frequency TOF phase unwrapping. Utilizing space, angle, and temporal coding, depth fields can improve depth sensing in the wild and generate new insights into the dimensions of light's plenoptic function.Comment: 9 pages, 8 figures, Accepted to 3DV 201

Background-Limited Imaging in the Near-Infrared with Warm InGaAs Sensors: Applications for Time-Domain Astronomy

We describe test observations made with a customized 640 x 512 pixel Indium Gallium Arsenide (InGaAs) prototype astronomical camera on the 100" DuPont telescope. This is the first test of InGaAs as a cost-effective alternative to HgCdTe for research-grade astronomical observations. The camera exhibits an instrument background of 113 e-/sec/pixel (dark + thermal) at an operating temperature of -40C for the sensor, maintained by a simple thermo-electric cooler. The optical train and mechanical structure float at ambient temperature with no cold stop, in contrast to most IR instruments which must be cooled to mitigate thermal backgrounds. Measurements of the night sky using a reimager with plate scale of 0.4 arc seconds / pixel show that the sky flux in Y is comparable to the dark current. At J the sky brightness exceeds dark current by a factor of four, and hence dominates the noise budget. The sensor read noise of ~43e- falls below sky+dark noise for exposures of t>7 seconds in Y and 3.5 seconds in J. We present test observations of several selected science targets, including high-significance detections of a lensed Type Ia supernova, a type IIb supernova, and a z=6.3 quasar. Deeper images are obtained for two local galaxies monitored for IR transients, and a galaxy cluster at z=0.87. Finally, we observe a partial transit of the hot JupiterHATS34b, demonstrating the photometric stability required over several hours to detect a 1.2% transit depth at high significance. A tiling of available larger-format sensors would produce an IR survey instrument with significant cost savings relative to HgCdTe-based cameras, if one is willing to forego the K band. Such a camera would be sensitive for a week or more to isotropic emission from r-process kilonova ejecta similar to that observed in GW170817, over the full 190 Mpc horizon of Advanced LIGO's design sensitivity for neutron star mergers.Comment: 13 pages, 12 figures, submitted to A

RGB-D datasets using microsoft kinect or similar sensors: a survey

RGB-D data has turned out to be a very useful representation of an indoor scene for solving fundamental computer vision problems. It takes the advantages of the color image that provides appearance information of an object and also the depth image that is immune to the variations in color, illumination, rotation angle and scale. With the invention of the low-cost Microsoft Kinect sensor, which was initially used for gaming and later became a popular device for computer vision, high quality RGB-D data can be acquired easily. In recent years, more and more RGB-D image/video datasets dedicated to various applications have become available, which are of great importance to benchmark the state-of-the-art. In this paper, we systematically survey popular RGB-D datasets for different applications including object recognition, scene classification, hand gesture recognition, 3D-simultaneous localization and mapping, and pose estimation. We provide the insights into the characteristics of each important dataset, and compare the popularity and the difficulty of those datasets. Overall, the main goal of this survey is to give a comprehensive description about the available RGB-D datasets and thus to guide researchers in the selection of suitable datasets for evaluating their algorithms