Quantum-inspired computational imaging

Computational imaging combines measurement and computational methods with the aim of forming images even when the measurement conditions are weak, few in number, or highly indirect. The recent surge in quantum-inspired imaging sensors, together with a new wave of algorithms allowing on-chip, scalable and robust data processing, has induced an increase of activity with notable results in the domain of low-light flux imaging and sensing. We provide an overview of the major challenges encountered in low-illumination (e.g., ultrafast) imaging and how these problems have recently been addressed for imaging applications in extreme conditions. These methods provide examples of the future imaging solutions to be developed, for which the best results are expected to arise from an efficient codesign of the sensors and data analysis tools.Y.A. acknowledges support from the UK Royal Academy of Engineering under the Research Fellowship Scheme (RF201617/16/31). S.McL. acknowledges financial support from the UK Engineering and Physical Sciences Research Council (grant EP/J015180/1). V.G. acknowledges support from the U.S. Defense Advanced Research Projects Agency (DARPA) InPho program through U.S. Army Research Office award W911NF-10-1-0404, the U.S. DARPA REVEAL program through contract HR0011-16-C-0030, and U.S. National Science Foundation through grants 1161413 and 1422034. A.H. acknowledges support from U.S. Army Research Office award W911NF-15-1-0479, U.S. Department of the Air Force grant FA8650-15-D-1845, and U.S. Department of Energy National Nuclear Security Administration grant DE-NA0002534. D.F. acknowledges financial support from the UK Engineering and Physical Sciences Research Council (grants EP/M006514/1 and EP/M01326X/1). (RF201617/16/31 - UK Royal Academy of Engineering; EP/J015180/1 - UK Engineering and Physical Sciences Research Council; EP/M006514/1 - UK Engineering and Physical Sciences Research Council; EP/M01326X/1 - UK Engineering and Physical Sciences Research Council; W911NF-10-1-0404 - U.S. Defense Advanced Research Projects Agency (DARPA) InPho program through U.S. Army Research Office; HR0011-16-C-0030 - U.S. DARPA REVEAL program; 1161413 - U.S. National Science Foundation; 1422034 - U.S. National Science Foundation; W911NF-15-1-0479 - U.S. Army Research Office; FA8650-15-D-1845 - U.S. Department of the Air Force; DE-NA0002534 - U.S. Department of Energy National Nuclear Security Administration)Accepted manuscrip

On evolution of CMOS image sensors

CMOS Image Sensors have become the principal technology in majority of digital cameras. They started replacing the film and Charge Coupled Devices in the last decade with the promise of lower cost, lower power requirement, higher integration and the potential of focal plane processing. However, the principal factor behind their success has been the ability to utilise the shrinkage in CMOS technology to make smaller pixels, and thereby have more resolution without increasing the cost. With the market of image sensors exploding courtesy their inte- gration with communication and computation devices, technology developers improved the CMOS processes to have better optical performance. Nevertheless, the promises of focal plane processing as well as on-chip integration have not been fulfilled. The market is still being pushed by the desire of having higher number of pixels and better image quality, however, differentiation is being difficult for any image sensor manufacturer. In the paper, we will explore potential disruptive growth directions for CMOS Image sensors and ways to achieve the same

Investigation of jitter on full-field amplitude modulated continuous wave time-of-flight range imaging cameras

The time-of-flight (ToF) range imaging cameras indirectly measure the time taken from the modulation light source to the scene and back to the camera and it is this principle that is used in depth cameras to perform depth measurements. This thesis is focused on ToF cameras that are based on the amplitude modulated continuous wave (AMCW) lidar techniques which measure the phase difference between the emitted and reflected light signals. Due to their portable size, feasible design, low weight and low energy consumption, these cameras have high demand in many applications. Commercially available AMCW ToF cameras have relatively high noise levels due to electronic sources such as shot noise, reset noise, amplifier noise, crosstalk, analogue to digital converters quantization and multipath light interference. Many noise sources in these cameras such as harmonic contamination, non-linearity, multipath interferences and light scattering are well investigated. In contrast, the effect of electronic jitter as a noise source in ranging cameras is barely studied. Jitter is defined to be any timing movement with reference to an ideal signal. An investigation of the effect of jitter on range imaging is important because timing errors potentially could cause errors in measuring phase, thus in range. The purpose of this research is to investigate the effect of jitter on range measurement in AMCW ToF range imaging. This is achieved by three main contributions: a development of a common algorithm for measurement of the jitter present in signals from depth cameras, secondly the proposal of a cost effective alternative method to measure jitter by using a software defined radio receiver, and finally an analysis of the influence of jitter on range measurement. Among the three contributions of this thesis, first, an algorithm for jitter extraction of a signal without access to a reference clock signal is proposed. The proposed algorithm is based upon Fourier analysis with signal processing techniques and it can be used for real time jitter extraction on a modulated signal with any kind of shape (sinusoidal, triangular, rectangular). The method is used to measure the amount of jitter in the light signals of two AMCW ToF range imaging cameras, namely, MESA Imaging SwissRanger 4000 and SoftKinetic DepthSense 325. Periodic and random jitter were found to be present in the light sources of both cameras with the MESA camera notably worse with random jitter of (159.6 +/- 0.1) ps RMS in amplitude. Next, in a novel approach, an inexpensive software defined radio (SDR) USB dongle is used with the proposed algorithm to extract the jitter in the light signal of the above two ToF cameras. This is a cost effective alternative to the expensive real-time medium speed digital oscilloscope. However, it is shown that this method has some significant limitations, (1) it can measure the jitter only up to half of the intermediate-frequency obtained from the down shift of the amplified radio frequency with the local oscillator which is less than the Nyquist frequency of the dongle and (2) if the number of samples per cycle captured from this dongle is not sufficient then the jitter extraction does not succeed since the signal is not properly (smoothly) represented. Finally, the periodic and random jitter influence on range measurements made with AMCW range imaging cameras are studied. An analytical model for the periodic jitter on the range measurements under the heterodyne and homodyne operations in AMCW ToF range imaging cameras is obtained in the frequency domain. The analytical model is tested through simulated data with various parameters in the system. The product of angular modulation frequency of the camera and the amplitude of the periodic jitter is a characteristic parameter for the phase error due to the presence of periodic jitter. We found that for currently available AMCW cameras (modulation frequency less than 100 MHz), neither periodic nor random jitter has a measurable effect on range measurement. But with modulation frequency increases and integration period decreases likely in the near future, periodic jitter may have a measurable detection affect on ranging. The influence of random jitter is also investigated by obtaining an analytical model based on stochastic calculus by using fundamental statistics and Fourier analysis. It is assumed that the random jitter follows the Gaussian distribution. Monte Carlo simulation is performed on the model obtained for a 1 ms integration period. We found increasing the modulation frequency above approximately 400 MHz with random jitter of 140 ps has a measurable affect on ranging

Research Brief

Approved for public release; distribution is unlimited

DragonflEYE: a passive approach to aerial collision sensing

"This dissertation describes the design, development and test of a passive wide-field optical aircraft collision sensing instrument titled 'DragonflEYE'. Such a ""sense-and-avoid"" instrument is desired for autonomous unmanned aerial systems operating in civilian airspace. The instrument was configured as a network of smart camera nodes and implemented using commercial, off-the-shelf components. An end-to-end imaging train model was developed and important figures of merit were derived. Transfer functions arising from intermediate mediums were discussed and their impact assessed. Multiple prototypes were developed. The expected performance of the instrument was iteratively evaluated on the prototypes, beginning with modeling activities followed by laboratory tests, ground tests and flight tests. A prototype was mounted on a Bell 205 helicopter for flight tests, with a Bell 206 helicopter acting as the target. Raw imagery was recorded alongside ancillary aircraft data, and stored for the offline assessment of performance. The ""range at first detection"" (R0), is presented as a robust measure of sensor performance, based on a suitably defined signal-to-noise ratio. The analysis treats target radiance fluctuations, ground clutter, atmospheric effects, platform motion and random noise elements. Under the measurement conditions, R0 exceeded flight crew acquisition ranges. Secondary figures of merit are also discussed, including time to impact, target size and growth, and the impact of resolution on detection range. The hardware was structured to facilitate a real-time hierarchical image-processing pipeline, with selected image processing techniques introduced. In particular, the height of an observed event above the horizon compensates for angular motion of the helicopter platform.

Methods for linear radial motion estimation in time-of-flight range imaging

Motion artefacts in time-of-flight range imaging are treated as a feature to measure. Methods for measuring linear radial velocity from range imaging cameras are developed and tested. With the measurement of velocity, the range to the position of the target object at the start of the data acquisition period is computed, effectively correcting the motion error. A new phase based pseudo-quadrature method designed for low speed measurement measures radial velocity up to ±1.8 m/s with RMSE 0.045 m/s and standard deviation of 0.09-0.33 m/s, and new high-speed Doppler extraction method measures radial velocity up to ±40 m/s with standard deviation better than 1 m/s and RMSE of 3.5 m/s

Advanced photon counting techniques for long-range depth imaging

The Time-Correlated Single-Photon Counting (TCSPC) technique has emerged as a candidate approach for Light Detection and Ranging (LiDAR) and active depth imaging applications. The work of this Thesis concentrates on the development and investigation of functional TCSPC-based long-range scanning time-of-flight (TOF) depth imaging systems. Although these systems have several different configurations and functions, all can facilitate depth profiling of remote targets at low light levels and with good surface-to-surface depth resolution. Firstly, a Superconducting Nanowire Single-Photon Detector (SNSPD) and an InGaAs/InP Single-Photon Avalanche Diode (SPAD) module were employed for developing kilometre-range TOF depth imaging systems at wavelengths of ~1550 nm. Secondly, a TOF depth imaging system at a wavelength of 817 nm that incorporated a Complementary Metal-Oxide-Semiconductor (CMOS) 32×32 Si-SPAD detector array was developed. This system was used with structured illumination to examine the potential for covert, eye-safe and high-speed depth imaging. In order to improve the light coupling efficiency onto the detectors, the arrayed CMOS Si-SPAD detector chips were integrated with microlens arrays using flip-chip bonding technology. This approach led to the improvement in the fill factor by up to a factor of 15. Thirdly, a multispectral TCSPC-based full-waveform LiDAR system was developed using a tunable broadband pulsed supercontinuum laser source which can provide simultaneous multispectral illumination, at wavelengths of 531, 570, 670 and ~780 nm. The investigated multispectral reflectance data on a tree was used to provide the determination of physiological parameters as a function of the tree depth profile relating to biomass and foliage photosynthetic efficiency. Fourthly, depth images were estimated using spatial correlation techniques in order to reduce the aggregate number of photon required for depth reconstruction with low error. A depth imaging system was characterised and re-configured to reduce the effects of scintillation due to atmospheric turbulence. In addition, depth images were analysed in terms of spatial and depth resolution