Illumination waveform optimization for time-of-flight range imaging cameras

Time-of-flight range imaging sensors acquire an image of a scene, where in addition to standard intensity information, the range (or distance) is also measured concurrently by each pixel. Range is measured using a correlation technique, where an amplitude modulated light source illuminates the scene and the reflected light is sampled by a gain modulated image sensor. Typically the illumination source and image sensor are amplitude modulated with square waves, leading to a range measurement linearity error caused by aliased harmonic components within the correlation waveform. A simple method to improve measurement linearity by reducing the duty cycle of the illumination waveform to suppress problematic aliased harmonic components is demonstrated. If the total optical power is kept constant, the measured correlation waveform amplitude also increases at these reduced illumination duty cycles. Measurement performance is evaluated over a range of illumination duty cycles, both for a standard range imaging camera configuration, and also using a more complicated phase encoding method that is designed to cancel aliased harmonics during the sampling process. The standard configuration benefits from improved measurement linearity for illumination duty cycles around 30%, while the measured amplitude, hence range precision, is increased for both methods as the duty cycle is reduced below 50% (while maintaining constant optical power)

Characterizing an image intensifier in an full-field range image system

We are developing a high precision full-field range imaging system. An integral component in this system is an image intensifier, which is modulated at frequencies up to 100 MHz. The range measurement precision is dictated by the image intensifier performance, in particular, the achievable modulation frequency, modulation depth, and waveform shape. By characterizing the image intensifier response, undesirable effects can be observed and quantified with regards to the consequence on the resulting range measurements, and the optimal operating conditions can be selected to minimize these disturbances. The characterization process utilizes a pulsed laser source to temporally probe the gain of the image intensifier. The laser is pulsed at a repetition rate slightly different to the image intensifier modulation frequency, producing a continuous phase shift between the two signals. A charge coupled device samples the image intensifier output, capturing the response over a complete modulation period. Deficiencies in our measured response are clearly identifiable and simple modifications to the configuration of our electrical driver circuit improve the modulation performance

Resolving depth measurement ambiguity with commercially available range imaging cameras

Time-of-flight range imaging is typically performed with the amplitude modulated continuous wave method. This involves illuminating a scene with amplitude modulated light. Reflected light from the scene is received by the sensor with the range to the scene encoded as a phase delay of the modulation envelope. Due to the cyclic nature of phase, an ambiguity in the measured range occurs every half wavelength in distance, thereby limiting the maximum useable range of the camera. This paper proposes a procedure to resolve depth ambiguity using software post processing. First, the range data is processed to segment the scene into separate objects. The average intensity of each object can then be used to determine which pixels are beyond the non-ambiguous range. The results demonstrate that depth ambiguity can be resolved for various scenes using only the available depth and intensity information. This proposed method reduces the sensitivity to objects with very high and very low reflectance, normally a key problem with basic threshold approaches. This approach is very flexible as it can be used with any range imaging camera. Furthermore, capture time is not extended, keeping the artifacts caused by moving objects at a minimum. This makes it suitable for applications such as robot vision where the camera may be moving during captures. The key limitation of the method is its inability to distinguish between two overlapping objects that are separated by a distance of exactly one non-ambiguous range. Overall the reliability of this method is higher than the basic threshold approach, but not as high as the multiple frequency method of resolving ambiguity

A high resolution full-field range imaging system for robotic devices

There has been considerable effort by many researchers to develop a high resolution full-field range imaging system. Traditionally these systems rely on a homodyne technique that modulates the illumination source and shutter speed at some high frequency. These systems tend to suffer from the need to be calibrated to account for changing ambient light conditions and generally cannot provide better than single centimeter range resolution, and even then over a range of only a few meters. We present a system, tested to proof-of-concept stage that is being developed for use on a range of mobile robots. The system has the potential for real-time, sub millimeter range resolution, with minimal power and space requirements

Improved linearity using harmonic error rejection in a full-field range imaging system

Full field range imaging cameras are used to simultaneously measure the distance for every pixel in a given scene using an intensity modulated illumination source and a gain modulated receiver array. The light is reflected from an object in the scene, and the modulation envelope experiences a phase shift proportional to the target distance. Ideally the waveforms are sinusoidal, allowing the phase, and hence object range, to be determined from four measurements using an arctangent function. In practice these waveforms are often not perfectly sinusoidal, and in some cases square waveforms are instead used to simplify the electronic drive requirements. The waveforms therefore commonly contain odd harmonics which contribute a nonlinear error to the phase determination, and therefore an error in the range measurement. We have developed a unique sampling method to cancel the effect of these harmonics, with the results showing an order of magnitude improvement in the measurement linearity without the need for calibration or lookup tables, while the acquisition time remains unchanged. The technique can be applied to existing range imaging systems without having to change or modify the complex illumination or sensor systems, instead only requiring a change to the signal generation and timing electronics

A fast Maximum Likelihood method for improving AMCW lidar precision using waveform shape

Amplitude Modulated Continuous Wave imaging lidar systems use the time-of-flight principle to determine the range to objects in a scene. Typical systems use modulated illumination of a scene and a modulated sensor or image intensifier. By changing the relative phase of the two modulation signals it is possible to measure the phase shift induced in the illumination signal, thus the range to the scene. In practical systems, the resultant correlation waveform contains harmonics that typically result in a non-linear range response. Nevertheless, these harmonics can be used to improve range precision. We model a waveform continuously variable in phase and intensity as a linear interpolation. By approximating the problem as a Maximum Likelihood problem, an analytic solution for the problem is derived that enables an entire range image to be processed in a few seconds. A substantial improvement in overall RMS error and precision over the standard Fourier phase analysis approach results

Image intensifier characterization

An image intensifier forms an integral part of a full-field image range finder under development at the University of Waikato. Operating as a high speed shutter with repetition rates up to 100 MHz, a method is described to characterise the response, both temporally and spatially, of the intensifier in order to correct for variations in the field of view and to optimise the operating conditions. A short pulse of visible light is emitted by a laser diode, uniformly illuminating the image intensifier, while a CCD camera captures the output from the intensifier. The phase of the laser pulse is continuously varied using a heterodyne configuration, automatically producing a set of samples covering the modulation cycle. The results show some anomalies in the response of our system and some simple solutions are proposed to correct for these

Characterizing an image intensifier in an full-field range image system

We are developing a high precision full-field range imaging system. An integral component in this system is an image intensifier, which is modulated at frequencies up to 100 MHz. The range measurement precision is dictated by the image intensifier performance, in particular, the achievable modulation frequency, modulation depth, and waveform shape. By characterizing the image intensifier response, undesirable effects can be observed and quantified with regards to the consequence on the resulting range measurements, and the optimal operating conditions can be selected to minimize these disturbances. The characterization process utilizes a pulsed laser source to temporally probe the gain of the image intensifier. The laser is pulsed at a repetition rate slightly different to the image intensifier modulation frequency, producing a continuous phase shift between the two signals. A charge coupled device samples the image intensifier output, capturing the response over a complete modulation period. Deficiencies in our measured response are clearly identifiable and simple modifications to the configuration of our electrical driver circuit improve the modulation performance

Multiple range imaging camera operation with minimal performance impact

Time-of-flight range imaging cameras operate by illuminating a scene with amplitude modulated light and measuring the phase shift of the modulation envelope between the emitted and reflected light. Object distance can then be calculated from this phase measurement. This approach does not work in multiple camera environments as the measured phase is corrupted by the illumination from other cameras. To minimize inaccuracies in multiple camera environments, replacing the traditional cyclic modulation with pseudo-noise amplitude modulation has been previously demonstrated. However, this technique effectively reduced the modulation frequency, therefore decreasing the distance measurement precision (which has a proportional relationship with the modulation frequency). A new modulation scheme using maximum length pseudo-random sequences binary phase encoded onto the existing cyclic amplitude modulation, is presented. The effective modulation frequency therefore remains unchanged, providing range measurements with high precision. The effectiveness of the new modulation scheme was verified using a custom time-of-flight camera based on the PMD19-K2 range imaging sensor. The new pseudo-noise modulation has no significant performance decrease in a single camera environment. In a two camera environment, the precision is only reduced by the increased photon shot noise from the second illumination source

Characterization of modulated time-of-flight range image sensors

A number of full field image sensors have been developed that are capable of simultaneously measuring intensity and distance (range) for every pixel in a given scene using an indirect time-of-flight measurement technique. A light source is intensity modulated at a frequency between 10–100 MHz, and an image sensor is modulated at the same frequency, synchronously sampling light reflected from objects in the scene (homodyne detection). The time of flight is manifested as a phase shift in the illumination modulation envelope, which can be determined from the sampled data simultaneously for each pixel in the scene. This paper presents a method of characterizing the high frequency modulation response of these image sensors, using a pico-second laser pulser. The characterization results allow the optimal operating parameters, such as the modulation frequency, to be identified in order to maximize the range measurement precision for a given sensor. A number of potential sources of error exist when using these sensors, including deficiencies in the modulation waveform shape, duty cycle, or phase, resulting in contamination of the resultant range data. From the characterization data these parameters can be identified and compensated for by modifying the sensor hardware or through post processing of the acquired range measurements