Multimodal Multispectral Imaging System for Small UAVs

Multispectral imaging is an attractive sensing modality for small unmanned aerial vehicles (UAVs) in numerous applications. The most compact spectral camera architecture is based on spectral filters in the focal plane. Vehicle movement can be used to scan the scene using multiple bandpass filters arranged perpendicular to the flight direction. With known camera trajectory and scene structure, it is possible to assemble a spectral image in software. In this letter, we demonstrate the feasibility of a novel concept for low-cost wide area multispectral imaging with integrated spectral consistency testing. Six bandpass filters are arranged in a periodically repeating pattern. Since different bands are recorded at different times and in different viewing directions, there is a risk of obtaining spectral artifacts in the image. We exploit the repeated sampling of bands to enable spectral consistency testing, which leads to significantly improved spectral integrity. In addition, an unfiltered region permits conventional 2D video imaging that can be used for image-based navigation and 3D reconstruction. The proposed multimodal imaging system was tested on a UAV in a realistic experiment. The results demonstrate that spectral reconstruction and consistency testing can be performed by image processing alone, based on visual simultaneous localization and mapping (VSLAM)

Multimodal Multispectral Imaging System for Small UAVs

Multispectral imaging is an attractive sensing modality for small unmanned aerial vehicles (UAVs) in numerous applications. The most compact spectral camera architecture is based on spectral filters in the focal plane. Vehicle movement can be used to scan the scene using multiple bandpass filters arranged perpendicular to the flight direction. With known camera trajectory and scene structure, it is possible to assemble a spectral image in software. In this letter, we demonstrate the feasibility of a novel concept for low-cost wide area multispectral imaging with integrated spectral consistency testing. Six bandpass filters are arranged in a periodically repeating pattern. Since different bands are recorded at different times and in different viewing directions, there is a risk of obtaining spectral artifacts in the image. We exploit the repeated sampling of bands to enable spectral consistency testing, which leads to significantly improved spectral integrity. In addition, an unfiltered region permits conventional 2D video imaging that can be used for image-based navigation and 3D reconstruction. The proposed multimodal imaging system was tested on a UAV in a realistic experiment. The results demonstrate that spectral reconstruction and consistency testing can be performed by image processing alone, based on visual simultaneous localization and mapping (VSLAM)

In-operation calibration of clock-bias and intrinsic parameters for pan-tilt-zoom cameras based on keypoint tracking

We propose a method for jointly estimating intrinsic calibration and internal clock synchronisation for a pantilt- zoom (PTZ) camera using only data that can be acquired in the field during normal operation. Results show that this method is a promising starting point towards using software to replace costly timing hardware in such cameras. Through experiments we provide calibration and clock synchronisation for an off-the-shelf low-cost PTZ camera, and observe a greatly improved directional accuracy, even during mild manoeuvres

Real-time anomaly detection in hyperspectral images using multivariate normal mixture models and GPU processing

International audienceHyperspectral imaging, which records a detailed spectrum of light arriving in each pixel, has many potential uses in remote sensing as well as other application areas. Practical applications will typically require real-time processing of large data volumes recorded by a hyperspectral imager. This paper investigates the use of graphics processing units (GPU) for such real-time processing. In particular, the paper studies a hyperspectral anomaly detection algorithm based on normal mixture modelling of the background spectral distribution, a computationally demanding task relevant to military target detection and numerous other applications. The algorithm parts are analysed with respect to complexity and potential for parallellization. The computationally dominating parts are implemented on an Nvidia GeForce 8800 GPU using the Compute Unified Device Architecture programming interface. GPU computing performance is compared to a multicore central processing unit implementation. Overall, the GPU implementation runs significantly faster, particularly for highly data-parallelizable and arithmetically intensive algorithm parts. For the parts related to covariance computation, the speed gain is less pronounced, probably due to a smaller ratio of arithmetic to memory access. Detection results on an actual data set demonstrate that the total speedup provided by the GPU is sufficient to enable realtime anomaly detection with normal mixture models even for an airborne hyperspectral imager with high spatial and spectral resolution

Compact multimodal multispectral sensor system for tactical reconnaissance

Multispectral imaging is an attractive sensing modality for small unmanned aerial vehicles (UAVs) in numerous military and civilian applications such as reconnaissance, target detection, and precision agriculture. Cameras based on patterned filters in the focal plane, such as conventional colour cameras, represent the most compact architecture for spectral imaging, but image reconstruction becomes challenging at higher band counts. We consider a camera configuration where six bandpass filters are arranged in a periodically repeating pattern in the focal plane. In addition, a large unfiltered region permits conventional monochromatic video imaging that can be used for situational awareness (SA), including estimating the camera motion and the 3D structure of the ground surface. By platform movement, the filters are scanned over the scene, capturing an irregular pattern of spectral samples of the ground surface. Through estimation of the camera trajectory and 3D scene structure, it is still possible to assemble a spectral image by fusing all measurements in software. The repeated sampling of bands enables spectral consistency testing, which can improve spectral integrity significantly. The result is a truly multimodal camera sensor system able to produce a range of image products. Here, we investigate its application in tactical reconnaissance by pushing towards on-board real-time spectral reconstruction based on visual odometry (VO) and full 3D reconstruction of the scene. The results are compared with offline processing based on estimates from visual simultaneous localisation and mapping (VSLAM) and indicate that the multimodal sensing concept has a clear potential for use in tactical reconnaissance scenarios

Anisotropic Scattered Data Interpolation for Pushbroom Image Rectification

This article deals with fast and accurate visualization of pushbroom image data from airborne and spaceborne platforms. A pushbroom sensor acquires images in a line-scanning fashion, and this results in scattered input data that needs to be resampled onto a uniform grid for geometrically correct visualization. To this end, we model the anisotropic spatial dependence structure caused by the acquisition process. Several methods for scattered data interpolation are then adapted to handle the induced anisotropic metric and compared for the pushbroom image rectification problem. A trick that exploits the semi-ordered line structure of pushbroom data to improve the computational complexity several orders of magnitude is also presented

Hyperspectral reconnaissance in urban environment

Seven countries within the European Defence Agency (EDA) framework are joining effort in a four year project (2009- 2013) on Detection in Urban scenario using Combined Airborne imaging Sensors (DUCAS). Data has been collected in a joint field trial including instrumentation for 3D mapping, hyperspectral and high resolution imagery together with in situ instrumentation for target, background and atmospheric characterization. Extensive analysis with respect to detection and classification has been performed. Progress in performance has been shown using combinations of hyperspectral and high spatial resolution sensors. © 2013 SPIE