A plan for the characterization, calibration, and evaluation of LAPR-2

A new airborne Linear Array Pushbroom Radiometer (LAPR-II) was built. LAPR-II will use linear arrays of silicon detectors to acquire four channels of digital image data for spectral bands within the visible and near infrared portions of the spectrum (0.4 - 1.0 micrometers). The data will be quantized to 10 bits, and spectral filters for each channel will be changeable in flight. The instrument will initially be flown aboard a NASA/Wallops' aircraft, and off nadir pointing of LAPR-II will be possible. Together, the instrument and its platform will provide a flexible readily available source of digital image data for scientific experiments. If LAPR-II is to serve as a precise scientific instrument, the instrument's characteristics must be quantitatively described and the data must be calibrated with respect to absolute radiometric units. The LAPR-II is described and the work required to characterize the instrument's spectral response, radiometric response, and spatial resolution and to calibrate the response from the many detectors per array is outlined

Combining transverse field detectors and color filter arrays to improve multispectral imaging systems

This work focuses on the improvement of a multispectral imaging sensor based on transverse field detectors (TFDs). We aimed to achieve a higher color and spectral accuracy in the estimation of spectral reflectances from sensor responses. Such an improvement was done by combining these recently developed silicon-based sensors with color filter arrays (CFAs). Consequently, we sacrificed the filter-less full spatial resolution property of TFDs to narrow down the spectrally broad sensitivities of these sensors.We designed and performed several experiments to test the influence of different design features on the estimation quality (type of sensor, tunability, interleaved polarization, use of CFAs, type of CFAs, number of shots), some of which are exclusive to TFDs.We compared systems that use a TFD with systems that use normal monochrome sensors, both combined with multispectral CFAs as well as common RGB filters present in commercial digital color cameras. Results showed that a system that combines TFDs and CFAs performs better than systems with the same type of multispectral CFA and other sensors, or even the same TFDs combined with different kinds of filters used in common imaging systems. We propose CFA+TFD-based systems with one or two shots, depending on the possibility of using longer capturing times or not. Improved TFD systems thus emerge as an interesting possibility for multispectral acquisition, which overcomes the limited accuracy found in previous studies.Spanish Ministry of Economy and Competitiveness through the research project DPI2011-2320

Integration, Testing, And Analysis Of Multispectral Imager On Small Unmanned Aerial System For Skin Detection

Small Unmanned Aerial Systems (SUAS) have been utilized by the military, geological researchers, and first responders, to provide information about the environment in real time. Hyperspectral Imagery (HSI) provides high resolution data in the spatial and spectral dimension; all objects, including skin have unique spectral signatures. However, little research has been done to integrate HSI into SUAS due to their cost and form factor. Multispectral Imagery (MSI) has proven capable of dismount detection with several distinct wavelengths. This research proposes a spectral imaging system that can detect dismounts on SUAS. Also, factors that pertain to accurate dismount detection with an SUAS are explored. Dismount skin detection from an aerial platform also has an inherent difficulty compared to ground-based platforms. Computer vision registration, stereo camera calibration, and geolocation from autopilot telemetry are utilized to design a dismount detection platform with the Systems Engineering methodology. An average 5.112% difference in ROC AUC values that compared a line scan spectral imager to the prototype area scan imager was recorded. Results indicated that an SUAS-based Spectral Imagers are capable tools in dismount detection protocols. Deficiencies associated with the test expedient prototype are discussed and recommendations for further improvements are provided

Efficient training procedures for multi-spectral demosaicing

The simultaneous acquisition of multi-spectral images on a single sensor can be efficiently performed by single shot capture using a mutli-spectral filter array. This paper focused on the demosaicing of color and near-infrared bands and relied on a convolutional neural network (CNN). To train the deep learning model robustly and accurately, it is necessary to provide enough training data, with sufficient variability. We focused on the design of an efficient training procedure by discovering an optimal training dataset. We propose two data selection strategies, motivated by slightly different concepts. The general term that will be used for the proposed models trained using data selection is data selection-based multi-spectral demosaicing (DSMD). The first idea is clustering-based data selection (DSMD-C), with the goal to discover a representative subset with a high variance so as to train a robust model. The second is an adaptive-based data selection (DSMD-A), a self-guided approach that selects new data based on the current model accuracy. We performed a controlled experimental evaluation of the proposed training strategies and the results show that a careful selection of data does benefit the speed and accuracy of training. We are still able to achieve high reconstruction accuracy with a lightweight model

Targeted Multispectral Filter Array Design for Endoscopic Cancer Detection in the Gastrointestinal Tract

Colour differences between healthy and diseased tissue in the gastrointestinal tract are detected visually by clinicians during white light endoscopy (WLE); however, the earliest signs of disease are often just a slightly different shade of pink compared to healthy tissue. Here, we propose to target alternative colours for imaging to improve contrast using custom multispectral filter arrays (MSFAs) that could be deployed in an endoscopic chip-on-tip configuration. Using an open-source toolbox, Opti-MSFA, we examined the optimal design of MSFAs for early cancer detection in the gastrointestinal tract. The toolbox was first extended to use additional classification models (k-Nearest Neighbour, Support Vector Machine, and Spectral Angle Mapper). Using input spectral data from published clinical trials examining the oesophagus and colon, we optimised the design of MSFAs with 3 to 9 different bands. We examined the variation of the spectral and spatial classification accuracy as a function of number of bands. The MSFA designs have high classification accuracies, suggesting that future implementation in endoscopy hardware could potentially enable improved early detection of disease in the gastrointestinal tract during routine screening and surveillance. Optimal MSFA configurations can achieve similar classification accuracies as the full spectral data in an implementation that could be realised in far simpler hardware. The reduced number of spectral bands could enable future deployment of multispectral imaging in an endoscopic chip-on-tip configuration.Comment: 29 page

Image fusion and reconstruction of compressed data: A joint approach

International audienceIn the context of data fusion, pansharpening refers to the combination of a panchromatic (PAN) and a multispectral (MS) image, aimed at generating an image that features both the high spatial resolution of the former and high spectral diversity of the latter. In this work we present a model to jointly solve the problem of data fusion and reconstruction of a compressed image; the latter is envisioned to be generated solely with optical on-board instruments, and stored in place of the original sources. The burden of data downlink is hence significantly reduced at the expense of a more laborious analysis done at the ground segment to estimate the missing information. The reconstruction algorithm estimates the target sharpened image directly instead of decompressing the original sources beforehand; a viable and practical novel solution is also introduced to show the effectiveness of the approach

InSPECtor: an end-to-end design framework for compressive pixelated hyperspectral instruments

Classic designs of hyperspectral instrumentation densely sample the spatial and spectral information of the scene of interest. Data may be compressed after the acquisition. In this paper we introduce a framework for the design of an optimized, micro-patterned snapshot hyperspectral imager that acquires an optimized subset of the spatial and spectral information in the scene. The data is thereby compressed already at the sensor level, but can be restored to the full hyperspectral data cube by the jointly optimized reconstructor. This framework is implemented with TensorFlow and makes use of its automatic differentiation for the joint optimization of the layout of the micro-patterned filter array as well as the reconstructor. We explore the achievable compression ratio for different numbers of filter passbands, number of scanning frames, and filter layouts using data collected by the Hyperscout instrument. We show resulting instrument designs that take snapshot measurements without losing significant information while reducing the data volume, acquisition time, or detector space by a factor of 40 as compared to classic, dense sampling. The joint optimization of a compressive hyperspectral imager design and the accompanying reconstructor provides an avenue to substantially reduce the data volume from hyperspectral imagers.Comment: 23 pages, 12 figures, published in Applied Optic

Joint demosaicing and fusion of multiresolution coded acquisitions: A unified image formation and reconstruction method

Novel optical imaging devices allow for hybrid acquisition modalities such as compressed acquisitions with locally different spatial and spectral resolutions captured by a single focal plane array. In this work, we propose to model the capturing system of a multiresolution coded acquisition (MRCA) in a unified framework, which natively includes conventional systems such as those based on spectral/color filter arrays, compressed coded apertures, and multiresolution sensing. We also propose a model-based image reconstruction algorithm performing a joint demosaicing and fusion (JoDeFu) of any acquisition modeled in the MRCA framework. The JoDeFu reconstruction algorithm solves an inverse problem with a proximal splitting technique and is able to reconstruct an uncompressed image datacube at the highest available spatial and spectral resolution. An implementation of the code is available at https://github.com/danaroth83/jodefu.Comment: 15 pages, 7 figures; regular pape