Integrated multispectral polarimetric sensor system

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2001.Includes bibliographical references (p. 115-121).In this thesis, a new type of integrated imaging sensor is introduced to detect multispectral and polarimetric signatures in an infrared scene. The sensor is a stack consisting of an infrared detector array, and an array of multispectral and polarimetric filters. In this first phase of the research, a single-pixel sensor was built. Multispectral filters were initially fabricated for the 3-5 micron waveband on sapphire substrates and polarimetric filters on silicon substrates. These were characterized separately and in mechanical contact as a single unit. The transmission characteristics of both filters show excellent agreement with the theoretical results. When the filters are integrated into an imaging sensor, such a sensor is anticipated to improve image contrast with sensor-fusion post processing. In addition, it will offer portability and robustness because of its integrated nature.by Dong-Hyun C. Kim.Ph.D

An Analysis of multimodal sensor fusion for target detection in an urban environment

This work makes a compelling case for simulation as an attractive tool in designing cutting-edge remote sensing systems to generate the sheer volume of data required for a reasonable trade study. The generalized approach presented here allows multimodal system designers to tailor target and sensor parameters for their particular scenarios of interest via synthetic image generation tools, ensuring that resources are best allocated while sensors are still in the design phase. Additionally, sensor operators can use the customizable process showcased here to optimize image collection parameters for existing sensors. In the remote sensing community, polarimetric capabilities are often seen as a tool without a widely accepted mission. This study proposes incorporating a polarimetric and spectral sensor in a multimodal architecture to improve target detection performance in an urban environment. Two novel multimodal fusion algorithms are proposed--one for the pixel level, and another for the decision level. A synthetic urban scene is rendered for 355 unique combinations of illumination condition and sensor viewing geometry with the Digital Imaging and Remote Sensing Image Generation (DIRSIG) model, and then validated to ensure the presence of enough background clutter. The utility of polarimetric information is shown to vary with the sun-target-sensor geometry, and the decision fusion algorithm is shown to generally outperform the pixel fusion algorithm. The results essentially suggest that polarimetric information may be leveraged to restore the capabilities of a spectral sensor if forced to image under less than ideal circumstances

Multidimensional Optical Sensing and Imaging Systems (MOSIS): From Macro to Micro Scales

Multidimensional optical imaging systems for information processing and visualization technologies have numerous applications in fields such as manufacturing, medical sciences, entertainment, robotics, surveillance, and defense. Among different three-dimensional (3-D) imaging methods, integral imaging is a promising multiperspective sensing and display technique. Compared with other 3-D imaging techniques, integral imaging can capture a scene using an incoherent light source and generate real 3-D images for observation without any special viewing devices. This review paper describes passive multidimensional imaging systems combined with different integral imaging configurations. One example is the integral-imaging-based multidimensional optical sensing and imaging systems (MOSIS), which can be used for 3-D visualization, seeing through obscurations, material inspection, and object recognition from microscales to long range imaging. This system utilizes many degrees of freedom such as time and space multiplexing, depth information, polarimetric, temporal, photon flux and multispectral information based on integral imaging to record and reconstruct the multidimensionally integrated scene. Image fusion may be used to integrate the multidimensional images obtained by polarimetric sensors, multispectral cameras, and various multiplexing techniques. The multidimensional images contain substantially more information compared with two-dimensional (2-D) images or conventional 3-D images. In addition, we present recent progress and applications of 3-D integral imaging including human gesture recognition in the time domain, depth estimation, mid-wave-infrared photon counting, 3-D polarimetric imaging for object shape and material identification, dynamic integral imaging implemented with liquid-crystal devices, and 3-D endoscopy for healthcare applications.B. Javidi wishes to acknowledge support by the National Science Foundation (NSF) under Grant NSF/IIS-1422179, and DARPA and US Army under contract number W911NF-13-1-0485. The work of P. Latorre Carmona, A. Martínez-Uso, J. M. Sotoca and F. Pla was supported by the Spanish Ministry of Economy under the project ESP2013-48458-C4-3-P, and by MICINN under the project MTM2013-48371-C2-2-PDGI, by Generalitat Valenciana under the project PROMETEO-II/2014/062, and by Universitat Jaume I through project P11B2014-09. The work of M. Martínez-Corral and G. Saavedra was supported by the Spanish Ministry of Economy and Competitiveness under the grant DPI2015-66458-C2-1R, and by the Generalitat Valenciana, Spain under the project PROMETEOII/2014/072

Polarimetric remote sensing system analysis: Digital Imaging and Remote Sensing Image Generation (DIRSIG) model validation and impact of polarization phenomenology on material discriminability

In addition to spectral information acquired by traditional multi/hyperspectral systems, passive electro optical and infrared (EO/IR) polarimetric sensors also measure the polarization response of different materials in the scene. Such an imaging modality can be useful in improving surface characterization; however, the characteristics of polarimetric systems have not been completely explored by the remote sensing community. Therefore, the main objective of this research was to advance our knowledge in polarimetric remote sensing by investigating the impact of polarization phenomenology on material discriminability. The first part of this research focuses on system validation, where the major goal was to assess the fidelity of the polarimetric images simulated using the Digital Imaging and Remote Sensing Image Generation (DIRSIG) model. A theoretical framework, based on polarization vision models used for animal vision studies and industrial defect detection applications, was developed within which the major components of the polarimetric image chain were validated. In the second part of this research, a polarization physics based approach for improved material discriminability was proposed. This approach utilizes the angular variation in the polarization response to infer the physical characteristics of the observed surface by imaging the scene in three different view directions. The usefulness of the proposed approach in improving detection performance in the absence of apriori knowledge about the target geometry was demonstrated. Sensitivity analysis of the proposed system for different scene related parameters was performed to identify the imaging conditions under which the material discriminability is maximized. Furthermore, the detection performance of the proposed polarimetric system was compared to that of the hyperspectral system to identify scenarios where polarization information can be very useful in improving the target contrast

Earth Observation – A Fundamental Input for Crisis Information Systems

Space-borne and airborne earth observation (EO) is a highly valuable source of spatio-temporal information promoting the ability for a rapid up-to-date assessment and (near-) real-time monitoring of natural or and man-made hazards and disasters. Such information has become indispensable in present-day disaster management activities. Thereby, EO based technologies have a role to play in each of the four phases of the disaster management cycle (i.e. mitigation, preparedness, response and recovery) with applications grouped into three main stages: - Pre-disaster (preparedness and mitigation): EO-based information extraction for assessing potential spatial distributions and severities of hazards as well as the vulnerability of a focus region for disaster risk evaluation and subsequent mitigation and preparedness activities. - Event crisis (response): Assessment and monitoring of regional extent and severities of the characteristics and impacts of a disaster to assist rapid crisis management. - Post-disaster (recovery): EO based information extraction to assist recovery activities. Within the PHAROS system a wide range of data products are used, which are varying in temporal, spatial and spectral resolution and coverage. The used sensor platforms comprise space-borne satellites and airborne systems, i.e. aircrafts as well as unmanned aerial systems (UAS)

POL-LWIR Vehicle Detection: Convolutional Neural Networks Meet Polarised Infrared Sensors

For vehicle autonomy, driver assistance and situational awareness, it is necessary to operate at day and night, and in all weather conditions. In particular, long wave infrared (LWIR) sensors that receive predominantly emitted radiation have the capability to operate at night as well as during the day. In this work, we employ a polarised LWIR (POL-LWIR) camera to acquire data from a mobile vehicle, to compare and contrast four different convolutional neural network (CNN) configurations to detect other vehicles in video sequences. We evaluate two distinct and promising approaches, two-stage detection (Faster-RCNN) and one-stage detection (SSD), in four different configurations. We also employ two different image decompositions: the first based on the polarisation ellipse and the second on the Stokes parameters themselves. To evaluate our approach, the experimental trials were quantified by mean average precision (mAP) and processing time, showing a clear trade-off between the two factors. For example, the best mAP result of 80.94% was achieved using Faster-RCNN, but at a frame rate of 6.4 fps. In contrast, MobileNet SSD achieved only 64.51% mAP, but at 53.4 fps.Comment: Computer Vision and Pattern Recognition Workshop 201

QUANTIFYING GRASSLAND NON-PHOTOSYNTHETIC VEGETATION BIOMASS USING REMOTE SENSING DATA

Non-photosynthetic vegetation (NPV) refers to vegetation that cannot perform a photosynthetic function. NPV, including standing dead vegetation and surface plant litter, plays a vital role in maintaining ecosystem function through controlling carbon, water and nutrient uptake as well as natural fire frequency and intensity in diverse ecosystems such as forest, savannah, wetland, cropland, and grassland. Due to its ecological importance, NPV has been selected as an indicator of grassland ecosystem health by the Alberta Public Lands Administration in Canada. The ecological importance of NPV has driven considerable research on quantifying NPV biomass with remote sensing approaches in various ecosystems. Although remote images, especially hyperspectral images, have demonstrated potential for use in NPV estimation, there has not been a way to quantify NPV biomass in semiarid grasslands where NPV biomass is affected by green vegetation (PV), bare soil and biological soil crust (BSC). The purpose of this research is to find a solution to quantitatively estimate NPV biomass with remote sensing approaches in semiarid mixed grasslands. Research was conducted in Grasslands National Park (GNP), a parcel of semiarid mixed prairie grassland in southern Saskatchewan, Canada. Multispectral images, including newly operational Landsat 8 Operational Land Imager (OLI) and Sentinel-2A Multi-spectral Instrument (MSIs) images and fine Quad-pol Radarsat-2 images were used for estimating NPV biomass in early, middle, and peak growing seasons via a simple linear regression approach. The results indicate that multispectral Landsat 8 OLI and Sentinel-2A MSIs have potential to quantify NPV biomass in peak and early senescence growing seasons. Radarsat-2 can also provide a solution for NPV biomass estimation. However, the performance of Radarsat-2 images is greatly affected by incidence angle of the image acquisition. This research filled a critical gap in applying remote sensing approaches to quantify NPV biomass in grassland ecosystems. NPV biomass estimates and approaches for estimating NPV biomass will contribute to grassland ecosystem health assessment (EHA) and natural resource (i.e. land, soil, water, plant, and animal) management

Material Characterization Using Passive Multispectral Polarimetric Imagery

A new method for characterization of unknown targets using passive multispectral polarimetric imagery is presented. Previous work makes use of a pBRDF derived equation for the degree of linear polarization and with the aid of measurements at multiple incident angles estimates refractive index and reflection angle. This work uses known incident and reflection angles along with dispersion equations and polarimetric data at multiple wavelengths to recover the index of refraction. Although imagery is collected with a division of time polarimeter and a spectral filter wheel in iterative, manual steps, the new algorithm could be applied to any set of registered multispectral polarimetric images most notably those produced by a recently introduced division of focal plane multispectral polarimetric sensor. Experimental results are presented showing the novel algorithm\u27s ability to classify and characterize a range of materials

MEDSAT: A Small Satellite for Malaria Early Warning and Control

This paper presents the design for a low cost, light satellite used to aid in the control of vector-borne diseases like malaria. The 340 kg satellite contains both a synthetic aperture radar and a visual/infrared multispectral scanner for remotely sensing the region of interest. Most of the design incorporates well established technology, but innovative features include the Pegasus launch vehicle, low mass and volume SAR and VIS/IR sensors, integrated design, low power SAR operation, microprocessor power system control, and advanced data compression and storage. This paper describes the main design considerations of the project which include, the remote sensing task, implementation for malaria control, launch vehicle, orbit, satellite bus, and satellite Subsystems

Three-dimensional imaging with multiple degrees of freedom using data fusion

This paper presents an overview of research work and some novel strategies and results on using data fusion in 3-D imaging when using multiple information sources. We examine a variety of approaches and applications such as 3-D imaging integrated with polarimetric and multispectral imaging, low levels of photon flux for photon-counting 3-D imaging, and image fusion in both multiwavelength 3-D digital holography and 3-D integral imaging. Results demonstrate the benefits data fusion provides for different purposes, including visualization enhancement under different conditions, and 3-D reconstruction quality improvement