253 research outputs found
Neural Spectro-polarimetric Fields
Modeling the spatial radiance distribution of light rays in a scene has been
extensively explored for applications, including view synthesis. Spectrum and
polarization, the wave properties of light, are often neglected due to their
integration into three RGB spectral bands and their non-perceptibility to human
vision. Despite this, these properties encompass substantial material and
geometric information about a scene. In this work, we propose to model
spectro-polarimetric fields, the spatial Stokes-vector distribution of any
light ray at an arbitrary wavelength. We present Neural Spectro-polarimetric
Fields (NeSpoF), a neural representation that models the physically-valid
Stokes vector at given continuous variables of position, direction, and
wavelength. NeSpoF manages inherently noisy raw measurements, showcases memory
efficiency, and preserves physically vital signals, factors that are crucial
for representing the high-dimensional signal of a spectro-polarimetric field.
To validate NeSpoF, we introduce the first multi-view
hyperspectral-polarimetric image dataset, comprised of both synthetic and
real-world scenes. These were captured using our compact
hyperspectral-polarimetric imaging system, which has been calibrated for
robustness against system imperfections. We demonstrate the capabilities of
NeSpoF on diverse scenes
Multidimensional Nonlinear Optical Imaging
The work in this dissertation is focused on extending the information content for second harmonic generation (SHG) and two-photon excited fluorescence (TPEF) imaging. Despite the simplicity and symmetry selectivity of nonlinear optical processes, limited information on chemical composition can be recovered solely based on intensity measurements. To further explore the potential for second order nonlinear optical (NLO) measurements, additional dimensions were added to the NLO imaging platforms. By combining NLO microscopy with powder X-ray diffraction, a novel approach was established for accessing percent crystallinity in amorphous solid dispersions (ASDs) with a limit of detection in the ppm range. ASDs are preferable alternative for crystalline forms when formulating poorly soluble active pharmaceutical ingredients (APIs). However, the high detection limit for current available methods limited the study of long term stability for ASDs at early stage. Besides adding additional modalities to NLO microscopy, polarization dependent SHG provides rich information on local structures for collagen fibers in tissues. However, significant loss in polarization purities occurs when light penetrate through the tissue. A new theoretical framework was introduced to extract information with partially or fully depolarized light. In addition, a video-rate hyperspectral TPEF imaging system was demonstrated with over 2,200 fluorescence channels throughput spatial-spectral multiplexing
Near-infrared active polarimetric and multispectral laboratory demonstrator for target detection
International audienceWe report on the design and exploitation of a real-field laboratory demonstrator combining active polarimetric and multispectral functions. Its building blocks, including a multiwavelength pulsed optical parametric oscillator at the emission side and a hyperspectral imager with polarimetric capability at the reception side, are described. The results obtained with this demonstrator are illustrated on some examples and discussed. In particular it is found that good detection performances rely on joint use of intensity and polarimetric images, with these images exhibiting complementary signatures in most cases
Food Tray Sealing Fault Detection in Multi-Spectral Images Using Data Fusion and Deep Learning Techniques
A correct food tray sealing is required to preserve food properties and safety for consumers. Traditional food packaging inspections are made by human operators to detect seal defects. Recent advances in the field of food inspection have been related to the use of hyperspectral imaging technology and automated vision-based inspection systems. A deep learning-based approach for food tray sealing fault detection using hyperspectral images is described. Several pixel-based image fusion methods are proposed to obtain 2D images from the 3D hyperspectral image datacube, which feeds the deep learning (DL) algorithms. Instead of considering all spectral bands in region of interest around a contaminated or faulty seal area, only relevant bands are selected using data fusion. These techniques greatly improve the computation time while maintaining a high classification ratio, showing that the fused image contains enough information for checking a food tray sealing state (faulty or normal), avoiding feeding a large image datacube to the DL algorithms. Additionally, the proposed DL algorithms do not require any prior handcraft approach, i.e., no manual tuning of the parameters in the algorithms are required since the training process adjusts the algorithm. The experimental results, validated using an industrial dataset for food trays, along with different deep learning methods, demonstrate the effectiveness of the proposed approach. In the studied dataset, an accuracy of 88.7%, 88.3%, 89.3%, and 90.1% was achieved for Deep Belief Network (DBN), Extreme Learning Machine (ELM), Stacked Auto Encoder (SAE), and Convolutional Neural Network (CNN), respectively
Application of Multi-Sensor Fusion Technology in Target Detection and Recognition
Application of multi-sensor fusion technology has drawn a lot of industrial and academic interest in recent years. The multi-sensor fusion methods are widely used in many applications, such as autonomous systems, remote sensing, video surveillance, and the military. These methods can obtain the complementary properties of targets by considering multiple sensors. On the other hand, they can achieve a detailed environment description and accurate detection of interest targets based on the information from different sensors.This book collects novel developments in the field of multi-sensor, multi-source, and multi-process information fusion. Articles are expected to emphasize one or more of the three facets: architectures, algorithms, and applications. Published papers dealing with fundamental theoretical analyses, as well as those demonstrating their application to real-world problems
Spectral and Polarization Vision: Spectro-polarimetric Real-world Dataset
Image datasets are essential not only in validating existing methods in
computer vision but also in developing new methods. Most existing image
datasets focus on trichromatic intensity images to mimic human vision. However,
polarization and spectrum, the wave properties of light that animals in harsh
environments and with limited brain capacity often rely on, remain
underrepresented in existing datasets. Although spectro-polarimetric datasets
exist, these datasets have insufficient object diversity, limited illumination
conditions, linear-only polarization data, and inadequate image count. Here, we
introduce two spectro-polarimetric datasets: trichromatic Stokes images and
hyperspectral Stokes images. These novel datasets encompass both linear and
circular polarization; they introduce multiple spectral channels; and they
feature a broad selection of real-world scenes. With our dataset in hand, we
analyze the spectro-polarimetric image statistics, develop efficient
representations of such high-dimensional data, and evaluate spectral dependency
of shape-from-polarization methods. As such, the proposed dataset promises a
foundation for data-driven spectro-polarimetric imaging and vision research.
Dataset and code will be publicly available
Multimodal Data Fusion: An Overview of Methods, Challenges and Prospects
International audienceIn various disciplines, information about the same phenomenon can be acquired from different types of detectors, at different conditions, in multiple experiments or subjects, among others. We use the term "modality" for each such acquisition framework. Due to the rich characteristics of natural phenomena, it is rare that a single modality provides complete knowledge of the phenomenon of interest. The increasing availability of several modalities reporting on the same system introduces new degrees of freedom, which raise questions beyond those related to exploiting each modality separately. As we argue, many of these questions, or "challenges" , are common to multiple domains. This paper deals with two key questions: "why we need data fusion" and "how we perform it". The first question is motivated by numerous examples in science and technology, followed by a mathematical framework that showcases some of the benefits that data fusion provides. In order to address the second question, "diversity" is introduced as a key concept, and a number of data-driven solutions based on matrix and tensor decompositions are discussed, emphasizing how they account for diversity across the datasets. The aim of this paper is to provide the reader, regardless of his or her community of origin, with a taste of the vastness of the field, the prospects and opportunities that it holds
Hyperspectral Image Super-Resolution via Dual-domain Network Based on Hybrid Convolution
Since the number of incident energies is limited, it is difficult to directly
acquire hyperspectral images (HSI) with high spatial resolution. Considering
the high dimensionality and correlation of HSI, super-resolution (SR) of HSI
remains a challenge in the absence of auxiliary high-resolution images.
Furthermore, it is very important to extract the spatial features effectively
and make full use of the spectral information. This paper proposes a novel HSI
super-resolution algorithm, termed dual-domain network based on hybrid
convolution (SRDNet). Specifically, a dual-domain network is designed to fully
exploit the spatial-spectral and frequency information among the hyper-spectral
data. To capture inter-spectral self-similarity, a self-attention learning
mechanism (HSL) is devised in the spatial domain. Meanwhile the pyramid
structure is applied to increase the acceptance field of attention, which
further reinforces the feature representation ability of the network. Moreover,
to further improve the perceptual quality of HSI, a frequency loss(HFL) is
introduced to optimize the model in the frequency domain. The dynamic weighting
mechanism drives the network to gradually refine the generated frequency and
excessive smoothing caused by spatial loss. Finally, In order to better fully
obtain the mapping relationship between high-resolution space and
low-resolution space, a hybrid module of 2D and 3D units with progressive
upsampling strategy is utilized in our method. Experiments on a widely used
benchmark dataset illustrate that the proposed SRDNet method enhances the
texture information of HSI and is superior to state-of-the-art methods
Spinning Metasurface Stack for Spectro-polarimetric Thermal Imaging
Spectro-polarimetric imaging in the long-wave infrared (LWIR) region plays a
crucial role in applications from night vision and machine perception to trace
gas sensing and thermography. However, the current generation of
spectro-polarimetric LWIR imagers suffer from limitations in size, spectral
resolution and field of view (FOV). While meta-optics-based strategies for
spectro-polarimetric imaging have been explored in the visible spectrum, their
potential for thermal imaging remains largely unexplored. In this work, we
introduce a novel approach for spectro-polarimetric decomposition by combining
large-area stacked meta-optical devices with advanced computational imaging
algorithms. The co-design of a stack of spinning dispersive metasurfaces along
with compressed sensing and dictionary learning algorithms allows simultaneous
spectral and polarimetric resolution without the need for bulky filter wheels
or interferometers. Our spinning-metasurface-based spectro polarimetric stack
is compact (< 10 x 10 x 10 cm), robust, and offers a wide field of view
(20.5{\deg}). We show that the spectral resolving power of our system
substantially enhances performance in machine learning tasks such as material
classification, a challenge for conventional panchromatic thermal cameras. Our
approach represents a significant advance in the field of thermal imaging for a
wide range of applications including heat-assisted detection and ranging
(HADAR)
- …