3,707 research outputs found
Multimodal Sensor Fusion In Single Thermal image Super-Resolution
With the fast growth in the visual surveillance and security sectors, thermal
infrared images have become increasingly necessary ina large variety of
industrial applications. This is true even though IR sensors are still more
expensive than their RGB counterpart having the same resolution. In this paper,
we propose a deep learning solution to enhance the thermal image resolution.
The following results are given:(I) Introduction of a multimodal,
visual-thermal fusion model that ad-dresses thermal image super-resolution, via
integrating high-frequency information from the visual image. (II)
Investigation of different net-work architecture schemes in the literature,
their up-sampling methods,learning procedures, and their optimization functions
by showing their beneficial contribution to the super-resolution problem. (III)
A bench-mark ULB17-VT dataset that contains thermal images and their visual
images counterpart is presented. (IV) Presentation of a qualitative evaluation
of a large test set with 58 samples and 22 raters which shows that our proposed
model performs better against state-of-the-arts
Concept for classifying facade elements based on material, geometry and thermal radiation using multimodal UAV remote sensing
This paper presents a concept for classification of facade elements, based on the material and the geometry of the elements in addition
to the thermal radiation of the facade with the usage of a multimodal Unmanned Aerial Vehicle (UAV) system. Once the concept is
finalized and functional, the workflow can be used for energy demand estimations for buildings by exploiting existing methods for
estimation of heat transfer coefficient and the transmitted heat loss. The multimodal system consists of a thermal, a hyperspectral and
an optical sensor, which can be operational with a UAV. While dealing with sensors that operate in different spectra and have different
technical specifications, such as the radiometric and the geometric resolution, the challenges that are faced are presented. Addressed
are the different approaches of data fusion, such as image registration, generation of 3D models by performing image matching and the
means for classification based on either the geometry of the object or the pixel values. As a first step towards realizing the concept, the
result from a geometric calibration with a designed multimodal calibration pattern is presented
In-situ surface porosity prediction in DED (directed energy deposition) printed SS316L parts using multimodal sensor fusion
This study aims to relate the time-frequency patterns of acoustic emission
(AE) and other multi-modal sensor data collected in a hybrid directed energy
deposition (DED) process to the pore formations at high spatial (0.5 mm) and
time (< 1ms) resolutions. Adapting an explainable AI method in LIME (Local
Interpretable Model-Agnostic Explanations), certain high-frequency waveform
signatures of AE are to be attributed to two major pathways for pore formation
in a DED process, namely, spatter events and insufficient fusion between
adjacent printing tracks from low heat input. This approach opens an exciting
possibility to predict, in real-time, the presence of a pore in every voxel
(0.5 mm in size) as they are printed, a major leap forward compared to prior
efforts. Synchronized multimodal sensor data including force, AE, vibration and
temperature were gathered while an SS316L material sample was printed and
subsequently machined. A deep convolution neural network classifier was used to
identify the presence of pores on a voxel surface based on time-frequency
patterns (spectrograms) of the sensor data collected during the process chain.
The results suggest signals collected during DED were more sensitive compared
to those from machining for detecting porosity in voxels (classification test
accuracy of 87%). The underlying explanations drawn from LIME analysis suggests
that energy captured in high frequency AE waveforms are 33% lower for porous
voxels indicating a relatively lower laser-material interaction in the melt
pool, and hence insufficient fusion and poor overlap between adjacent printing
tracks. The porous voxels for which spatter events were prevalent during
printing had about 27% higher energy contents in the high frequency AE band
compared to other porous voxels. These signatures from AE signal can further
the understanding of pore formation from spatter and insufficient fusion
Infrared Image Super-Resolution: Systematic Review, and Future Trends
Image Super-Resolution (SR) is essential for a wide range of computer vision
and image processing tasks. Investigating infrared (IR) image (or thermal
images) super-resolution is a continuing concern within the development of deep
learning. This survey aims to provide a comprehensive perspective of IR image
super-resolution, including its applications, hardware imaging system dilemmas,
and taxonomy of image processing methodologies. In addition, the datasets and
evaluation metrics in IR image super-resolution tasks are also discussed.
Furthermore, the deficiencies in current technologies and possible promising
directions for the community to explore are highlighted. To cope with the rapid
development in this field, we intend to regularly update the relevant excellent
work at \url{https://github.com/yongsongH/Infrared_Image_SR_SurveyComment: Submitted to IEEE TNNL
Investigations of closed source registration method of depth sensor technologies for human-robot collaboration
Productive teaming is the new form of human-robot interaction. The multimodal 3D imaging has a key role in this to gain a more comprehensive understanding of production system as well as to enable trustful collaboration from the teams. For a complete scene capture, the registration of the image modalities is required. Currently, low-cost RGB-D sensors are often used. These come with a closed source registration function. In order to have an efficient and freely available method for any sensors, we have developed a new method, called Triangle-Mesh-Rasterization-Projection (TMRP). To verify the performance of our method, we compare it with the closed-source projection function of the Azure Kinect Sensor (Microsoft). The qualitative comparison showed that both methods produce almost identical results. Minimal differences at the edges indicate that our TMRP interpolation is more accurate. With our method, a freely available open-source registration method is now available that can be applied to almost any multimodal 3D/2D image dataset and is not like the Microsoft SDK optimized for Microsoft products
Infrared Image Super-Resolution via GAN
The ability of generative models to accurately fit data distributions has
resulted in their widespread adoption and success in fields such as computer
vision and natural language processing. In this chapter, we provide a brief
overview of the application of generative models in the domain of infrared (IR)
image super-resolution, including a discussion of the various challenges and
adversarial training methods employed. We propose potential areas for further
investigation and advancement in the application of generative models for IR
image super-resolution.Comment: Applications of Generative AI, Chapter 2
- …