13 research outputs found
Image Restoration for Remote Sensing: Overview and Toolbox
Remote sensing provides valuable information about objects or areas from a
distance in either active (e.g., RADAR and LiDAR) or passive (e.g.,
multispectral and hyperspectral) modes. The quality of data acquired by
remotely sensed imaging sensors (both active and passive) is often degraded by
a variety of noise types and artifacts. Image restoration, which is a vibrant
field of research in the remote sensing community, is the task of recovering
the true unknown image from the degraded observed image. Each imaging sensor
induces unique noise types and artifacts into the observed image. This fact has
led to the expansion of restoration techniques in different paths according to
each sensor type. This review paper brings together the advances of image
restoration techniques with particular focuses on synthetic aperture radar and
hyperspectral images as the most active sub-fields of image restoration in the
remote sensing community. We, therefore, provide a comprehensive,
discipline-specific starting point for researchers at different levels (i.e.,
students, researchers, and senior researchers) willing to investigate the
vibrant topic of data restoration by supplying sufficient detail and
references. Additionally, this review paper accompanies a toolbox to provide a
platform to encourage interested students and researchers in the field to
further explore the restoration techniques and fast-forward the community. The
toolboxes are provided in https://github.com/ImageRestorationToolbox.Comment: This paper is under review in GRS
A General Destriping Framework for Remote Sensing Images Using Flatness Constraint
This paper proposes a general destriping framework using flatness
constraints, where we can handle various regularization functions in a unified
manner. Removing stripe noise, i.e., destriping, from remote sensing images is
an essential task in terms of visual quality and subsequent processing. Most of
the existing methods are designed by combining a particular image
regularization with a stripe noise characterization that cooperates with the
regularization, which precludes us to examine different regularizations to
adapt to various target images. To resolve this, we formulate the destriping
problem as a convex optimization problem involving a general form of image
regularization and the flatness constraints, a newly introduced stripe noise
characterization. This strong characterization enables us to consistently
capture the nature of stripe noise, regardless of the choice of image
regularization. For solving the optimization problem, we also develop an
efficient algorithm based on a diagonally preconditioned primal-dual splitting
algorithm (DP-PDS), which can automatically adjust the stepsizes. The
effectiveness of our framework is demonstrated through destriping experiments,
where we comprehensively compare combinations of image regularizations and
stripe noise characterizations using hyperspectral images (HSI) and infrared
(IR) videos.Comment: submitted to IEEE Transactions on Geoscience and Remote Sensin
Robust Constrained Hyperspectral Unmixing Using Reconstructed-Image Regularization
Hyperspectral (HS) unmixing is the process of decomposing an HS image into
material-specific spectra (endmembers) and their spatial distributions
(abundance maps). Existing unmixing methods have two limitations with respect
to noise robustness. First, if the input HS image is highly noisy, even if the
balance between sparse and piecewise-smooth regularizations for abundance maps
is carefully adjusted, noise may remain in the estimated abundance maps or
undesirable artifacts may appear. Second, existing methods do not explicitly
account for the effects of stripe noise, which is common in HS measurements, in
their formulations, resulting in significant degradation of unmixing
performance when such noise is present in the input HS image. To overcome these
limitations, we propose a new robust hyperspectral unmixing method based on
constrained convex optimization. Our method employs, in addition to the two
regularizations for the abundance maps, regularizations for the HS image
reconstructed by mixing the estimated abundance maps and endmembers. This
strategy makes the unmixing process much more robust in highly-noisy scenarios,
under the assumption that the abundance maps used to reconstruct the HS image
with desirable spatio-spectral structure are also expected to have desirable
properties. Furthermore, our method is designed to accommodate a wider variety
of noise including stripe noise. To solve the formulated optimization problem,
we develop an efficient algorithm based on a preconditioned primal-dual
splitting method, which can automatically determine appropriate stepsizes based
on the problem structure. Experiments on synthetic and real HS images
demonstrate the advantages of our method over existing methods.Comment: Submitted to IEEE Transactions on Geoscience and Remote Sensin
AI Security for Geoscience and Remote Sensing: Challenges and Future Trends
Recent advances in artificial intelligence (AI) have significantly
intensified research in the geoscience and remote sensing (RS) field. AI
algorithms, especially deep learning-based ones, have been developed and
applied widely to RS data analysis. The successful application of AI covers
almost all aspects of Earth observation (EO) missions, from low-level vision
tasks like super-resolution, denoising and inpainting, to high-level vision
tasks like scene classification, object detection and semantic segmentation.
While AI techniques enable researchers to observe and understand the Earth more
accurately, the vulnerability and uncertainty of AI models deserve further
attention, considering that many geoscience and RS tasks are highly
safety-critical. This paper reviews the current development of AI security in
the geoscience and RS field, covering the following five important aspects:
adversarial attack, backdoor attack, federated learning, uncertainty and
explainability. Moreover, the potential opportunities and trends are discussed
to provide insights for future research. To the best of the authors' knowledge,
this paper is the first attempt to provide a systematic review of AI
security-related research in the geoscience and RS community. Available code
and datasets are also listed in the paper to move this vibrant field of
research forward
Remote Sensing
This dual conception of remote sensing brought us to the idea of preparing two different books; in addition to the first book which displays recent advances in remote sensing applications, this book is devoted to new techniques for data processing, sensors and platforms. We do not intend this book to cover all aspects of remote sensing techniques and platforms, since it would be an impossible task for a single volume. Instead, we have collected a number of high-quality, original and representative contributions in those areas
Visual image processing in various representation spaces for documentary preservation
This thesis establishes an advanced image processing framework for the enhancement and restoration of historical document images (HDI) in both intensity (gray-scale or color) and multispectral (MS) representation spaces. It provides three major contributions: 1) the binarization of gray-scale HDI; 2) the visual quality restoration of MS HDI; and 3) automatic reference data (RD) estimation for HDI binarization. HDI binarization is one of the enhancement techniques that produces bi-level information which is easy to handle using methods of analysis (OCR, for instance) and is less computationally costly to process than 256 levels of grey or color images. Restoring the visual quality of HDI in an MS representation space enhances their legibility, which is not possible with conventional intensity-based restoration methods, and HDI legibility is the main concern of historians and librarians wishing to transfer knowledge and revive ancient cultural heritage. The use of MS imaging systems is a new and attractive research trend in the field of numerical processing of cultural heritage documents. In this thesis, these systems are also used for automatically estimating more accurate RD to be used for the evaluation of HDI binarization algorithms in order to track the level of human performance.
Our first contribution, which is a new adaptive method of intensity-based binarization, is defined at the outset. Since degradation is present over document images, binarization methods must be adapted to handle degradation phenomena locally. Unfortunately, these methods are not effective, as they are not able to capture weak text strokes, which results in a deterioration of the performance of character recognition engines. The proposed approach first detects a subset of the most probable text pixels, which are used to locally estimate the parameters of the two classes of pixels (text and background), and then performs a simple maximum likelihood (ML) to locally classify the remaining pixels based on their class membership. To the best of our knowledge, this is the first time local parameter estimation and classification in an ML framework has been introduced for HDI binarization with promising results. A limitation of this method in the case with as the intensity-based methods of enhancement is that they are not effective in dealing with severely degraded HDI. Developing more advanced methods based on MS information would be a promising alternative avenue of research.
In the second contribution, a novel approach to the visual restoration of HDI is defined. The approach is aimed at providing end users (historians, librarians, etc..) with better HDI visualization, specifically; it aims to restore them from degradations, while keeping the original appearance of the HDI intact. Practically, this problem cannot be solved by conventional intensity-based restoration methods. To cope with these limitations, MS imaging is used to produce additional spectral images in the invisible light (infrared and ultraviolet) range, which gives greater contrast to objects in the documents. The inpainting-based variational framework proposed here for HDI restoration involves isolating the degradation phenomena in the infrared spectral images, and then inpainting them in the visible spectral images. The final color image to visualize is therefore reconstructed from the restored visible spectral images. To the best of our knowledge, this is the first time the inpainting technique has been introduced for MS HDI. The experimental results are promising, and our objective, in collaboration with the BAnQ (Bibliothèque et Archives nationales de Québec), is to push heritage documents into the public domain and build an intelligent engine for accessing them. It is useful to note that the proposed model can be extended to other MS-based image processing tasks.
Our third contribution is presented, which is to consider a new problem of RD (reference data) estimation, in order to show the importance of working with MS images rather than gray-scale or color images. RDs are mandatory for comparing different binarization algorithms, and they are usually generated by an expert. However, an expert’s RD is always subject to mislabeling and judgment errors, especially in the case of degraded data in restricted representation spaces (gray-scale or color images). In the proposed method, multiple RD generated by several experts are used in combination with MS HDI to estimate new, more accurate RD. The idea is to include the agreement of experts about labels and the multivariate data fidelity in a single Bayesian classification framework to estimate the a posteriori probability of new labels forming the final estimated RD. Our experiments show that estimated RD are more accurate than an expert’s RD. To the best of our knowledge, no similar work to combine binary data and multivariate data for the estimation of RD has been conducted
Integration Frameworks for Merging Satellite Remote Sensing Observations with Hydrological Model Outputs
With a growing number of available datasets especially from satellite remote sensing, there is a great opportunity to improve our knowledge of hydrological processes by integrating them with hydrological models. In this regard, data assimilation technique can be used to constrain the dynamic of a model with available observations in order to improve its estimates. In this thesis, a comprehensive data assimilation framework containing multiple stages is proposed and tested over various areas