Fusion of MultiSpectral and Panchromatic Images Based on Morphological Operators

International audienceNonlinear decomposition schemes constitute an alternative to classical approaches for facing the problem of data fusion. In this paper we discuss the application of this methodology to a popular remote sensing application called pansharpening, which consists in the fusion of a low resolution multispectral image and a high resolution panchromatic image. We design a complete pansharpening scheme based on the use of morphological half gradients operators and demonstrate the suitability of this algorithm through the comparison with state of the art approaches. Four datasets acquired by the Pleiades, Worldview-2, Ikonos and Geoeye-1 satellites are employed for the performance assessment, testifying the effectiveness of the proposed approach in producing top-class images with a setting independent of the specific sensor

Contributions en morphologie mathématique pour l'analyse d'images multivariées

This thesis contributes to the field of mathematical morphology and illustrates how multivariate statistics and machine learning techniques can be exploited to design vector ordering and to include results of morphological operators in the pipeline of multivariate image analysis. In particular, we make use of supervised learning, random projections, tensor representations and conditional transformations to design new kinds of multivariate ordering, and morphological filters for color and multi/hyperspectral images. Our key contributions include the following points:• Exploration and analysis of supervised ordering based on kernel methods.• Proposition of an unsupervised ordering based on statistical depth function computed by random projections. We begin by exploring the properties that an image requires to ensure that the ordering and the associated morphological operators can be interpreted in a similar way than in the case of grey scale images. This will lead us to the notion of background/foreground decomposition. Additionally, invariance properties are analyzed and theoretical convergence is showed.• Analysis of supervised ordering in morphological template matching problems, which corresponds to the extension of hit-or-miss operator to multivariate image by using supervised ordering.• Discussion of various strategies for morphological image decomposition, specifically, the additive morphological decomposition is introduced as an alternative for the analysis of remote sensing multivariate images, in particular for the task of dimensionality reduction and supervised classification of hyperspectral remote sensing images.• Proposition of an unified framework based on morphological operators for contrast enhancement and salt- and-pepper denoising.• Introduces a new framework of multivariate Boolean models using a complete lattice formulation. This theoretical contribution is useful for characterizing and simulation of multivariate textures.Cette thèse contribue au domaine de la morphologie mathématique et illustre comment la statistique multivariée et les techniques d'apprentissage numérique peuvent être exploitées pour concevoir un ordre dans l'espace des vecteurs et pour inclure les résultats d'opérateurs morphologiques au processus d'analyse d'images multivariées. En particulier, nous utilisons l'apprentissage supervisé, les projections aléatoires, les représentations tensorielles et les transformations conditionnelles pour concevoir de nouveaux types d'ordres multivariés et de nouveaux filtres morphologiques pour les images multi/hyperspectrales. Nos contributions clés incluent les points suivants :• Exploration et analyse d'ordre supervisé, basé sur les méthodes à noyaux.• Proposition d'un ordre nonsupervisé, basé sur la fonction de profondeur statistique calculée par projections aléatoires. Nous commençons par explorer les propriétés nécessaires à une image pour assurer que l'ordre ainsi que les opérateurs morphologiques associés, puissent être interprétés de manière similaire au cas d'images en niveaux de gris. Cela nous amènera à la notion de décomposition en arrière plan. De plus, les propriétés d'invariance sont analysées et la convergence théorique est démontrée.• Analyse de l'ordre supervisé dans les problèmes de correspondance morphologique de patrons, qui correspond à l'extension de l'opérateur tout-ou-rien aux images multivariées grâce à l‘utilisation de l'ordre supervisé.• Discussion sur différentes stratégies pour la décomposition morphologique d'images. Notamment, la décomposition morphologique additive est introduite comme alternative pour l'analyse d'images de télédétection, en particulier pour les tâches de réduction de dimension et de classification supervisée d'images hyperspectrales de télédétection.• Proposition d'un cadre unifié basé sur des opérateurs morphologiques, pour l'amélioration de contraste et pour le filtrage du bruit poivre-et-sel.• Introduction d'un nouveau cadre de modèles Booléens multivariés en utilisant une formulation en treillis complets. Cette contribution théorique est utile pour la caractérisation et la simulation de textures multivariées

Proceedings of the FEniCS Conference 2017

Proceedings of the FEniCS Conference 2017 that took place 12-14 June 2017 at the University of Luxembourg, Luxembourg

Classification and Segmentation of Galactic Structuresin Large Multi-spectral Images

Extensive and exhaustive cataloguing of astronomical objects is imperative for studies seeking to understand mechanisms which drive the universe. Such cataloguing tasks can be tedious, time consuming and demand a high level of domain specific knowledge. Past astronomical imaging surveys have been catalogued through mostly manual effort. Immi-nent imaging surveys, however, will produce a magnitude of data that cannot be feasibly processed through manual cataloguing. Furthermore, these surveys will capture objects fainter than the night sky, termed low surface brightness objects, and at unprecedented spatial resolution owing to advancements in astronomical imaging. In this thesis, we in-vestigate the use of deep learning to automate cataloguing processes, such as detection, classification and segmentation of objects. A common theme throughout this work is the adaptation of machine learning methods to challenges specific to the domain of low surface brightness imaging.We begin with creating an annotated dataset of structures in low surface brightness images. To facilitate supervised learning in neural networks, a dataset comprised of input and corresponding ground truth target labels is required. An online tool is presented, allowing astronomers to classify and draw over objects in large multi-spectral images. A dataset produced using the tool is then detailed, containing 227 low surface brightness images from the MATLAS survey and labels made by four annotators. We then present a method for synthesising images of galactic cirrus which appear similar to MATLAS images, allowing pretraining of neural networks.A method for integrating sensitivity to orientation in convolutional neural networks is then presented. Objects in astronomical images can present in any given orientation, and thus the ability for neural networks to handle rotations is desirable. We modify con-volutional filters with sets of Gabor filters with different orientations. These orientations are learned alongside network parameters during backpropagation, allowing exact optimal orientations to be captured. The method is validated extensively on multiple datasets and use cases.We propose an attention based neural network architecture to process global contami-nants in large images. Performing analysis of low surface brightness images requires plenty of contextual information and local textual patterns. As a result, a network for processing low surface brightness images should ideally be able to accommodate large high resolu-tion images without compromising on either local or global features. We utilise attention to capture long range dependencies, and propose an efficient attention operator which significantly reduces computational cost, allowing the input of large images. We also use Gabor filters to build an attention mechanism to better capture long range orientational patterns. These techniques are validated on the task of cirrus segmentation in MAT-LAS images, and cloud segmentation on the SWIMSEG database, where state of the art performance is achieved.Following, cirrus segmentation in MATLAS images is further investigated, and a com-prehensive study is performed on the task. We discuss challenges associated with cirrus segmentation and low surface brightness images in general, and present several tech-niques to accommodate them. A novel loss function is proposed to facilitate training of the segmentation model on probabilistic targets. Results are presented on the annotated MATLAS images, with extensive ablation studies and a final benchmark to test the limits of the detailed segmentation pipeline.Finally, we develop a pipeline for multi-class segmentation of galactic structures and surrounding contaminants. Techniques of previous chapters are combined with a popu-lar instance segmentation architecture to create a neural network capable of segmenting localised objects and extended amorphous regions. The process of data preparation for training instance segmentation models is thoroughly detailed. The method is tested on segmentation of five object classes in MATLAS images. We find that unifying the tasks of galactic structure segmentation and contaminant segmentation improves model perfor-mance in comparison to isolating each task

Geometric Surface Processing and Virtual Modeling

In this work we focus on two main topics "Geometric Surface Processing" and "Virtual Modeling". The inspiration and coordination for most of the research work contained in the thesis has been driven by the project New Interactive and Innovative Technologies for CAD (NIIT4CAD), funded by the European Eurostars Programme. NIIT4CAD has the ambitious aim of overcoming the limitations of the traditional approach to surface modeling of current 3D CAD systems by introducing new methodologies and technologies based on subdivision surfaces in a new virtual modeling framework. These innovations will allow designers and engineers to transform quickly and intuitively an idea of shape in a high-quality geometrical model suited for engineering and manufacturing purposes. One of the objective of the thesis is indeed the reconstruction and modeling of surfaces, representing arbitrary topology objects, starting from 3D irregular curve networks acquired through an ad-hoc smart-pen device. The thesis is organized in two main parts: "Geometric Surface Processing" and "Virtual Modeling". During the development of the geometric pipeline in our Virtual Modeling system, we faced many challenges that captured our interest and opened new areas of research and experimentation. In the first part, we present these theories and some applications to Geometric Surface Processing. This allowed us to better formalize and give a broader understanding on some of the techniques used in our latest advancements on virtual modeling and surface reconstruction. The research on both topics led to important results that have been published and presented in articles and conferences of international relevance

REAL-TIME 4D ULTRASOUND RECONSTRUCTION FOR IMAGE-GUIDED INTRACARDIAC INTERVENTIONS

Image-guided therapy addresses the lack of direct vision associated with minimally- invasive interventions performed on the beating heart, but requires effective intraoperative imaging. Gated 4D ultrasound reconstruction using a tracked 2D probe generates a time-series of 3D images representing the beating heart over the cardiac cycle. These images have a relatively high spatial resolution and wide field of view, and ultrasound is easily integrated into the intraoperative environment. This thesis presents a real-time 4D ultrasound reconstruction system incorporated within an augmented reality environment for surgical guidance, whose incremental visualization reduces common acquisition errors. The resulting 4D ultrasound datasets are intended for visualization or registration to preoperative images. A human factors experiment demonstrates the advantages of real-time ultrasound reconstruction, and accuracy assessments performed both with a dynamic phantom and intraoperatively reveal RMS localization errors of 2.5-2.7 mm, and 0.8 mm, respectively. Finally, clinical applicability is demonstrated by both porcine and patient imaging

Transition Path Theory for Markov Processes

In this thesis, we present the framework of transition path theory (TPT) for time continuous Markov processes with continuous and discrete state space. TPT provides statistical properties of the ensemble of reactive trajectories between some start and target sets and yields properties such as the committor function, the probability distribution of the reactive trajectories, their probability current and their rate of occurrence. We shown that knowing these objects allows one to arrive at a complete understanding of the mechanism of the reaction. The main objects of TPT for Markov diffusion processes are explicitly derived for the Langevin and Smoluchowski dynamics and illustrate them on a various number of low-dimensional examples. Despite the simplicity of these examples compared to those encountered in real applications, they already demonstrate the ability of TPT to handle complex dynamical scenarios. The main challenge in TPT for diffusion processes is the numerical computation of the committor function as a solution of a Dirichlet-Neumann boundary value problem involving the generator of the process. Beside the derivation of TPT for Markov jump processes, we focus on the development of efficient graph algorithms to determine reaction pathways in discrete state space. One approach via shortest-path algorithms turns out to give only a rough picture of possible reaction channels whereas the network approach allows a hierarchical decomposition of the set of reaction pathways such that the dominant channels can be identified. We successfully apply the latter approach to an example of conformational dynamics of a bio-molecule. In particular, we make use of a maximum likelihood method to estimate the infinitesimal generator of a jump process from an incomplete observation. Finally, we address the question of error propagation in the committor function computation for Markov chains