On morphological hierarchical representations for image processing and spatial data clustering

Hierarchical data representations in the context of classi cation and data clustering were put forward during the fties. Recently, hierarchical image representations have gained renewed interest for segmentation purposes. In this paper, we briefly survey fundamental results on hierarchical clustering and then detail recent paradigms developed for the hierarchical representation of images in the framework of mathematical morphology: constrained connectivity and ultrametric watersheds. Constrained connectivity can be viewed as a way to constrain an initial hierarchy in such a way that a set of desired constraints are satis ed. The framework of ultrametric watersheds provides a generic scheme for computing any hierarchical connected clustering, in particular when such a hierarchy is constrained. The suitability of this framework for solving practical problems is illustrated with applications in remote sensing

Connected Filtering on Tree-Based Shape-Spaces

International audienceConnected filters are well-known for their good contour preservation property. A popular implementation strategy relies on tree-based image representations: for example, one can compute an attribute characterizing the connected component represented by each node of the tree and keep only the nodes for which the attribute is sufficiently high. This operation can be seen as a thresholding of the tree, seen as a graph whose nodes are weighted by the attribute. Rather than being satisfied with a mere thresholding, we propose to expand on this idea, and to apply connected filters on this latest graph. Consequently, the filtering is performed not in the space of the image, but in the space of shapes built from the image. Such a processing of shape-space filtering is a generalization of the existing tree-based connected operators. Indeed, the framework includes the classical existing connected operators by attributes. It also allows us to propose a class of novel connected operators from the leveling family, based on non-increasing attributes. Finally, we also propose a new class of connected operators that we call morphological shapings. Some illustrations and quantitative evaluations demonstrate the usefulness and robustness of the proposed shape-space filters

A study of observation scales based on Felzenswalb-Huttenlocher dissimilarity measure for hierarchical segmentation

International audienceHierarchical image segmentation provides a region-oriented scale-space, i.e., a set of image segmentations at different detail levels in which the segmentations at finer levels are nested with respect to those at coarser levels. Guimarães et al. proposed a hierarchical graph based image segmentation (HGB) method based on the Felzenszwalb-Huttenlocher dissimilarity. This HGB method computes, for each edge of a graph, the minimum scale in a hierarchy at which two regions linked by this edge should merge according to the dissimilarity. In order to generalize this method, we first propose an algorithm to compute the intervals which contain all the observation scales at which the associated regions should merge. Then, following the current trend in mathematical morphology to study criteria which are not increasing on a hierarchy, we present various strategies to select a significant observation scale in these intervals. We use the BSDS dataset to assess our observation scale selection methods. The experiments show that some of these strategies lead to better segmentation results than the ones obtained with the original HGB method

A graph-based mathematical morphology reader

This survey paper aims at providing a "literary" anthology of mathematical morphology on graphs. It describes in the English language many ideas stemming from a large number of different papers, hence providing a unified view of an active and diverse field of research

Zyklische Levelzeichnungen gerichteter Graphen

The Sugiyama framework proposed in the seminal paper of 1981 is one of the most important algorithms in graph drawing and is widely used for visualizing directed graphs. In its common version, it draws graphs hierarchically and, hence, maps the topological direction to a geometric direction. However, such a hierarchical layout is not possible if the graph contains cycles, which have to be destroyed in a preceding step. In certain application and problem settings, e.g., bio sciences or periodic scheduling problems, it is important that the cyclic structure of the input graph is preserved and clearly visible in drawings. Sugiyama et al. also suggested apart from the nowadays standard horizontal algorithm a cyclic version they called recurrent hierarchies. However, this cyclic drawing style has not received much attention since. In this thesis we consider such cyclic drawings and investigate the Sugiyama framework for this new scenario. As our goal is to visualize cycles directly, the first phase of the Sugiyama framework, which is concerned with removing such cycles, can be neglected. The cyclic structure of the graph leads to new problems in the remaining phases, however, for which solutions are proposed in this thesis. The aim is a complete adaption of the Sugiyama framework for cyclic drawings. To complement our adaption of the Sugiyama framework, we also treat the problem of cyclic level planarity and present a linear time cyclic level planarity testing and embedding algorithm for strongly connected graphs

Un arbre des formes pour les images multivariées

Nowadays, the demand for multi-scale and region-based analysis in many computer vision and pattern recognition applications is obvious. No one would consider a pixel-based approach as a good candidate to solve such problems. To meet this need, the Mathematical Morphology (MM) framework has supplied region-based hierarchical representations of images such as the Tree of Shapes (ToS). The ToS represents the image in terms of a tree of the inclusion of its level-lines. The ToS is thus self-dual and contrast-change invariant which make it well-adapted for high-level image processing. Yet, it is only defined on grayscale images and most attempts to extend it on multivariate images - e.g. by imposing an “arbitrary” total ordering - are not satisfactory. In this dissertation, we present the Multivariate Tree of Shapes (MToS) as a novel approach to extend the grayscale ToS on multivariate images. This representation is a mix of the ToS's computed marginally on each channel of the image; it aims at merging the marginal shapes in a “sensible” way by preserving the maximum number of inclusion. The method proposed has theoretical foundations expressing the ToS in terms of a topographic map of the curvilinear total variation computed from the image border; which has allowed its extension on multivariate data. In addition, the MToS features similar properties as the grayscale ToS, the most important one being its invariance to any marginal change of contrast and any marginal inversion of contrast (a somewhat “self-duality” in the multidimensional case). As the need for efficient image processing techniques is obvious regarding the larger and larger amount of data to process, we propose an efficient algorithm that can be build the MToS in quasi-linear time w.r.t. the number of pixels and quadraticw.r.t. the number of channels. We also propose tree-based processing algorithms to demonstrate in practice, that the MToS is a versatile, easy-to-use, and efficient structure. Eventually, to validate the soundness of our approach, we propose some experiments testing the robustness of the structure to non-relevant components (e.g. with noise or with low dynamics) and we show that such defaults do not affect the overall structure of the MToS. In addition, we propose many real-case applications using the MToS. Many of them are just a slight modification of methods employing the “regular” ToS and adapted to our new structure. For example, we successfully use the MToS for image filtering, image simplification, image segmentation, image classification and object detection. From these applications, we show that the MToS generally outperforms its ToS-based counterpart, demonstrating the potential of our approachDe nombreuses applications issues de la vision par ordinateur et de la reconnaissance des formes requièrent une analyse de l'image multi-échelle basée sur ses régions. De nos jours, personne ne considérerait une approche orientée « pixel » comme une solution viable pour traiter ce genre de problèmes. Pour répondre à cette demande, la Morphologie Mathématique a fourni des représentations hiérarchiques des régions de l'image telles que l'Arbre des Formes (AdF). L'AdF représente l'image par un arbre d'inclusion de ses lignes de niveaux. L'AdF est ainsi auto-dual et invariant au changement de contraste, ce qui fait de lui une structure bien adaptée aux traitements d'images de haut niveau. Néanmoins, il est seulement défini aux images en niveaux de gris et la plupart des tentatives d'extension aux images multivariées (e.g. en imposant un ordre total «arbitraire ») ne sont pas satisfaisantes. Dans ce manuscrit, nous présentons une nouvelle approche pour étendre l'AdF scalaire aux images multivariées : l'Arbre des Formes Multivarié (AdFM). Cette représentation est une « fusion » des AdFs calculés marginalement sur chaque composante de l'image. On vise à fusionner les formes marginales de manière « sensée » en préservant un nombre maximal d'inclusion. La méthode proposée a des fondements théoriques qui consistent en l'expression de l'AdF par une carte topographique de la variation totale curvilinéaire depuis la bordure de l'image. C'est cette reformulation qui a permis l'extension de l'AdF aux données multivariées. De plus, l'AdFM partage des propriétés similaires avec l'AdF scalaire ; la plus importante étant son invariance à tout changement ou inversion de contraste marginal (une sorte d'auto-dualité dans le cas multidimensionnel). Puisqu'il est évident que, vis-à-vis du nombre sans cesse croissant de données à traiter, nous ayons besoin de techniques rapides de traitement d'images, nous proposons un algorithme efficace qui permet de construire l'AdF en temps quasi-linéaire vis-à-vis du nombre de pixels et quadratique vis-à-vis du nombre de composantes. Nous proposons également des algorithmes permettant de manipuler l'arbre, montrant ainsi que, en pratique, l'AdFM est une structure facile à manipuler, polyvalente, et efficace. Finalement, pour valider la pertinence de notre approche, nous proposons quelques expériences testant la robustesse de notre structure aux composantes non-pertinentes (e.g. avec du bruit ou à faible dynamique) et nous montrons que ces défauts n'affectent pas la structure globale de l'AdFM. De plus, nous proposons des applications concrètes utilisant l'AdFM. Certaines sont juste des modifications mineures aux méthodes employant d'ores et déjà l'AdF scalaire mais adaptées à notre nouvelle structure. Par exemple, nous utilisons l'AdFM à des fins de filtrage, segmentation, classification et de détection d'objet. De ces applications, nous montrons ainsi que les méthodes basées sur l'AdFM surpassent généralement leur analogue basé sur l'AdF, démontrant ainsi le potentiel de notre approch

Regularizing Deep Models for Visual Recognition

Image understanding is a shared goal in all computer vision problems. This objective includes decomposing the image into a set of primitive components through which one can perform region segmentation, region labeling, object recognition and finally modeling the interactions between recognized objects. However, due to the large intra-class variations in appearance, shape and structure, extracting image primitives is highly challenging. While images come in the form of intensity matrices, in order to cope with this large variations, a high-level abstraction of images is required. Therefore, the main challenge is to bridge the gap between the low-level pixel representation and the high-level abstract image descriptors. In recent years, we have witnessed a striking popularity of the learned image descriptors using deep networks for visual recognition. The multi-layer architecture of these networks is particularly useful in capturing the hierarchical structure of the image data: simple features are detected at lower layers and fed into higher layers for extracting more complex and abstract representations. Despite the remarkable representational power of deep networks, training these models is computationally expensive. In addition, considering the lack of enough labeled training data in many applications, over-fitting is a serious threat for deep models with large number of free parameters. Also, there are innate issues with the gradient-based optimization procedure used for parameter learning in these models. This research is aimed at addressing the above issues by leveraging domain knowledge. Particularly, we focus on tailoring deep networks for visual recognition through exploiting the characteristics of the image data. These modifications tend to regularize deep models and therefore, improve their generalization performance. We propose novel ways for incorporation of image-specific domain knowledge into deep networks. As part of this thesis, we show how one can significantly decrease the number of free parameters in fully-connected architectures by exploiting the global characteristics of the image data. For convolutional networks, a new multi-neighborhood architecture is introduced which can capture scale-dependent features. In this architecture, the fine-scale image structures (i.e., appearance features) are captured using a small-sized neighborhood while coarse-scale characteristics (i.e., shape features) are detected by considering a wider range area around each pixel. Besides, we propose an effective regularization method for deep networks in which a frequency parameter is devised to specifically treat the issues of gradient-based optimization for training these models. Finally, we introduce a stage-wise training framework for deep networks in which the learning process is broken down into a number of related sub-tasks completed stage-by-stage, where the learned parameters at each stage acts as a prior for the next stage. This goal is achieved through "gradual" injection of the information presented in the training data so that in the early stages of training, the "coarse-scale" properties of the data are captured while the "finer-scale" characteristics are learned in later stages. The performance of the proposed methods are assessed on a number of image classification data sets. Our comprehensive empirical analysis demonstrates that these "regularized" networks offer a better discrimination and generalization performance compared to their domain-oblivious counterparts

Watersheds on edge or node weighted graphs

The literature on the watershed is separated in two families: the watersheds on node weighted graphs and the watersheds on edge weighted graphs. The simplest node weighted graphs are images, where the nodes are the pixels ; neighboring pixels being linked by unweighted pixels. The edge weights on an edge weighted graph express dissimilarities between the unweighted nodes. Distinct definitions of minima and catchment basins have been given for both types of graphs from which different algorithms have been derived. This paper aims at showing that watersheds on edge or node weighted graphs are strictly equivalent. Moreover, all algorithms developed for edge weighted graphs may be applied on node weighted graphs and vice versa. From any node or edge weighted graph it is possible to derive a flooding graph with node and edge weights. Its regional minima and catchment basins are identical whether one considers the node weights alone or the edge weights alone. A lexicographic order relation permits to compare non ascending paths with the same origin according to their steepness. Overlapping zones between neighboring catchment basins are reduced or even suppressed by pruning edges in the flooding graph through which does not pass a steepest path and reduces, without arbitrary choices the overlapping zones between adjacent catchment basins. We propose several ways to break the remaining ties, the simplest being to assign slightly distinct weights to regional minima with the same weight. Like that each node is linked with only one regional minimum by a path of maximal steepness

Advances of Mathematical Morphology and Its Applications in Signal Processing

This thesis describes some advances of Mathematical Morphology (MM), in order to improve the performance of MM filters in I-D signal processing, . especially in the application to power system protection. MM methodologies are founded on set-theoretic concepts and nonlinear superpositions of signals and images. The morphological operations possess outstanding geometrical properties which make it undoubted that they are efficient image processing methods. However in I-D signal processing, MM filters are not widely employed. To explore the applications of MM for I-D signal processing, our contributions in this area can be summarized in the following two aspects. Firstly, the fram.ework of the traditional signal processing methods is based on the frequency domain representation of the signal and the analysis of the operators' transfer function in the frequency domain. But to the morphological operations, their representations in the frequency domain are uncertain. In order to tackle this problem, this thesis presents our attempt to describe the weighted morphological dilation in the frequency domain. Under certain restrictions to the signal and the structuring element, weighted dilation is transformed to a mathematical expression in the frequency domain. Secondly, although the frequency domain analysis plays an important role in signal processing, the geometrical properties of a signal such as the shape of the signal cannot be ignored. MM is an effective method in dealing with such problems. In this thesis, based on the theory of Morphological Wavelet (MW), three multi-resolution signal decomposition schemes are presented. They are Multiresolution Morphological Top-Hat scheme (MMTH), Multi-resolution Morphov logical Gradient scheme (MMG) and Multi-resolution Noise Tolerant Morphological Gradient scheme (MNTMG). The MMTH scheme shows its significant effect in distinguishing symmetrical features from asymmetrical features on the waveform, which owes to its signal analysis operator: morphological Top-Hat transformation, an effective morphological technique. In this thesis, the MMTH scheme is employed in the identification of transformer magnetizing inrush curr~nt from internal fault. Decomposing the signal by MMTH, the asymmetrical features of the inrush waveform are exposed, and the other irrelevant components are attenuated. The MMG scheme adopts morphological gradient, a commonly used operator for edge detection in image and signal processing, as its signal analysis / operator. The MMG scheme bears significant property in characterizing and recognizing the sudden changes with sharp peaks and valleys on the waveform. Furthermore, to the MMG scheme, by decomposing the signal into different levels, the higher the level is processed, the more details of the sudden changes are revealed. In this thesis, the MMG scheme is applied for the design of fault locator of power transmission lines, by extracting the transient features directly from fault-generated transient signals. The MNTMG decomposition scheme can effectively reduce the noise and extract transient features at the same time. In this thesis, the MNTMG scheme is applied to extract the fault generated transient wavefronts from noise imposed signals in the application of fault location of power transmission lines. The proposed contributions focus on the effect of weighted dilation in the frequency domain, constructions of morphological multi-resolution decomposition schemes and their applications in power systems