Generalizable automated pixel-level structural segmentation of medical and biological data

Over the years, the rapid expansion in imaging techniques and equipments has driven the demand for more automation in handling large medical and biological data sets. A wealth of approaches have been suggested as optimal solutions for their respective imaging types. These solutions span various image resolutions, modalities and contrast (staining) mechanisms. Few approaches generalise well across multiple image types, contrasts or resolution. This thesis proposes an automated pixel-level framework that addresses 2D, 2D+t and 3D structural segmentation in a more generalizable manner, yet has enough adaptability to address a number of specific image modalities, spanning retinal funduscopy, sequential fluorescein angiography and two-photon microscopy. The pixel-level segmentation scheme involves: i ) constructing a phase-invariant orientation field of the local spatial neighbourhood; ii ) combining local feature maps with intensity-based measures in a structural patch context; iii ) using a complex supervised learning process to interpret the combination of all the elements in the patch in order to reach a classification decision. This has the advantage of transferability from retinal blood vessels in 2D to neural structures in 3D. To process the temporal components in non-standard 2D+t retinal angiography sequences, we first introduce a co-registration procedure: at the pairwise level, we combine projective RANSAC with a quadratic homography transformation to map the coordinate systems between any two frames. At the joint level, we construct a hierarchical approach in order for each individual frame to be registered to the global reference intra- and inter- sequence(s). We then take a non-training approach that searches in both the spatial neighbourhood of each pixel and the filter output across varying scales to locate and link microvascular centrelines to (sub-) pixel accuracy. In essence, this \link while extract" piece-wise segmentation approach combines the local phase-invariant orientation field information with additional local phase estimates to obtain a soft classification of the centreline (sub-) pixel locations. Unlike retinal segmentation problems where vasculature is the main focus, 3D neural segmentation requires additional exibility, allowing a variety of structures of anatomical importance yet with different geometric properties to be differentiated both from the background and against other structures. Notably, cellular structures, such as Purkinje cells, neural dendrites and interneurons, all display certain elongation along their medial axes, yet each class has a characteristic shape captured by an orientation field that distinguishes it from other structures. To take this into consideration, we introduce a 5D orientation mapping to capture these orientation properties. This mapping is incorporated into the local feature map description prior to a learning machine. Extensive performance evaluations and validation of each of the techniques presented in this thesis is carried out. For retinal fundus images, we compute Receiver Operating Characteristic (ROC) curves on existing public databases (DRIVE & STARE) to assess and compare our algorithms with other benchmark methods. For 2D+t retinal angiography sequences, we compute the error metrics ("Centreline Error") of our scheme with other benchmark methods. For microscopic cortical data stacks, we present segmentation results on both surrogate data with known ground-truth and experimental rat cerebellar cortex two-photon microscopic tissue stacks.Open Acces

A discrete graph Laplacian for signal processing

In this thesis we exploit diffusion processes on graphs to effect two fundamental problems of image processing: denoising and segmentation. We treat these two low-level vision problems on the pixel-wise level under a unified framework: a graph embedding. Using this framework opens us up to the possibilities of exploiting recently introduced algorithms from the semi-supervised machine learning literature. We contribute two novel edge-preserving smoothing algorithms to the literature. Furthermore we apply these edge-preserving smoothing algorithms to some computational photography tasks. Many recent computational photography tasks require the decomposition of an image into a smooth base layer containing large scale intensity variations and a residual layer capturing fine details. Edge-preserving smoothing is the main computational mechanism in producing these multi-scale image representations. We, in effect, introduce a new approach to edge-preserving multi-scale image decompositions. Where as prior approaches such as the Bilateral filter and weighted-least squares methods require multiple parameters to tune the response of the filters our method only requires one. This parameter can be interpreted as a scale parameter. We demonstrate the utility of our approach by applying the method to computational photography tasks that utilise multi-scale image decompositions. With minimal modification to these edge-preserving smoothing algorithms we show that we can extend them to produce interactive image segmentation. As a result the operations of segmentation and denoising are conducted under a unified framework. Moreover we discuss how our method is related to region based active contours. We benchmark our proposed interactive segmentation algorithms against those based upon energy-minimisation, specifically graph-cut methods. We demonstrate that we achieve competitive performance

Modèle de la diffusion pour l'amélioration de la qualité de la vidéo : débruitage et constance des couleurs

De nombreux problèmes en vision artificielle s’apparentent à la diffusion. Ils sont en général posés sous la forme d’équations aux dérivées partielles (EDP) qui expriment un problème physique et qui doivent être résolues dans l’ensemble du domaine, image ou vidéo. Une grande classe d’EDP est constituée par les processus de diffusion, assimilables à des phénomènes de transfert d’énergie dans le milieu étudié. Les méthodes classiques pour résoudre ces problèmes utilisent des processus locaux purement mathématiques, qui ne permettent pas d’approche intuitive possible. Dans ce mémoire, nous présentons une extension au domaine vidéo de travaux visant à dépasser les limitations des modèles mathématiques locaux en résolvant le problème physique global plutôt que les EDP, grâce à un modèle image basé sur la Topologie Algébrique Calculatoire (TAC). Cette approche consiste à extraire du problème modélisé les lois élémentaires qui le composent grâce à des analogies dans le domaine de la Physique. Ces lois sont décomposées en deux classes: les lois de conservation sont exprimées sans approximation grâce au support topologique du modèle, alors que les lois constitutives nécessitent des approximations qui peuvent être choisies en fonction du comportement souhaité de l’algorithme. Une méthodologie de résolution des problèmes de diffusion vidéo est présentée, ce qui amène à deux applications: le débruitage vidéo et la constance de la couleur selon le modèle retinex, pour la vidéo. Des résultats expérimentaux valident ces applications

Patient-specific modeling of the biomechanics of vulnerable coronary artery plaques

Coronary artery atherosclerosis is a local, multifactorial, complex disease, and the leading cause of death in the US. Complex interactions between biochemical transport and biomechanical forces influence disease growth. Wall shear stress (WSS) affects coronary artery atherosclerosis by inducing endothelial cell mechanotransduction and by controlling the near- wall transport processes involved in atherosclerosis. The current management guidelines for detection of atherosclerotic plaques focus on morphological characterizations and the blockage percentage of the stenosis based on coronary computed tomography angiography (CCTA). Despite the progress achieved in therapeutics, the relation between hemodynamic environment and the composition of atherosclerotic plaques remains unexplored. This dissertation is divided into two main sections: the association between hemodynamics/biotransport and longitudinal changes in the plaque vulnerability characteristics and developing a 1D automatic vascular network generation package with the ability to be coupled with a 3D patient-specific model. Biochemical-specific mass transport models were developed to study low-density lipoprotein, nitric oxide, adenosine triphosphate, oxygen, monocyte chemoattractant protein-1, and monocyte transport. The transport results were compared with WSS vectors and WSS Lagrangian coherent structures (WSS LCS). High WSS magnitude protected against atherosclerosis by increasing the production or flux of atheroprotective biochemicals and decreasing the near-wall localization of atherogenic biochemicals. Low WSS magnitude promoted atherosclerosis by increasing atherogenic biochemical localization. To find the association between hemodynamics/biotransport and longitudinal changes in the atherosclerotic plaque characteristics, a plaque quantification software was developed with the aim of performing a segment-specific assessment to accurately calculate the volumes of low attenuation plaque (LAP), fibrous plaque (FP), calcium plaque (CP), and vessel wall and identify the quantitative plaque characteristics including spotty calcification, presence of napkin-ring sign, and positive remodeling. The changes in the different plaque characteristics were compared against the hemodynamic/biotransport parameters. The results showed that WSS magnitude is moderately correlated with the longitudinal changes in LAP, FP, and vessel wall volumes. Also, WSS magnitude and local concentration of nitric oxide (NO) showed a meaningful correlation with the presence of positive remodeling in the follow-up. A hybrid 1D-3D solver was developed in Simvascular software and validated against the existing data in the literature. The results of our coupled 1D-3D solver showed a good agreement with the 3D, deformable wall models. This solver can be used to solve the blood flow in a large network of 1D vessels coupled with a patient-specific 3D model. Finally, an automatic vascular network generation framework was developed using the Constraint Constructive Optimization (CCO) algorithm to study the generation of arterial trees based on theoretical perfusion maps. The algorithm simulated angiogenesis by optimizing the total vessel volume governed by physiological and geometrical constraints

Patch-based semantic labelling of images.

PhDThe work presented in this thesis is focused at associating a semantics to the content of an image, linking the content to high level semantic categories. The process can take place at two levels: either at image level, towards image categorisation, or at pixel level, in se- mantic segmentation or semantic labelling. To this end, an analysis framework is proposed, and the different steps of part (or patch) extraction, description and probabilistic modelling are detailed. Parts of different nature are used, and one of the contributions is a method to complement information associated to them. Context for parts has to be considered at different scales. Short range pixel dependences are accounted by associating pixels to larger patches. A Conditional Random Field, that is, a probabilistic discriminative graphical model, is used to model medium range dependences between neighbouring patches. Another contribution is an efficient method to consider rich neighbourhoods without having loops in the inference graph. To this end, weak neighbours are introduced, that is, neighbours whose label probability distribution is pre-estimated rather than mutable during the inference. Longer range dependences, that tend to make the inference problem intractable, are addressed as well. A novel descriptor based on local histograms of visual words has been proposed, meant to both complement the feature descriptor of the patches and augment the context awareness in the patch labelling process. Finally, an alternative approach to consider multiple scales in a hierarchical framework based on image pyramids is proposed. An image pyramid is a compositional representation of the image based on hierarchical clustering. All the presented contributions are extensively detailed throughout the thesis, and experimental results performed on publicly available datasets are reported to assess their validity. A critical comparison with the state of the art in this research area is also presented, and the advantage in adopting the proposed improvements are clearly highlighted

Hyperspectral image representation and processing with binary partition trees

The optimal exploitation of the information provided by hyperspectral images requires the development of advanced image processing tools. Therefore, under the title Hyperspectral image representation and Processing with Binary Partition Trees, this PhD thesis proposes the construction and the processing of a new region-based hierarchical hyperspectral image representation: the Binary Partition Tree (BPT). This hierarchical region-based representation can be interpreted as a set of hierarchical regions stored in a tree structure. Hence, the Binary Partition Tree succeeds in presenting: (i) the decomposition of the image in terms of coherent regions and (ii) the inclusion relations of the regions in the scene. Based on region-merging techniques, the construction of BPT is investigated in this work by studying hyperspectral region models and the associated similarity metrics. As a matter of fact, the very high dimensionality and the complexity of the data require the definition of specific region models and similarity measures. Once the BPT is constructed, the fixed tree structure allows implementing efficient and advanced application-dependent techniques on it. The application-dependent processing of BPT is generally implemented through a specific pruning of the tree. Accordingly, some pruning techniques are proposed and discussed according to different applications. This Ph.D is focused in particular on segmentation, object detection and classification of hyperspectral imagery. Experimental results on various hyperspectral data sets demonstrate the interest and the good performances of the BPT representatio