Multi-object segmentation using coupled nonparametric shape and relative pose priors

We present a new method for multi-object segmentation in a maximum a posteriori estimation framework. Our method is motivated by the observation that neighboring or coupling objects in images generate configurations and co-dependencies which could potentially aid in segmentation if properly exploited. Our approach employs coupled shape and inter-shape pose priors that are computed using training images in a nonparametric multi-variate kernel density estimation framework. The coupled shape prior is obtained by estimating the joint shape distribution of multiple objects and the inter-shape pose priors are modeled via standard moments. Based on such statistical models, we formulate an optimization problem for segmentation, which we solve by an algorithm based on active contours. Our technique provides significant improvements in the segmentation of weakly contrasted objects in a number of applications. In particular for medical image analysis, we use our method to extract brain Basal Ganglia structures, which are members of a complex multi-object system posing a challenging segmentation problem. We also apply our technique to the problem of handwritten character segmentation. Finally, we use our method to segment cars in urban scenes

Volumetric segmentation of multiple basal ganglia structures

We present a new active contour-based, statistical method for simultaneous volumetric segmentation of multiple subcortical structures in the brain. Neighboring anatomical structures in the human brain exhibit co-dependencies which can aid in segmentation, if properly analyzed and modeled. Motivated by this observation, we formulate the segmentation problem as a maximum a posteriori estimation problem, in which we incorporate statistical prior models on the shapes and inter-shape (relative) poses of the structures of interest. This provides a principled mechanism to bring high level information about the shapes and the relationships of anatomical structures into the segmentation problem. For learning the prior densities based on training data, we use a nonparametric multivariate kernel density estimation framework. We combine these priors with data in a variational framework, and develop an active contour-based iterative segmentation algorithm. We test our method on the problem of volumetric segmentation of basal ganglia structures in magnetic resonance (MR) images. We compare our technique with existing methods and demonstrate the improvements it provides in terms of segmentation accuracy

Coupled non-parametric shape and moment-based inter-shape pose priors for multiple basal ganglia structure segmentation

This paper presents a new active contour-based, statistical method for simultaneous volumetric segmentation of multiple subcortical structures in the brain. In biological tissues, such as the human brain, neighboring structures exhibit co-dependencies which can aid in segmentation, if properly analyzed and modeled. Motivated by this observation, we formulate the segmentation problem as a maximum a posteriori estimation problem, in which we incorporate statistical prior models on the shapes and inter-shape (relative) poses of the structures of interest. This provides a principled mechanism to bring high level information about the shapes and the relationships of anatomical structures into the segmentation problem. For learning the prior densities we use a nonparametric multivariate kernel density estimation framework. We combine these priors with data in a variational framework and develop an active contour-based iterative segmentation algorithm. We test our method on the problem of volumetric segmentation of basal ganglia structures in magnetic resonance (MR) images. We present a set of 2D and 3D experiments as well as a quantitative performance analysis. In addition, we perform a comparison to several existent segmentation methods and demonstrate the improvements provided by our approach in terms of segmentation accuracy

Coupled nonparametric shape priors for segmentation of multiple basal ganglia structures

This paper presents a new method for multiple structure segmentation, using a maximum a posteriori (MAP) estimation framework, based on prior shape densities involving nonparametric multivariate kernel density estimation of multiple shapes. Our method is motivated by the observation that neighboring or coupling structures in medical images generate configurations and co-dependencies which could potentially aid in segmentation if properly exploited. Our technique allows simultaneous segmentation of multiple objects, where highly contrasted, easy-to-segment structures can help improve the segmentation of weakly contrasted objects. We demonstrate the effectiveness of our method on both synthetic images and real magnetic resonance images (MRI) for segmentation of basal ganglia structures

Coupled non-parametric shape and moment-based inter-shape pose priors for multiple basal ganglia structure segmentation

Brain tissue and structure segmentation in magnetic resonance (MR) images is a fundamental problem in clinical studies of brain structure and function. Due to limitations such as low contrast, partial volume effects, and field inhomogeneities, the delineation of subcortical (basal ganglia) structures such as caudate nucleus, putamen, and thalamus from white matter, gray matter and cerebrospinal fluid (CSF) is a very challenging problem. This thesis presents a new method for simultaneous segmentation of multiple brain structures. We formulate the segmentation problem as a maximum a posteriori estimation problem, in which we incorporate statistical prior models on the shapes and relative poses of the structures of interest. Our method is motivated by the observation that neighboring or coupling structures in medical images generate configurations and co-dependencies which could potentially aid in segmentation if properly exploited. Our coupled shape priors are learned through nonparametric multivariate kernel density estimation based on training data. Relative pose priors are modeled via standard moments. Given this framework, the segmentation problems turns into an optimization problem, which we solve using active contours. We present experimental results on synthetic data as well as on a rich set of real MR images demonstrating the effectiveness of the proposed method in segmenting basal ganglia structures as well as improvements it provides over existing approaches

Computational Anatomy for Multi-Organ Analysis in Medical Imaging: A Review

The medical image analysis field has traditionally been focused on the development of organ-, and disease-specific methods. Recently, the interest in the development of more 20 comprehensive computational anatomical models has grown, leading to the creation of multi-organ models. Multi-organ approaches, unlike traditional organ-specific strategies, incorporate inter-organ relations into the model, thus leading to a more accurate representation of the complex human anatomy. Inter-organ relations are not only spatial, but also functional and physiological. Over the years, the strategies 25 proposed to efficiently model multi-organ structures have evolved from the simple global modeling, to more sophisticated approaches such as sequential, hierarchical, or machine learning-based models. In this paper, we present a review of the state of the art on multi-organ analysis and associated computation anatomy methodology. The manuscript follows a methodology-based classification of the different techniques 30 available for the analysis of multi-organs and multi-anatomical structures, from techniques using point distribution models to the most recent deep learning-based approaches. With more than 300 papers included in this review, we reflect on the trends and challenges of the field of computational anatomy, the particularities of each anatomical region, and the potential of multi-organ analysis to increase the impact of 35 medical imaging applications on the future of healthcare.Comment: Paper under revie

Multiphase Geometric Couplings for the Segmentation of Neural Processes

The ability to constrain the geometry of deformable models for image segmentation can be useful when information about the expected shape or positioning of the objects in a scene is known a priori. An example of this occurs when segmenting neural cross sections in electron microscopy. Such images often contain multiple nested boundaries separating regions of homogeneous intensities. For these applications, multiphase level sets provide a partitioning framework that allows for the segmentation of multiple deformable objects by combining several level set functions. Although there has been much effort in the study of statistical shape priors that can be used to constrain the geometry of each partition, none of these methods allow for the direct modeling of geometric arrangements of partitions. In this paper, we show how to define elastic couplings between multiple level set functions to model ribbon-like partitions. We build such couplings using dynamic force fields that can depend on the image content and relative location and shape of the level set functions. To the best of our knowledge, this is the first work that shows a direct way of geometrically constraining multiphase level sets for image segmentation. We demonstrate the robustness of our method by comparing it with previous level set segmentation methods.Engineering and Applied Science

Facial Point Detection using Boosted Regression and Graph Models

Finding fiducial facial points in any frame of a video showing rich naturalistic facial behaviour is an unsolved problem. Yet this is a crucial step for geometric-featurebased facial expression analysis, and methods that use appearance-based features extracted at fiducial facial point locations. In this paper we present a method based on a combination of Support Vector Regression and Markov Random Fields to drastically reduce the time needed to search for a point’s location and increase the accuracy and robustness of the algorithm. Using Markov Random Fields allows us to constrain the search space by exploiting the constellations that facial points can form. The regressors on the other hand learn a mapping between the appearance of the area surrounding a point and the positions of these points, which makes detection of the points very fast and can make the algorithm robust to variations of appearance due to facial expression and moderate changes in head pose. The proposed point detection algorithm was tested on 1855 images, the results of which showed we outperform current state of the art point detectors

Robust multimodal dense SLAM

To enable increasingly intelligent behaviours, autonomous robots will need to be equipped with a deep understanding of their surrounding environment. It would be particularly desirable if this level of perception could be achieved automatically through the use of vision-based sensing, as passive cameras make a compelling sensor choice for robotic platforms due to their low cost, low weight, and low power consumption. Fundamental to extracting a high-level understanding from a set of 2D images is an understanding of the underlying 3D geometry of the environment. In mobile robotics, the most popular and successful technique for building a representation of 3D geometry from 2D images is Visual Simultaneous Localisation and Mapping (SLAM). While sparse, landmark-based SLAM systems have demonstrated high levels of accuracy and robustness, they are only capable of producing sparse maps. In general, to move beyond simple navigation to scene understanding and interaction, dense 3D reconstructions are required. Dense SLAM systems naturally allow for online dense scene reconstruction, but suffer from a lack of robustness due to the fact that the dense image alignment used in the tracking step has a narrow convergence basin and that the photometric-based depth estimation used in the mapping step is typically poorly constrained due to the presence of occlusions and homogeneous textures. This thesis develops methods that can be used to increase the robustness of dense SLAM by fusing additional sensing modalities into standard dense SLAM pipelines. In particular, this thesis will look at two sensing modalities: acceleration and rotation rate measurements from an inertial measurement unit (IMU) to address the tracking issue, and learned priors on dense reconstructions from deep neural networks (DNNs) to address the mapping issue.Open Acces