Regmentation: A New View of Image Segmentation and Registration

Image segmentation and registration have been the two major areas of research in the medical imaging community for decades and still are. In the context of radiation oncology, segmentation and registration methods are widely used for target structure definition such as prostate or head and neck lymph node areas. In the past two years, 45% of all articles published in the most important medical imaging journals and conferences have presented either segmentation or registration methods. In the literature, both categories are treated rather separately even though they have much in common. Registration techniques are used to solve segmentation tasks (e.g. atlas based methods) and vice versa (e.g. segmentation of structures used in a landmark based registration). This article reviews the literature on image segmentation methods by introducing a novel taxonomy based on the amount of shape knowledge being incorporated in the segmentation process. Based on that, we argue that all global shape prior segmentation methods are identical to image registration methods and that such methods thus cannot be characterized as either image segmentation or registration methods. Therefore we propose a new class of methods that are able solve both segmentation and registration tasks. We call it regmentation. Quantified on a survey of the current state of the art medical imaging literature, it turns out that 25% of the methods are pure registration methods, 46% are pure segmentation methods and 29% are regmentation methods. The new view on image segmentation and registration provides a consistent taxonomy in this context and emphasizes the importance of regmentation in current medical image processing research and radiation oncology image-guided applications

Segmentation, registration, and fusion of medical images

6.1 Segmentation 252 6.1.1 Intensity-based Segmentation 252 6.1.1.1 Point-based Segmentation 252 6.1.1.2 Edge-based Segmentation 254 6.1.1.3 Region-based Segmentation 256 6.1.2 Model-based Segmentation 258 6.1.3 Atlas-based Segmentation 258 6.2 Registration 259 6.2.1 Choice of Transformation 260 6.2.1.1 Rigid 261 6.2.1.2 Affine 261 6.2.1.3 Projective 262 6.2.1.4 Elastic 263 6.2.2 Finding Correspondences 263 6.2.2.1 Extrinsic Landmarks 264 6.2.2.2 Intrinsic Landmarks 264 6.2.2.3 Segmentations 264 6.2.2.4 Grids and Voxel Properties 265 6.2.3 Computing Transformations 266 6.2.3.1 Known Point to Point Correspondences 266 6.2.3.2 Point Clouds 266 6.2.3.3 Voxel Based Registration 268 6.2.4 Integration 270 6.2.4.1 Rigid Registration 270 6.2.4.2 Elastic Registration 271 6.3 Fusion 271 Acknowledgments 273 References 27

Non-uniform deformable volumetric objects for medical organ segmentation and registration

In medical imaging, large amounts of data are created during each patient examination, especially using 3-dimensional image acquisition techniques such as Computed Tomography. This data becomes more and more difficult to handle by humans without the aid of automated or semi-automated image processing means and analysis. Particularly, the manual segmentation of target structures in 3D image data is one of the most time consuming tasks for the physician in the context of using computerized medical applications. In addition, 3D image data increases the difficulty of mentally comparing two different images of the same structure. Robust automated organ segmentation and registration methods are therefore needed in order to fully utilize the potentials of modern medical imaging. This thesis addresses the described issues by introducing a new model based method for automated segmentation and registration of organs in 3D Computed Tomography images. In order to be able to robustly segment organs in low contrast images, a volumetric model based approach is proposed that incorporates texture information from the model’s interior during adaptation. It is generalizable and extendable such that it can be combined with statistical shape modeling methods and standard boundary detection approaches. In order to increase the robustness of the segmentation in cases where the shape of the target organ significantly deviates from the model, local elasticity constraints are proposed. They limit the flexibility of the model in areas where shape deviation is unlikely. This allows for a better segmentation of untrained shapes and improves the segmentation of organs with complex shape variation like the liver. The model based methods are evaluated on the liver in the portal venous and arterial contrast phase, the bladder, the pancreas, and the kidneys. An average surface distance error between 0.5 mm and 2.0 mm is obtained for the tested structures which is in most cases close to the interobserver variability between different humans segmenting the same structure. In the case of the pancreas, for the first time, an automatic segmentation from single phase contrast enhanced CT becomes feasible. In the context of organ registration, the developed methods are applied to deformable registration of multi-phase contrast enhanced liver CT data. The method is integrated into a clinical demonstrator and is currently in use for testing in two clinics. The presented method for automatic deformable multi-phase registration has been quantitatively and qualitatively evaluated in the clinic. In nearly all tested cases, the registration quality is sufficient for clinical needs. The result of this thesis is a new approach for automatic organ segmentation and registration that can be applied to various clinical problems. In many cases, it can be used to significantly reduce or even remove the amount of manual contour drawing. In the context of registration, the approach can be used to improve clinical diagnosis by overlaying different images of the same anatomical structure with higher quality than existing methods

Computer Aided Segmentation of Kidneys Using Locally Shape Constrained Deformable Models on CT Images

This work presents a novel approach for model based segmentation of the kidney in images acquired by Computed Tomography (CT). The developed computer aided segmentation system is expected to support computer aided diagnosis and operation planning. We have developed a deformable model based approach based on local shape constraints that prevents the model from deforming into neighboring structures while allowing the global shape to adapt freely to the data. Those local constraints are derived from the anatomical structure of the kidney and the presence and appearance of neighboring organs. The adaptation process is guided by a rule-based deformation logic in order to improve the robustness of the segmentation in areas of diffuse organ boundaries. Our work flow consists of two steps: 1.) a user guided positioning and 2.) an automatic model adaptation using affine and free form deformation in order to robustly extract the kidney. In cases which show pronounced pathologies, the system also offers real time mesh editing tools for a quick refinement of the segmentation result. Evaluation results based on 30 clinical cases using CT data sets show an average dice correlation coefficient of 93 compared to the ground truth. The results are therefore in most cases comparable to manual delineation. Computation times of the automatic adaptation step are lower than 6 seconds which makes the proposed system suitable for an application in clinical practice

An appearance-driven method for converting polygon soup building models for 3D geospatial applications

Polygon soup building models are fine for visualization purposes such as in games and movies. They, however, are not suitable for 3D geospatial applications which require geometrical analysis, since they lack connectivity information and may contain intersections internally between their parts. In this paper, we propose an appearance-driven method to interactively convert an input polygon soup building model to a two-manifold mesh, which is more suitable for 3D geospatial applications. Since a polygon soup model is not suitable for geometrical analysis, our key idea is to extract and utilize the visual appearance of the input building model for the conversion. We extract the silhouettes and use them to identify the features of the building. We then generate horizontal cross sections based on the locations of the features and then reconstruct the building by connecting two neighbouring cross sections. We propose to integrate various rasterization techniques to facilitate the conversion. Experimental results show the effectiveness of the proposed method.NRF (Natl Research Foundation, S’pore)Accepted versio

Simultaneous Segmentation and Correspondence Establishment for Statistical Shape Models

Statistical Shape Models have been proven to be valuable tools for segmenting anatomical structures of arbitrary topology. Being based on the statistical description of representative shapes, an initial segmentation is required - preferably done by an expert. For this purpose, mostly manual segmentation methods followed by a mesh generation step are employed. A prerequisite for generating the training data based on these segmentations is the establishment of correspondences between all training meshes. While existing approaches decouple the expert segmentation from the correspondence establishment step, we propose in this work a segmentation approach that simultaneously establishes the landmark correspondences needed for the subsequent generation of shape models. Our approach uses a reference segmentation given as a regular mesh. After an initial placement of this reference mesh, it is manually deformed in order to best match the boundaries of the considered anatomical structure. This deformation is coupled with a real time optimization that preserves point correspondences and thus ensures that a pair of landmark points in two different data sets represents the same anatomical feature. We applied our new method to different anatomical structures: vertebra of the spinal chord, kidney, and cardiac left ventricle. In order to perform a visual evaluation of the degree of correspondence between different data sets, we have developed well adapted visualization methods. From our tests we conclude that the expected correspondences are established during the manual mesh deformation. Furthermore, our approach considerably speeds up the shape model generation, since there is no need for an independent correspondence establishment step. Finally, it allows the creation of shape models of arbitrary topology and removes potential error sources of landmark and correspondence optimization algorithms needed so far

Erweiterung modellbasierter Segmentierung durch lokale Deformationskriterien

Modellbasierte Ansätze sind heutzutage Stand der Technik zur automatischen Organsegmentierung in medizinischen Bilddatensätzen. In dieser Arbeit wird ein Verfahren vorgestellt, welches die modellbasierte Segmentierung durch lokale Deformationskriterien erweitert, um eine bessere lokale Anpassung der Oberflächenmodelle an Bildstrukturen sowohl hoher als auch niedriger Frequenz zu erreichen. Die beschriebene Methode wird anhand von Computer-Tomographie Datensätzen der Niere beschrieben und evaluiert

Smart Manual Landmarking of Organs

Statistical shape models play a very important role in most modern medical segmentation frameworks. In this work we propose an extension to an existing approach for statistical shape model generation based on manual mesh deformation. Since the manual acquisition of ground truth segmentation data is a prerequisite for shape model creation, we developed a method that integrates a solution to the landmark correspondence problem in this particular step. This is done by coupling a user guided mesh adaptation for ground truth segmentation with a simultaneous real time optimization of the mesh in order to preserve point correspondences. First, a reference model with evenly distributed points is created that is taken as the basis of manual deformation. Afterwards the user adapts the model to the data set using a 3D Gaussian deformation of varying stiffness. The resulting meshes can be directly used for shape model construction. Furthermore, our approach allows the creation of shape models of arbitrary topology. We evaluate our method on CT data sets of the kidney and 4D MRI time series images of the cardiac left ventricle. A comparison with standard ICP-based and population-based optimization based correspondence algorithms showed better results both in terms of generalization capability and specificity for the model generated by our approach. The proposed method can therefore be used to considerably speed up and ease the process of shape model generation as well as remove potential error sources of landmark and correspondence optimization algorithms needed so far

Simultaneous Segmentation and Correspondence Establishment for Statistical Shape Models

Statistical Shape Models have been proven to be valuable tools for segmenting anatomical structures of arbitrary topology. Being based on the statistical description of representative shapes, an initial segmentation is required - preferably done by an expert. For this purpose, mostly manual segmentation methods followed by a mesh generation step are employed. A prerequisite for generating the training data based on these segmentations is the establishment of correspondences between all training meshes. While existing approaches decouple the expert segmentation from the correspondence establishment step, we propose in this work a segmentation approach that simultaneously establishes the landmark correspondences needed for the subsequent generation of shape models. Our approach uses a reference segmentation given as a regular mesh. After an initial placement of this reference mesh, it is manually deformed in order to best match the boundaries of the considered anatomical structure. This deformation is coupled with a real time optimization that preserves point correspondences and thus ensures that a pair of landmark points in two different data sets represents the same anatomical feature. We applied our new method to different anatomical structures: vertebra of the spinal chord, kidney, and cardiac left ventricle. In order to perform a visual evaluation of the degree of correspondence between different data sets, we have developed well adapted visualization methods. From our tests we conclude that the expected correspondences are established during the manual mesh deformation. Furthermore, our approach considerably speeds up the shape model generation, since there is no need for an independent correspondence establishment step. Finally, it allows the creation of shape models of arbitrary topology and removes potential error sources of landmark and correspondence optimization algorithms needed so far