Volumetric memory network for interactive medical image segmentation

Despite recent progress of automatic medical image segmentation techniques, fully automatic results usually fail to meet clinically acceptable accuracy, thus typically require further refinement. To this end, we propose a novel Volumetric Memory Network, dubbed as VMN, to enable segmentation of 3D medical images in an interactive manner. Provided by user hints on an arbitrary slice, a 2D interaction network is firstly employed to produce an initial 2D segmentation for the chosen slice. Then, the VMN propagates the initial segmentation mask bidirectionally to all slices of the entire volume. Subsequent refinement based on additional user guidance on other slices can be incorporated in the same manner. To facilitate smooth human-in-the-loop segmentation, a quality assessment module is introduced to suggest the next slice for interaction based on the segmentation quality of each slice produced in the previous round. Our VMN demonstrates two distinctive features: First, the memory-augmented network design offers our model the ability to quickly encode past segmentation information, which will be retrieved later for the segmentation of other slices; Second, the quality assessment module enables the model to directly estimate the quality of each segmentation prediction, which allows for an active learning paradigm where users preferentially label the lowest-quality slice for multi-round refinement. The proposed network leads to a robust interactive segmentation engine, which can generalize well to various types of user annotations (e.g., scribble, bounding box, extreme clicking). Extensive experiments have been conducted on three public medical image segmentation datasets (i.e., MSD, KiTS19, CVC-ClinicDB), and the results clearly confirm the superiority of our approach in comparison with state-of-the-art segmentation models. The code is made publicly available at https://github.com/0liliulei/Mem3D

Level set modeling and segmentation of diffusion tensor magnetic resonance imaging brain data

Segmentation of anatomical regions of the brain is one of the fundamental problems in medical image analysis. It is traditionally solved by iso-surfacing or through the use of active contours/deformable models on a gray-scale magnetic resonance imaging (MRI) data. We develop a technique that uses anisotropic diffusion properties of brain tissue available from diffusion tensor (DT)-MRI to segment brain structures. We develop a computational pipeline starting from raw diffusion tensor data through computation of invariant anisotropy measures to construction of geometric models of the brain structures. This provides an environment for user-controlled 3-D segmentation of DT-MRI datasets. We use a level set approach to remove noise from the data and to produce smooth, geometric models. We apply our technique to DT-MRI data of a human subject and build models of the isotropic and strongly anisotropic regions of the brain. Once geometric models have been constructed they can be combined to study spatial relationships and quantitatively analyzed to produce the volume and surface area of the segmented regions

3-D image segmentation and rendering

Finding methods for detecting objects in computer tomography images has been an active area of research in the medical and industrial imaging communities. While the raw image can be readily displayed as 2-D slices, 3-D analysis and visualization require explicitly defined object boundaries when creating 3-D models. A basic task in 3-D image processing is the segmentation of an image that classifies voxels/pixels into objects or groups. It is very computation intensive for processing because of the huge volume of data. The objective of this research is to find an efficient way to identify, isolate and enumerate 3-D objects in a given data set consisting of tomographic cross-sections of a device under test. In this research, an approach to 3-D image segmentation and rendering of CT data has been developed. Objects are first segmented from the background and then segmented between each other before 3-D rendering. During the first step of segmentation, current techniques of thresholding and image morphology provide a fast way to accomplish the work. During the second step of segmentation, a new method based on the watershed transform has been developed to deal with objects with deep connections. The new method takes advantage of the similarity between consecutive cross section images. The projections of the objects in the first image are taken as catchment basins for the second image. Only the different pixels in the second image are processed during segmentation. This not only saves time to find catchment basins, but also splits objects with deep connections that cannot be simply implemented by the watershed transform. A unique label has been issued to each object after segmentation. Objects can be distinguished well from each 2-D slice by their labels. This is a good preparation for 3-D rendering and quantitative analysis of each object. In this thesis, a novel 3-D rendering has been developed by surface rendering approach. A new and easier rendering model has been invented under the assumptions that light comes from the same side as the viewer, both of which are situated at infinity. It works fast because only surface pixels are being processed and interior pixels are left unprocessed. The surface intensity of the objects is attenuated by coefficients according to their distance from the viewer. The objects finally are shown from top and side views. Volume rendering was accomplished by sample images as well. In this research, the new method works several times faster than previous methods. After successful segmentation and rendering, the volume of each object can be easily calculated and the objects are recognizable in 3-D visualization. Keywords: 3-D Image Segmentation, 3-D Image Rendering, Watershed Transform, Surface Rendering, Thresholding, Morphological Transform

Automatic segmentation of the left ventricle cavity and myocardium in MRI data

A novel approach for the automatic segmentation has been developed to extract the epi-cardium and endo-cardium boundaries of the left ventricle (lv) of the heart. The developed segmentation scheme takes multi-slice and multi-phase magnetic resonance (MR) images of the heart, transversing the short-axis length from the base to the apex. Each image is taken at one instance in the heart's phase. The images are segmented using a diffusion-based filter followed by an unsupervised clustering technique and the resulting labels are checked to locate the (lv) cavity. From cardiac anatomy, the closest pool of blood to the lv cavity is the right ventricle cavity. The wall between these two blood-pools (interventricular septum) is measured to give an approximate thickness for the myocardium. This value is used when a radial search is performed on a gradient image to find appropriate robust segments of the epi-cardium boundary. The robust edge segments are then joined using a normal spline curve. Experimental results are presented with very encouraging qualitative and quantitative results and a comparison is made against the state-of-the art level-sets method

Image and Volume Segmentation by Water Flow

A general framework for image segmentation is presented in this paper, based on the paradigm of water flow. The major water flow attributes like water pressure, surface tension and capillary force are defined in the context of force field generation and make the model adaptable to topological and geometrical changes. A flow-stopping image functional combining edge- and region-based forces is introduced to produce capability for both range and accuracy. The method is assessed qualitatively and quantitatively on synthetic and natural images. It is shown that the new approach can segment objects with complex shapes or weak-contrasted boundaries, and has good immunity to noise. The operator is also extended to 3-D, and is successfully applied to medical volume segmentation

Segmentation of the left ventricle of the heart in 3-D+t MRI data using an optimized nonrigid temporal model

Modern medical imaging modalities provide large amounts of information in both the spatial and temporal domains and the incorporation of this information in a coherent algorithmic framework is a significant challenge. In this paper, we present a novel and intuitive approach to combine 3-D spatial and temporal (3-D + time) magnetic resonance imaging (MRI) data in an integrated segmentation algorithm to extract the myocardium of the left ventricle. A novel level-set segmentation process is developed that simultaneously delineates and tracks the boundaries of the left ventricle muscle. By encoding prior knowledge about cardiac temporal evolution in a parametric framework, an expectation-maximization algorithm optimally tracks the myocardial deformation over the cardiac cycle. The expectation step deforms the level-set function while the maximization step updates the prior temporal model parameters to perform the segmentation in a nonrigid sense