Multi-Atlas Segmentation using Partially Annotated Data: Methods and Annotation Strategies

Multi-atlas segmentation is a widely used tool in medical image analysis, providing robust and accurate results by learning from annotated atlas datasets. However, the availability of fully annotated atlas images for training is limited due to the time required for the labelling task. Segmentation methods requiring only a proportion of each atlas image to be labelled could therefore reduce the workload on expert raters tasked with annotating atlas images. To address this issue, we first re-examine the labelling problem common in many existing approaches and formulate its solution in terms of a Markov Random Field energy minimisation problem on a graph connecting atlases and the target image. This provides a unifying framework for multi-atlas segmentation. We then show how modifications in the graph configuration of the proposed framework enable the use of partially annotated atlas images and investigate different partial annotation strategies. The proposed method was evaluated on two Magnetic Resonance Imaging (MRI) datasets for hippocampal and cardiac segmentation. Experiments were performed aimed at (1) recreating existing segmentation techniques with the proposed framework and (2) demonstrating the potential of employing sparsely annotated atlas data for multi-atlas segmentation

Segmentation of pelvic structures from preoperative images for surgical planning and guidance

Prostate cancer is one of the most frequently diagnosed malignancies globally and the second leading cause of cancer-related mortality in males in the developed world. In recent decades, many techniques have been proposed for prostate cancer diagnosis and treatment. With the development of imaging technologies such as CT and MRI, image-guided procedures have become increasingly important as a means to improve clinical outcomes. Analysis of the preoperative images and construction of 3D models prior to treatment would help doctors to better localize and visualize the structures of interest, plan the procedure, diagnose disease and guide the surgery or therapy. This requires efficient and robust medical image analysis and segmentation technologies to be developed. The thesis mainly focuses on the development of segmentation techniques in pelvic MRI for image-guided robotic-assisted laparoscopic radical prostatectomy and external-beam radiation therapy. A fully automated multi-atlas framework is proposed for bony pelvis segmentation in MRI, using the guidance of MRI AE-SDM. With the guidance of the AE-SDM, a multi-atlas segmentation algorithm is used to delineate the bony pelvis in a new \ac{MRI} where there is no CT available. The proposed technique outperforms state-of-the-art algorithms for MRI bony pelvis segmentation. With the SDM of pelvis and its segmented surface, an accurate 3D pelvimetry system is designed and implemented to measure a comprehensive set of pelvic geometric parameters for the examination of the relationship between these parameters and the difficulty of robotic-assisted laparoscopic radical prostatectomy. This system can be used in both manual and automated manner with a user-friendly interface. A fully automated and robust multi-atlas based segmentation has also been developed to delineate the prostate in diagnostic MR scans, which have large variation in both intensity and shape of prostate. Two image analysis techniques are proposed, including patch-based label fusion with local appearance-specific atlases and multi-atlas propagation via a manifold graph on a database of both labeled and unlabeled images when limited labeled atlases are available. The proposed techniques can achieve more robust and accurate segmentation results than other multi-atlas based methods. The seminal vesicles are also an interesting structure for therapy planning, particularly for external-beam radiation therapy. As existing methods fail for the very onerous task of segmenting the seminal vesicles, a multi-atlas learning framework via random decision forests with graph cuts refinement has further been proposed to solve this difficult problem. Motivated by the performance of this technique, I further extend the multi-atlas learning to segment the prostate fully automatically using multispectral (T1 and T2-weighted) MR images via hybrid \ac{RF} classifiers and a multi-image graph cuts technique. The proposed method compares favorably to the previously proposed multi-atlas based prostate segmentation. The work in this thesis covers different techniques for pelvic image segmentation in MRI. These techniques have been continually developed and refined, and their application to different specific problems shows ever more promising results.Open Acces

Deformable MR Prostate Segmentation via Deep Feature Learning and Sparse Patch Matching

Automatic and reliable segmentation of the prostate is an important but difficult task for various clinical applications such as prostate cancer radiotherapy. The main challenges for accurate MR prostate localization lie in two aspects: (1) inhomogeneous and inconsistent appearance around prostate boundary, and (2) the large shape variation across different patients. To tackle these two problems, we propose a new deformable MR prostate segmentation method by unifying deep feature learning with the sparse patch matching. First, instead of directly using handcrafted features, we propose to learn the latent feature representation from prostate MR images by the stacked sparse auto-encoder (SSAE). Since the deep learning algorithm learns the feature hierarchy from the data, the learned features are often more concise and effective than the handcrafted features in describing the underlying data. To improve the discriminability of learned features, we further refine the feature representation in a supervised fashion. Second, based on the learned features, a sparse patch matching method is proposed to infer a prostate likelihood map by transferring the prostate labels from multiple atlases to the new prostate MR image. Finally, a deformable segmentation is used to integrate a sparse shape model with the prostate likelihood map for achieving the final segmentation. The proposed method has been extensively evaluated on the dataset that contains 66 T2-wighted prostate MR images. Experimental results show that the deep-learned features are more effective than the handcrafted features in guiding MR prostate segmentation. Moreover, our method shows superior performance than other state-of-the-art segmentation methods

Multiatlas-Based Segmentation Editing With Interaction-Guided Patch Selection and Label Fusion

We propose a novel multi-atlas based segmentation method to address the segmentation editing scenario, where an incomplete segmentation is given along with a set of existing reference label images (used as atlases). Unlike previous multi-atlas based methods, which depend solely on appearance features, we incorporate interaction-guided constraints to find appropriate atlas label patches in the reference label set and derive their weights for label fusion. Specifically, user interactions provided on the erroneous parts are first divided into multiple local combinations. For each combination, the atlas label patches well-matched with both interactions and the previous segmentation are identified. Then, the segmentation is updated through the voxel-wise label fusion of selected atlas label patches with their weights derived from the distances of each underlying voxel to the interactions. Since the atlas label patches well-matched with different local combinations are used in the fusion step, our method can consider various local shape variations during the segmentation update, even with only limited atlas label images and user interactions. Besides, since our method does not depend on either image appearance or sophisticated learning steps, it can be easily applied to general editing problems. To demonstrate the generality of our method, we apply it to editing segmentations of CT prostate, CT brainstem, and MR hippocampus, respectively. Experimental results show that our method outperforms existing editing methods in all three data sets

Multi-Atlas Segmentation of Biomedical Images: A Survey

Abstract Multi-atlas segmentation (MAS), first introduced and popularized by the pioneering work of Rohlfing

Automatic 3D segmentation of the prostate on magnetic resonance images for radiotherapy planning

Abstract. Accurate segmentation of the prostate, the seminal vesicles, the bladder and the rectum is a crucial step for planning radiotherapy (RT) procedures. Modern radiotherapy protocols have included the delineation of the pelvic organs in magnetic resonance images (MRI), as the guide to the therapeutic beam irradiation over the target organ. However, this task is highly inter and intra-expert variable and may take about 20 minutes per patient, even for trained experts, constituting an important burden in most radiological services. Automatic or semi-automatic segmentation strategies might then improve the efficiency by decreasing the measured times while conserving the required accuracy. This thesis presents a fully automatic prostate segmentation framework that selects the most similar prostates w.r.t. a test prostate image and combines them to estimate the segmentation for the test prostate. A robust multi-scale analysis establishes the set of most similar prostates from a database, independently of the acquisition protocol. Those prostates are then non-rigidly registered towards the test image and fusioned by a linear combination. The proposed approach was evaluated using a MRI public dataset of patients with benign hyperplasia or cancer, following different acquisition protocols, namely 26 endorectal and 24 external. Evaluating under a leave-one-out scheme, results show reliable segmentations, obtaining an average dice coefficient of 79%, when comparing with the expert manual segmentation.La delineación exacta de la próstata, las vesículas seminales, la vejiga y el recto es un paso fundamental para el planeamiento de procedimientos de radioterapia. Protocolos modernos han incluido la delineación de los órganos pélvicos en imágenes de resonancia magnética (IRM), como la guia para la irradiación del haz terapéutico sobre el órgano objetivo. Sin embargo, esta tarea es altamente variable intra e inter-experto y puede tomar al rededor de 20 minutos por paciente, incluso para expertos entrenados, convirtiéndose en una carga importante en la mayoría de los servicios de radiología. Métodos automáticos o semi-automáticos podrían mejorar la eficiencia disminuyendo los tiempos medidos mientras se conserva la precisión requerida. Este trabajo presenta una estrategia de segmentación de la próstata completamente automático que selecciona las prostatas más similares con respecto a una imagen de resonancia magnética de prueba y combina las delineaciones asociadas a dichas imágenes para estimar la segmentación de la imagen de prueba. Un análisis multiescala robusto permite establecer el conjunto de las próstatas más parecidas de una base de datos, independiente del protocolo de adquisición. Las imágenes seleccionadas son registradas de forma no rigida con respecto a la imagen de prueba y luego son fusionadas mediante una combinación lineal. El enfoque propuesto fue evaluado utilizando un conjunto público de imágenes de resonancia magnética de pacientes con hiperplasia benigna o con cancer, con diferentes protocolos de adquisición, esto es 26 externas y 24 endorectales. Este trabajo fue evaluado bajo un esquema leave-one-out, cuyos resultados mostraron segmentaciones confiables, obteniendo un DSC promedio de 79%, cuando se compararon los resultados obtenidos con las segmentaciones manuales de expertos.Maestrí

Atlas-Based Methods in Radiotherapy Treatment of Head and Neck Cancer

Radiotherapy is one of the principal methods for treating head and neck cancer (HNC). It plays an important role in the curative and palliative treatment of HNC. It uses high-energy radiation beams to kill cancer cells by damaging their DNA. Radiotherapy planning depends upon complex algorithms to determine the best trajectories and intensities of those beams by simulating their effects passing through designated areas. This requires accurate segmentation of anatomical structures and knowledge of the relative electron density within a patient body. Computed tomography (CT) has been the modality of choice in radiotherapy planning. It offers a wealth of anatomical information and is critical in providing information about the relative electron density of tissues required to calculate radiation deposited at any one site. Manual segmentation is time-consuming and is becoming impractical with the increasing demand in image acquisition for planning. Recently, planning solely based on magnetic resonance (MR) imaging has gained popularity as it provides superior soft tissue contrast compared to CT imaging and can better facilitate the process of segmentation. However, MR imaging does not provide electron density information for dose calculation. With the growing volumes of data and data repositories, algorithms based on atlases have gained popularity as they provide prior information for structure segmentation and tissue classification. In this PhD thesis, I demonstrate that atlas-based methods can be used for segmenting head and neck structures giving results as comparable as manual segmentation. In addition, I demonstrate that those methods can be used to support radiotherapy treatment solely based on MR imaging by generating synthetic CT images. The radiation doses calculated from a synthetic and real CT image agreed well, showing the clinical feasibility of methods based on atlases. In conclusion, I show that atlas-based methods are clinically relevant in radiotherapy treatment