On the Effect of Inter-observer Variability for a Reliable Estimation of Uncertainty of Medical Image Segmentation

Uncertainty estimation methods are expected to improve the understanding and quality of computer-assisted methods used in medical applications (e.g., neurosurgical interventions, radiotherapy planning), where automated medical image segmentation is crucial. In supervised machine learning, a common practice to generate ground truth label data is to merge observer annotations. However, as many medical image tasks show a high inter-observer variability resulting from factors such as image quality, different levels of user expertise and domain knowledge, little is known as to how inter-observer variability and commonly used fusion methods affect the estimation of uncertainty of automated image segmentation. In this paper we analyze the effect of common image label fusion techniques on uncertainty estimation, and propose to learn the uncertainty among observers. The results highlight the negative effect of fusion methods applied in deep learning, to obtain reliable estimates of segmentation uncertainty. Additionally, we show that the learned observers' uncertainty can be combined with current standard Monte Carlo dropout Bayesian neural networks to characterize uncertainty of model's parameters.Comment: Appears in Medical Image Computing and Computer Assisted Interventions (MICCAI), 201

Cortical enhanced tissue segmentation of neonatal brain MR images acquired by a dedicated phased array coil

pre-printThe acquisition of high quality MR images of neonatal brains is largely hampered by their characteristically small head size and low tissue contrast. As a result, subsequent image processing and analysis, especially for brain tissue segmentation, are often hindered. To overcome this problem, a dedicated phased array neonatal head coil is utilized to improve MR image quality by effectively combing images obtained from 8 coil elements without lengthening data acquisition time. In addition, a subject-specific atlas based tissue segmentation algorithm is specifically developed for the delineation of fine structures in the acquired neonatal brain MR images. The proposed tissue segmentation method first enhances the sheet-like cortical gray matter (GM) structures in neonatal images with a Hessian filter for generation of cortical GM prior. Then, the prior is combined with our neonatal population atlas to form a cortical enhanced hybrid atlas, which we refer to as the subject-specific atlas. Various experiments are conducted to compare the proposed method with manual segmentation results, as well as with additional two population atlas based segmentation methods. Results show that the proposed method is capable of segmenting the neonatal brain with the highest accuracy, compared to other two methods

Case study: an evaluation of user-assisted hierarchical watershed segmentation

technical reportWhile level sets have demonstrated a great potential for 3D medical image segmentation, their usefulness has been limited by two problems. First, 3D level sets are relatively slow to compute. Second, their formulation usually entails several free parameters which can be very difficult to correctly tune for specific applications. The second problem is compounded by the first. This paper describes a new tool for 3D segmentation that addresses these problems by computing level-set surface models at interactive rates. This tool employs two important, novel technologies. First is the mapping of a 3D level-set solver onto a commodity graphics card (GPU). This mapping relies on a novel mechanism for GPU memory management. The interactive rates level-set PDE solver give the user immediate feedback on the parameter settings, and thus users can tune free parameters and control the shape of the model in real time. The second technology is the use of region-based speed functions, which allow a user to quickly and intuitively specify the behavior of the deformable model. We have found that the combination of these interactive tools enables users to produce good, reliable segmentations. To support this observation, this paper presents qualitative results from several different datasets as well as a quantitative evaluation from a study of brain tumor segmentations

SoftSeg: Advantages of soft versus binary training for image segmentation

Most image segmentation algorithms are trained on binary masks formulated as a classification task per pixel. However, in applications such as medical imaging, this "black-and-white" approach is too constraining because the contrast between two tissues is often ill-defined, i.e., the voxels located on objects' edges contain a mixture of tissues. Consequently, assigning a single "hard" label can result in a detrimental approximation. Instead, a soft prediction containing non-binary values would overcome that limitation. We introduce SoftSeg, a deep learning training approach that takes advantage of soft ground truth labels, and is not bound to binary predictions. SoftSeg aims at solving a regression instead of a classification problem. This is achieved by using (i) no binarization after preprocessing and data augmentation, (ii) a normalized ReLU final activation layer (instead of sigmoid), and (iii) a regression loss function (instead of the traditional Dice loss). We assess the impact of these three features on three open-source MRI segmentation datasets from the spinal cord gray matter, the multiple sclerosis brain lesion, and the multimodal brain tumor segmentation challenges. Across multiple cross-validation iterations, SoftSeg outperformed the conventional approach, leading to an increase in Dice score of 2.0% on the gray matter dataset (p=0.001), 3.3% for the MS lesions, and 6.5% for the brain tumors. SoftSeg produces consistent soft predictions at tissues' interfaces and shows an increased sensitivity for small objects. The richness of soft labels could represent the inter-expert variability, the partial volume effect, and complement the model uncertainty estimation. The developed training pipeline can easily be incorporated into most of the existing deep learning architectures. It is already implemented in the freely-available deep learning toolbox ivadomed (https://ivadomed.org)

GIST: an interactive, GPU-based level set segmentation tool for 3D medical images

technical reportWhile level sets have demonstrated a great potential for 3D medical image segmentation, their usefulness has been limited by two problems. First, 3D level sets are relatively slow to compute. Second, their formulation usually entails several free parameters which can be very difficult to correctly tune for specific applications. The second problem is compounded by the first. This paper describes a new tool for 3D segmentation that addresses these problems by computing level-set surface models at interactive rates. This tool employs two important, novel technologies. First is the mapping of a 3D level-set solver onto a commodity graphics card (GPU). This mapping relies on a novel mechanism for GPU memory management. The interactive rates level-set PDE solver give the user immediate feedback on the parameter settings, and thus users can tune free parameters and control the shape of the model in real time. The second technology is the use of region-based speed functions, which allow a user to quickly and intuitively specify the behavior of the deformable model. We have found that the combination of these interactive tools enables users to produce good, reliable segmentations. To support this observation, this paper presents qualitative results from several different datasets as well as a quantitative evaluation from a study of brain tumor segmentations

Computer Aided Tools for the Design and Planning of Personalized Shoulder Arthroplasty

La artroplastia de hombro es el tercer procedimiento de reemplazo articular más común, después de la artroplastia de rodilla y cadera, y actualmentees el de más rápido crecimiento en el campo ortopédico. Las principales opciones quirúrgicas incluyen la artroplastia total de hombro (TSA), en la quese restaura la anatomía articular normal, y, para pacientes con un manguito rotador completamente desgarrado, la artroplastia inversa de hombro (RSA), en la que la bola y la cavidad de la articulación glenohumeral se cambian. A pesar del progreso reciente y los avances en el diseño, las tasas de complicaciones reportadas para RSA son más altas que las de la artroplastia de hombro convencional. Un enfoque específico para el paciente, en el que los médicos adaptan el tratamiento quirúrgico a las características del mismo y al estado preoperatorio, por ejemplo mediante implantes personalizados y planificación previa, puede ayudar a reducir los problemas postoperatorios y mejorar el resultado funcional. El objetivo principal de esta tesis es desarrollar y evaluar métodos novedosos para RSA personalizado, utilizando tecnologías asistidas por ordenador de última generación para estandarizar y automatizar las fases de diseño y planificación.Los implantes personalizados son una solución adecuada para el tratamiento de pacientes con pérdida extensa de hueso glenoideo. Sin embargo, los ingenieros clínicos se enfrentan a muchas variables en el diseño de implantes (número y tipo de tornillos, superficie de contacto, etc.) y una gran variabilidad anatómica y patológica. Actualmente, no existen herramientas objetivas para guiarlos a la hora de elegir el diseño óptimo, es decir, con suficiente estabilidad inicial del implante, lo que hace que el proceso de diseño sea tedioso, lento y dependiente del usuario. En esta tesis, se desarrolló una simulación de Virtual Bench Test (VBT) utilizando un modelo de elementos finitos para evaluar automáticamente la estabilidad inicial de los implantes de hombro personalizados. A través de un experimento de validación, se demostró que los ingenieros clínicos pueden utilizar el resultado de Virtual Bench Test como referencia para respaldar sus decisiones y adaptaciones durante el proceso de diseño del implante.Al diseñar implantes de hombro, el conocimiento de la morfología y la calidad ósea de la escápula en toda la población es fundamental. En particular, se tienen en cuenta las regiones con la mejor reserva ósea (hueso cortical) para definir la posición y orientación de los orificios de los tornillos, mientras se busca una fijación óptima. Como alternativa a las mediciones manuales, cuya generalización está limitada por el análisis de pequeños subconjuntos de pacientes potenciales, Statistical Shape Models (SSMs) se han utilizado comúnmente para describir la variabilidad de la forma dentro de una población. Sin embargo, estos SSMs normalmente no contienen información sobre el grosor cortical.Por lo tanto, se desarrolló una metodología para combinar la forma del hueso escapular y la morfología de la cortical en un SSM. Primero, se presentó y evaluó un método para estimar el espesor cortical, a partir de un análisis de perfil de Hounsfield Unit (HU). Luego, utilizando 32 escápulas sanas segmentadas manualmente, se creó y evaluó un modelo de forma estadística que incluía información de la cortical. La herramienta desarrollada se puede utilizar para implantar virtualmente un nuevo diseño y probar su congruencia dentro de una población virtual generada, reduciendo así el número de iteraciones de diseño y experimentos con cadáveres.Las mediciones del alargamiento de los músculos deltoides y del manguito rotador durante la planificación quirúrgica pueden ayudar a los médicos aseleccionar un diseño y una posición de implante adecuados. Sin embargo, tal evaluación requiere la indicación de puntos anatómicos como referencia para los puntos de unión de los músculos, un proceso que requiere mucho tiempo y depende del usuario, ya que a menudo se realiza manualmente. Además, las imágenes médicas, que se utilizan normalmente para la artroplastia de hombro,contienen en su mayoría solo el húmero proximal, lo que hace imposible indicarlos puntos de unión de los músculos que se encuentran fuera del campo de visión de la exploración. Por lo tanto, se desarrolló y evaluó un método totalmente automatizado, basado en SSM, para medir la elongación del deltoides y del manguito rotador. Su aplicabilidad clínica se demostró mediante la evaluación del rendimiento de la estimación automatizada de la elongación muscular para un conjunto de articulaciones artríticas del hombro utilizadas para la planificación preoperatoria de RSA, lo que confirma que es una herramienta adecuada para los cirujanos a la hora de evaluar y refinar las decisiones clínicas.En esta investigación, se dio un paso importante en la dirección de un enfoque más personalizado de la artroplastia inversa de hombro, en el que el manejo quirúrgico, es decir, el diseño y la posición del implante, se adapta a las características específicas del paciente y al estado preoperatorio. Al aplicar tecnologías asistidas por computadora en la práctica clínica, el proceso de diseño y planificación se puede automatizar y estandarizar, reduciendo así los costos y los plazos de entrega. Además, gracias a los métodos novedosos presentados en esta tesis, esperamos en el futuro una adopción más amplia del enfoque personalizado, con importantes beneficios tanto para los cirujanos como para los pacientes.Shoulder arthroplasty is the third most common joint replacement procedure, after knee and hip arthroplasty, and currently the most rapidly growing one in the orthopaedic field. The main surgical options include total shoulder arthroplasty (TSA), in which the normal joint anatomy is restored, and, for patients with a completely torn rotator cuff, reverse shoulder arthroplasty (RSA), in which the ball and the socket of the glenohumeral joint are switched. Despite the recent progress and advancement in design, the reported rates of complication for RSA are higher than those of conventional shoulder arthroplasty. A patient-specific approach, in which clinicians adapt the surgical management to patient characteristics and preoperative condition, e.g. through custom implants and pre-planning, can help to reduce postoperative problems and improve the functional outcome. The main goal of this thesis is to develop and evaluate novel methods for personalized RSA, using state-of-the-art computer aided technologies to standardize and automate the design and planning phases. Custom implants are a suitable solution when treating patients with extensive glenoid bone loss. However, clinical engineers are confronted with an enormous implant design space (number and type of screws, contact surface, etc.) and large anatomical and pathological variability. Currently, no objective tools exist to guide them when choosing the optimal design, i.e. with sufficient initial implant stability, thus making the design process tedious, time-consuming, and user-dependent. In this thesis, a Virtual Bench Test (VBT) simulation was developed using a finite element model to automatically evaluate the initial stability of custom shoulder implants. Through a validation experiment, it was shown that the virtual test bench output can be used by clinical engineers as a reference to support their decisions and adaptations during the implant design process. When designing shoulder implants, knowledge about bone morphology and bone quality of the scapula throughout a certain population is fundamental. In particular, regions with the best bone stock (cortical bone) are taken into account to define the position and orientation of the screw holes, while aiming for an optimal fixation. As an alternative to manual measurements, whose generalization is limited by the analysis of small sub-sets of the potential patients, Statistical Shape Models (SSMs) have been commonly used to describe shape variability within a population. However, these SSMs typically do not contain information about cortical thickness. Therefore, a methodology to combine scapular bone shape and cortex morphology in an SSM was developed. First, a method to estimate cortical thickness, starting from a profile analysis of Hounsfield Unit (HU), was presented and evaluated. Then, using 32 manually segmented healthy scapulae, a statistical shape model including cortical information was created and assessed. The developed tool can be used to virtually implant a new design and test its congruency inside a generated virtual population, thus reducing the number of design iterations and cadaver labs. Measurements of deltoid and rotator cuff muscle elongation during surgical planning can help clinicians to select a suitable implant design and position. However, such an assessment requires the indication of anatomical landmarks as a reference for the muscle attachment points, a process that is time-consuming and user-dependent, since often performed manually. Additionally, the medical images, which are normally used for shoulder arthroplasty, mostly contain only the proximal humerus, making it impossible to indicate those muscle attachment points which lie outside of the field of view of the scan. Therefore, a fully-automated method, based on SSM, for measuring deltoid and rotator cuff elongation was developed and evaluated. Its clinical applicability was demonstrated by assessing the performance of the automated muscle elongation estimation for a set of arthritic shoulder joints used for preoperative planning of RSA, thus confirming it a suitable tool for surgeons when evaluating and refining clinical decisions. In this research, a major step was taken into the direction of a more personalized approach to Reverse Shoulder Arthroplasty, in which the surgical management, i.e. implant design and position, is adapted to the patient-specific characteristics and preoperative condition. By applying computer aided technologies in the clinical practice, design and planning process can be automated and standardized, thus reducing costs and lead times. Additionally, thanks to the novel methods presented in this thesis, we expect in the future a wider adoption of the personalized approach, with important benefits both for surgeons and patients.<br /

Validation of Image Segmentation and Expert Quality with an Expectation-Maximization Algorithm

Characterizing the performance of image segmentation approaches has been a persistent challenge. Performance analysis is important since segmentation algorithms often have limited accuracy and precision. Interactive drawing of the desired segmentation by domain experts has often been the only acceptable approach, and yet suffers from intra-expert and inter-expert variability. Automated algorithms have been sought in order to remove the variability introduced by experts, but no single methodology for the assessment and validation of such algorithms has yet been widely adopted. The accuracy of segmentations of medical images has been difficult to quantify in the absence of a &quot;ground truth&quot; segmentation for clinical data. Although physical or digital phantoms can help, they have so far been unable to reproduce the full range of imaging and anatomical characteristics observed in clinical data. An attractive alternative is comparison to a collection of segmentations by experts, but the most appropriate way to compare segmentations has been unclear

Prior information for brain parcellation

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2005.Includes bibliographical references (p. 171-184).To better understand brain disease, many neuroscientists study anatomical differences between normal and diseased subjects. Frequently, they analyze medical images to locate brain structures influenced by disease. Many of these structures have weakly visible boundaries so that standard image analysis algorithms perform poorly. Instead, neuroscientists rely on manual procedures, which are time consuming and increase risks related to inter- and intra-observer reliability [53]. In order to automate this task, we develop an algorithm that robustly segments brain structures. We model the segmentation problem in a Bayesian framework, which is applicable to a variety of problems. This framework employs anatomical prior information in order to simplify the detection process. In this thesis, we experiment with different types of prior information such as spatial priors, shape models, and trees describing hierarchical anatomical relationships. We pose a maximum a posteriori probability estimation problem to find the optimal solution within our framework. From the estimation problem we derive an instance of the Expectation Maximization algorithm, which uses an initial imperfect estimate to converge to a good approximation.(cont.) The resulting implementation is tested on a variety of studies, ranging from the segmentation of the brain into the three major brain tissue classes, to the parcellation of anatomical structures with weakly visible boundaries such as the thalamus or superior temporal gyrus. In general, our new method performs significantly better than other :standard automatic segmentation techniques. The improvement is due primarily to the seamless integration of medical image artifact correction, alignment of the prior information to the subject, detection of the shape of anatomical structures, and representation of the anatomical relationships in a hierarchical tree.by Kilian Maria Pohl.Ph.D

Minimally Interactive Segmentation with Application to Human Placenta in Fetal MR Images

Placenta segmentation from fetal Magnetic Resonance (MR) images is important for fetal surgical planning. However, accurate segmentation results are difficult to achieve for automatic methods, due to sparse acquisition, inter-slice motion, and the widely varying position and shape of the placenta among pregnant women. Interactive methods have been widely used to get more accurate and robust results. A good interactive segmentation method should achieve high accuracy, minimize user interactions with low variability among users, and be computationally fast. Exploiting recent advances in machine learning, I explore a family of new interactive methods for placenta segmentation from fetal MR images. I investigate the combination of user interactions with learning from a single image or a large set of images. For learning from a single image, I propose novel Online Random Forests to efficiently leverage user interactions for the segmentation of 2D and 3D fetal MR images. I also investigate co-segmentation of multiple volumes of the same patient with 4D Graph Cuts. For learning from a large set of images, I first propose a deep learning-based framework that combines user interactions with Convolutional Neural Networks (CNN) based on geodesic distance transforms to achieve accurate segmentation and good interactivity. I then propose image-specific fine-tuning to make CNNs adaptive to different individual images and able to segment previously unseen objects. Experimental results show that the proposed algorithms outperform traditional interactive segmentation methods in terms of accuracy and interactivity. Therefore, they might be suitable for segmentation of the placenta in planning systems for fetal and maternal surgery, and for rapid characterization of the placenta by MR images. I also demonstrate that they can be applied to the segmentation of other organs from 2D and 3D images