Silver Standard Masks for Data Augmentation Applied to Deep-Learning-Based Skull-Stripping

The bottleneck of convolutional neural networks (CNN) for medical imaging is the number of annotated data required for training. Manual segmentation is considered to be the "gold-standard". However, medical imaging datasets with expert manual segmentation are scarce as this step is time-consuming and expensive. We propose in this work the use of what we refer to as silver standard masks for data augmentation in deep-learning-based skull-stripping also known as brain extraction. We generated the silver standard masks using the consensus algorithm Simultaneous Truth and Performance Level Estimation (STAPLE). We evaluated CNN models generated by the silver and gold standard masks. Then, we validated the silver standard masks for CNNs training in one dataset, and showed its generalization to two other datasets. Our results indicated that models generated with silver standard masks are comparable to models generated with gold standard masks and have better generalizability. Moreover, our results also indicate that silver standard masks could be used to augment the input dataset at training stage, reducing the need for manual segmentation at this step

Performance Evaluation of Vanilla, Residual, and Dense 2D U-Net Architectures for Skull Stripping of Augmented 3D T1-weighted MRI Head Scans

Skull Stripping is a requisite preliminary step in most diagnostic neuroimaging applications. Manual Skull Stripping methods define the gold standard for the domain but are time-consuming and challenging to integrate into processing pipelines with a high number of data samples. Automated methods are an active area of research for head MRI segmentation, especially deep learning methods such as U-Net architecture implementations. This study compares Vanilla, Residual, and Dense 2D U-Net architectures for Skull Stripping. The Dense 2D U-Net architecture outperforms the Vanilla and Residual counterparts by achieving an accuracy of 99.75% on a test dataset. It is observed that dense interconnections in a U-Net encourage feature reuse across layers of the architecture and allow for shallower models with the strengths of a deeper network.Comment: Research Article submitted to the 2nd International Conference on Biomedical Engineering Science and Technology: Roadway from Laboratory to Market, at the National Institute of Technology Raipur, Chhattisgarh, Indi

One-shot Joint Extraction, Registration and Segmentation of Neuroimaging Data

Brain extraction, registration and segmentation are indispensable preprocessing steps in neuroimaging studies. The aim is to extract the brain from raw imaging scans (i.e., extraction step), align it with a target brain image (i.e., registration step) and label the anatomical brain regions (i.e., segmentation step). Conventional studies typically focus on developing separate methods for the extraction, registration and segmentation tasks in a supervised setting. The performance of these methods is largely contingent on the quantity of training samples and the extent of visual inspections carried out by experts for error correction. Nevertheless, collecting voxel-level labels and performing manual quality control on high-dimensional neuroimages (e.g., 3D MRI) are expensive and time-consuming in many medical studies. In this paper, we study the problem of one-shot joint extraction, registration and segmentation in neuroimaging data, which exploits only one labeled template image (a.k.a. atlas) and a few unlabeled raw images for training. We propose a unified end-to-end framework, called JERS, to jointly optimize the extraction, registration and segmentation tasks, allowing feedback among them. Specifically, we use a group of extraction, registration and segmentation modules to learn the extraction mask, transformation and segmentation mask, where modules are interconnected and mutually reinforced by self-supervision. Empirical results on real-world datasets demonstrate that our proposed method performs exceptionally in the extraction, registration and segmentation tasks. Our code and data can be found at https://github.com/Anonymous4545/JERSComment: Published as a research track paper at KDD 2023. Code: https://github.com/Anonymous4545/JER

Aprendizado profundo para análise do cérebro em imagens de ressonância magnética

Orientadores: Roberto de Alencar Lotufo, Sebastien Ourselin e Leticia RittnerDissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Elétrica e de Computação e University College LondonResumo: Redes neurais convolucionais (CNNs-Convolutional neural networks) são uma vertente do apredizado profundo que obtiveram muito sucesso quando aplicadas em várias análises em imagens de ressonância magnética (MR-magnetic resonance) do cérebro. As CNNs são métodos de aprendizagem de representação com várias camadas empilhadas compostas por uma operação de convolução seguida de uma ativação não linear e de camadas de agru- pamento. Nessas redes, cada camada gera uma representação mais alta e mais abstrata de uma determinada entrada, na qual os pesos das camadas convolucionais são aprendidos por um problema de otimização. Neste trabalho, tratamos dois problemas usando aborda- gens baseadas em aprendizagem profunda: remoção da calota craniana (SS) e tractografia. Primeiramente, propusemos um SS completo baseado em CNN treinado com o que nos referimos como máscaras de padrão de prata. A segmentação de tecido cerebral a partir de tecido não cerebral é um processo conhecido como extração da calota craniana ou re- moção de crânios. As máscaras de padrão de prata são geradas pela formação do consenso a partir de um conjunto de oito métodos de SS públicos, não baseados em aprendizagem profunda, usando o algoritmo Verdade Simultânea e Estimativa do Nível de Desempenho (STAPLE-Simultaneous Truth and Performance Level Estimation). Nossa abordagem al- cançou o desempenho do estado da arte, generalizou de forma otimizada, diminuiu a variabilidade inter / intra-avaliador e evitou a super-especialização da segmentação da CNN em relação a uma anotação manual específica. Em segundo lugar, investigamos uma solução de tractografia baseada em CNN para cirurgia de epilepsia. O principal objetivo desta análise foi estruturar uma linha de base para uma regressão baseada em aprendiza- gem profunda para prever as orientações da fibra da matéria branca. Tractografia é uma visualização das fibras ou tratos da substância branca; seu objetivo no planejamento pré- operatório é simplesmente identificar a posição de caminhos eloqüentes, como os tratos motor, sensorial e de linguagem, para reduzir o risco de danificar essas estruturas críticas. Realizamos uma análise em um único paciente e também uma análise entre 10 pacientes em uma abordagem de validação cruzada. Nossos resultados não foram ótimos, entretanto, as fibras preditas pelo algoritmo tenderam a ter um comprimento similar e convergiram para os locais médios do trato das fibras. Além disso, até onde sabemos, nosso método é a primeira abordagem que investiga CNNs para tractografia, e assim, nosso trabalho é uma base para este tópicoAbstract: Convolutional neural networks (CNNs) are one branch of deep learning that have per- formed successfully in many brain magnetic resonance (MR) imaging analysis. CNNs are representation-learning methods with stacked layers comprised of a convolution op- eration followed by a non-linear activation and pooling layers. In these networks, each layer outputs a higher and more abstract representation from a given input, in which the weights of the convolutional layers are learned by an optimization problem. In this work, we tackled two problems using deep-learning-based approaches: skull-stripping (SS) and tractography. We firstly proposed a full CNN-based SS trained with what we refer to as silver standard masks. Segmenting brain tissue from non-brain tissue is a process known as brain extraction or skull-stripping. Silver standard masks are generated by forming the consensus from a set of eight, public, non-deep-learning-based SS methods using the algo- rithm Simultaneous Truth and Performance Level Estimation (STAPLE). Our approach reached state-of-the-art performance, generalized optimally, decreased inter-/intra-rater variability, and avoided CNN segmentation overfitting towards one specific manual anno- tation. Secondly, we investigated a CNN-based tractography solution for epilepsy surgery. The main goal of this analysis was to structure a baseline for a deep-learning-based- regression to predict white matter fiber orientations. Tractography is a visualization of the white matter fibers or tracts; its goal in presurgical planing is simply to identify the position of eloquent pathways, such as the motor, sensory, and language tracts to reduce the risk of damaging these critical structures. We performed analysis cross-validation us- ing only in a single patient per time, and also, training with data from 10 patients for training the CNN. Our results were not optimal, however, the tracts tended to be of a similar length and converged to the mean fiber tract locations. Additionally, to the best of our knowledge, our method is the first approach that investigates CNNs for tractography, and thus, our work is a baseline for this topicMestradoEngenharia de ComputaçãoMestre em Engenharia Elétrica2016/18332-8, 2017/23747-5FAPES

Convolutional neural networks for the segmentation of small rodent brain MRI

Image segmentation is a common step in the analysis of preclinical brain MRI, often performed manually. This is a time-consuming procedure subject to inter- and intra- rater variability. A possible alternative is the use of automated, registration-based segmentation, which suffers from a bias owed to the limited capacity of registration to adapt to pathological conditions such as Traumatic Brain Injury (TBI). In this work a novel method is developed for the segmentation of small rodent brain MRI based on Convolutional Neural Networks (CNNs). The experiments here presented show how CNNs provide a fast, robust and accurate alternative to both manual and registration-based methods. This is demonstrated by accurately segmenting three large datasets of MRI scans of healthy and Huntington disease model mice, as well as TBI rats. MU-Net and MU-Net-R, the CCNs here presented, achieve human-level accuracy while eliminating intra-rater variability, alleviating the biases of registration-based segmentation, and with an inference time of less than one second per scan. Using these segmentation masks I designed a geometric construction to extract 39 parameters describing the position and orientation of the hippocampus, and later used them to classify epileptic vs. non-epileptic rats with a balanced accuracy of 0.80, five months after TBI. This clinically transferable geometric approach detects subjects at high-risk of post-traumatic epilepsy, paving the way towards subject stratification for antiepileptogenesis studies

Medical Image Segmentation: Thresholding and Minimum Spanning Trees

I bildesegmentering deles et bilde i separate objekter eller regioner. Det er et essensielt skritt i bildebehandling for å definere interesseområder for videre behandling eller analyse. Oppdelingsprosessen reduserer kompleksiteten til et bilde for å forenkle analysen av attributtene oppnådd etter segmentering. Det forandrer representasjonen av informasjonen i det opprinnelige bildet og presenterer pikslene på en måte som er mer meningsfull og lettere å forstå. Bildesegmentering har forskjellige anvendelser. For medisinske bilder tar segmenteringsprosessen sikte på å trekke ut bildedatasettet for å identifisere områder av anatomien som er relevante for en bestemt studie eller diagnose av pasienten. For eksempel kan man lokalisere berørte eller anormale deler av kroppen. Segmentering av oppfølgingsdata og baseline lesjonssegmentering er også svært viktig for å vurdere behandlingsresponsen. Det er forskjellige metoder som blir brukt for bildesegmentering. De kan klassifiseres basert på hvordan de er formulert og hvordan segmenteringsprosessen utføres. Metodene inkluderer de som er baserte på terskelverdier, graf-baserte, kant-baserte, klynge-baserte, modell-baserte og hybride metoder, og metoder basert på maskinlæring og dyp læring. Andre metoder er baserte på å utvide, splitte og legge sammen regioner, å finne diskontinuiteter i randen, vannskille segmentering, aktive kontuter og graf-baserte metoder. I denne avhandlingen har vi utviklet metoder for å segmentere forskjellige typer medisinske bilder. Vi testet metodene på datasett for hvite blodceller (WBCs) og magnetiske resonansbilder (MRI). De utviklede metodene og analysen som er utført på bildedatasettet er presentert i tre artikler. I artikkel A (Paper A) foreslo vi en metode for segmentering av nukleuser og cytoplasma fra hvite blodceller. Metodene estimerer terskelen for segmentering av nukleuser automatisk basert på lokale minima. Metoden segmenterer WBC-ene før segmentering av cytoplasma avhengig av kompleksiteten til objektene i bildet. For bilder der WBC-ene er godt skilt fra røde blodlegemer (RBC), er WBC-ene segmentert ved å ta gjennomsnittet av $n$ bilder som allerede var filtrert med en terskelverdi. For bilder der RBC-er overlapper WBC-ene, er hele WBC-ene segmentert ved hjelp av enkle lineære iterative klynger (SLIC) og vannskillemetoder. Cytoplasmaet oppnås ved å trekke den segmenterte nukleusen fra den segmenterte WBC-en. Metoden testes på to forskjellige offentlig tilgjengelige datasett, og resultatene sammenlignes med toppmoderne metoder. I artikkel B (Paper B) foreslo vi en metode for segmentering av hjernesvulster basert på minste dekkende tre-konsepter (minimum spanning tree, MST). Metoden utfører interaktiv segmentering basert på MST. I denne artikkelen er bildet lastet inn i et interaktivt vindu for segmentering av svulsten. Fokusregion og bakgrunn skilles ved å klikke for å dele MST i to trær. Ett av disse trærne representerer fokusregionen og det andre representerer bakgrunnen. Den foreslåtte metoden ble testet ved å segmentere to forskjellige 2D-hjerne T1 vektede magnetisk resonans bildedatasett. Metoden er enkel å implementere og resultatene indikerer at den er nøyaktig og effektiv. I artikkel C (Paper C) foreslår vi en metode som behandler et 3D MRI-volum og deler det i hjernen, ikke-hjernevev og bakgrunnsegmenter. Det er en grafbasert metode som bruker MST til å skille 3D MRI inn i de tre regiontypene. Grafen lages av et forhåndsbehandlet 3D MRI-volum etterfulgt av konstrueringen av MST-en. Segmenteringsprosessen gir tre merkede, sammenkoblende komponenter som omformes tilbake til 3D MRI-form. Etikettene brukes til å segmentere hjernen, ikke-hjernevev og bakgrunn. Metoden ble testet på tre forskjellige offentlig tilgjengelige datasett og resultatene ble sammenlignet med ulike toppmoderne metoder.In image segmentation, an image is divided into separate objects or regions. It is an essential step in image processing to define areas of interest for further processing or analysis. The segmentation process reduces the complexity of an image to simplify the analysis of the attributes obtained after segmentation. It changes the representation of the information in the original image and presents the pixels in a way that is more meaningful and easier to understand. Image segmentation has various applications. For medical images, the segmentation process aims to extract the image data set to identify areas of the anatomy relevant to a particular study or diagnosis of the patient. For example, one can locate affected or abnormal parts of the body. Segmentation of follow-up data and baseline lesion segmentation is also very important to assess the treatment response. There are different methods used for image segmentation. They can be classified based on how they are formulated and how the segmentation process is performed. The methods include those based on threshold values, edge-based, cluster-based, model-based and hybrid methods, and methods based on machine learning and deep learning. Other methods are based on growing, splitting and merging regions, finding discontinuities in the edge, watershed segmentation, active contours and graph-based methods. In this thesis, we have developed methods for segmenting different types of medical images. We tested the methods on datasets for white blood cells (WBCs) and magnetic resonance images (MRI). The developed methods and the analysis performed on the image data set are presented in three articles. In Paper A we proposed a method for segmenting nuclei and cytoplasm from white blood cells. The method estimates the threshold for segmentation of nuclei automatically based on local minima. The method segments the WBCs before segmenting the cytoplasm depending on the complexity of the objects in the image. For images where the WBCs are well separated from red blood cells (RBCs), the WBCs are segmented by taking the average of $n$ images that were already filtered with a threshold value. For images where RBCs overlap the WBCs, the entire WBCs are segmented using simple linear iterative clustering (SLIC) and watershed methods. The cytoplasm is obtained by subtracting the segmented nucleus from the segmented WBC. The method is tested on two different publicly available datasets, and the results are compared with state of the art methods. In Paper B, we proposed a method for segmenting brain tumors based on minimum spanning tree (MST) concepts. The method performs interactive segmentation based on the MST. In this paper, the image is loaded in an interactive window for segmenting the tumor. The region of interest and the background are selected by clicking to split the MST into two trees. One of these trees represents the region of interest and the other represents the background. The proposed method was tested by segmenting two different 2D brain T1-weighted magnetic resonance image data sets. The method is simple to implement and the results indicate that it is accurate and efficient. In Paper C, we propose a method that processes a 3D MRI volume and partitions it into brain, non-brain tissues, and background segments. It is a graph-based method that uses MST to separate the 3D MRI into the brain, non-brain, and background regions. The graph is made from a preprocessed 3D MRI volume followed by constructing the MST. The segmentation process produces three labeled connected components which are reshaped back to the shape of the 3D MRI. The labels are used to segment the brain, non-brain tissues, and the background. The method was tested on three different publicly available data sets and the results were compared to different state of the art methods.Doktorgradsavhandlin

Fighting the scanner effect in brain MRI segmentation with a progressive level-of-detail network trained on multi-site data

Many clinical and research studies of the human brain require an accurate structural MRI segmentation. While traditional atlas-based methods can be applied to volumes from any acquisition site, recent deep learning algorithms ensure very high accuracy only when tested on data from the same sites exploited in training (i.e., internal data). The performance degradation experienced on external data (i.e., unseen volumes from unseen sites) is due to the inter-site variabilities in intensity distributions induced by different MR scanner models, acquisition parameters, and unique artefacts. To mitigate this site-dependency, often referred to as the scanner effect, we propose LOD-Brain, a 3D convolutional neural network with progressive levels-of-detail (LOD) able to segment brain data from any site. Coarser network levels are responsible to learn a robust anatomical prior useful for identifying brain structures and their locations, while finer levels refine the model to handle site-specific intensity distributions and anatomical variations. We ensure robustness across sites by training the model on an unprecedented rich dataset aggregating data from open repositories: almost 27,000 T1w volumes from around 160 acquisition sites, at 1.5 - 3T, from a population spanning from 8 to 90 years old. Extensive tests demonstrate that LOD-Brain produces state-of-the-art results, with no significant difference in performance between internal and external sites, and robust to challenging anatomical variations. Its portability opens the way for large scale application across different healthcare institutions, patient populations, and imaging technology manufacturers. Code, model, and demo are available at the project website