Methods for three-dimensional Registration of Multimodal Abdominal Image Data

Multimodal image registration benefits the diagnosis, treatment planning and the performance of image-guided procedures in the liver, since it enables the fusion of complementary information provided by pre- and intrainterventional data about tumor localization and access. Although there exist various registration methods, approaches which are specifically optimized for the registration of multimodal abdominal scans are only scarcely available. The work presented in this thesis aims to tackle this problem by focusing on the development, optimization and evaluation of registration methods specifically for the registration of multimodal liver scans. The contributions to the research field of medical image registration include the development of a registration evaluation methodology that enables the comparison and optimization of linear and non-linear registration algorithms using a point-based accuracy measure. This methodology has been used to benchmark standard registration methods as well as novel approaches that were developed within the frame of this thesis. The results of the methodology showed that the employed similarity measure used during the registration has a major impact on the registration accuracy of the method. Due to this influence, two alternative similarity metrics bearing the potential to be used on multimodal image data are proposed and evaluated. The first metric relies on the use of gradient information in form of Histograms of Oriented Gradients (HOG) whereas the second metric employs a siamese neural network to learn a similarity measure directly on the image data. The evaluation showed, that both metrics could compete with state of the art similarity measures in terms of registration accuracy. The HOG-metric offers the advantage that it does not require ground truth data to learn a similarity estimation, but instead it is applicable to various data sets with the sole requirement of distinct gradients. However, the Siamese metric is characterized by a higher robustness for large rotations than the HOG-metric. To train such a network, registered ground truth data is required which may be critical for multimodal image data. Yet, the results show that it is possible to apply models trained on registered synthetic data on real patient data. The last part of this thesis focuses on methods to learn an entire registration process using neural networks, thereby offering the advantage to replace the traditional, time-consuming iterative registration procedure. Within the frame of this thesis, the so-called VoxelMorph network which was originally proposed for monomodal, non-linear registration learning is extended for affine and multimodal registration learning tasks. This extension includes the consideration of an image mask during metric evaluation as well as loss functions for multimodal data, such as the pretrained Siamese metric and a loss relying on the comparison of deformation fields. Based on the developed registration evaluation methodology, the performance of the original network as well as the extended variants are evaluated for monomodal and multimodal registration tasks using multiple data sets. With the extended network variants, it is possible to learn an entire multimodal registration process for the correction of large image displacements. As for the Siamese metric, the results imply a general transferability of models trained with synthetic data to registration tasks including real patient data. Due to the lack of multimodal ground truth data, this transfer represents an important step towards making Deep Learning based registration procedures clinically usable

Quantitative Analysis of Radiation-Associated Parenchymal Lung Change

Radiation-induced lung damage (RILD) is a common consequence of thoracic radiotherapy (RT). We present here a novel classification of the parenchymal features of RILD. We developed a deep learning algorithm (DLA) to automate the delineation of 5 classes of parenchymal texture of increasing density. 200 scans were used to train and validate the network and the remaining 30 scans were used as a hold-out test set. The DLA automatically labelled the data with Dice Scores of 0.98, 0.43, 0.26, 0.47 and 0.92 for the 5 respective classes. Qualitative evaluation showed that the automated labels were acceptable in over 80% of cases for all tissue classes, and achieved similar ratings to the manual labels. Lung registration was performed and the effect of radiation dose on each tissue class and correlation with respiratory outcomes was assessed. The change in volume of each tissue class over time generated by manual and automated segmentation was calculated. The 5 parenchymal classes showed distinct temporal patterns We quantified the volumetric change in textures after radiotherapy and correlate these with radiotherapy dose and respiratory outcomes. The effect of local dose on tissue class revealed a strong dose-dependent relationship We have developed a novel classification of parenchymal changes associated with RILD that show a convincing dose relationship. The tissue classes are related to both global and local dose metrics, and have a distinct evolution over time. Although less strong, there is a relationship between the radiological texture changes we can measure and respiratory outcomes, particularly the MRC score which directly represents a patient’s functional status. We have demonstrated the potential of using our approach to analyse and understand the morphological and functional evolution of RILD in greater detail than previously possible

Quantitative analysis with machine learning models for multi-parametric brain imaging data

Gliomas are considered to be the most common primary adult malignant brain tumor. With the dramatic increases in computational power and improvements in image analysis algorithms, computer-aided medical image analysis has been introduced into clinical applications. Precision tumor grading and genotyping play an indispensable role in clinical diagnosis, treatment and prognosis. Gliomas diagnostic procedures include histopathological imaging tests, molecular imaging scans and tumor grading. Pathologic review of tumor morphology in histologic sections is the traditional method for cancer classification and grading, yet human study has limitations that can result in low reproducibility and inter-observer agreement. Compared with histopathological images, Magnetic resonance (MR) imaging present the different structure and functional features, which might serve as noninvasive surrogates for tumor genotypes. Therefore, computer-aided image analysis has been adopted in clinical application, which might partially overcome these shortcomings due to its capacity to quantitatively and reproducibly measure multilevel features on multi-parametric medical information. Imaging features obtained from a single modal image do not fully represent the disease, so quantitative imaging features, including morphological, structural, cellular and molecular level features, derived from multi-modality medical images should be integrated into computer-aided medical image analysis. The image quality differentiation between multi-modality images is a challenge in the field of computer-aided medical image analysis. In this thesis, we aim to integrate the quantitative imaging data obtained from multiple modalities into mathematical models of tumor prediction response to achieve additional insights into practical predictive value. Our major contributions in this thesis are: 1. Firstly, to resolve the imaging quality difference and observer-dependent in histological image diagnosis, we proposed an automated machine-learning brain tumor-grading platform to investigate contributions of multi-parameters from multimodal data including imaging parameters or features from Whole Slide Images (WSI) and the proliferation marker KI-67. For each WSI, we extract both visual parameters such as morphology parameters and sub-visual parameters including first-order and second-order features. A quantitative interpretable machine learning approach (Local Interpretable Model-Agnostic Explanations) was followed to measure the contribution of features for single case. Most grading systems based on machine learning models are considered “black boxes,” whereas with this system the clinically trusted reasoning could be revealed. The quantitative analysis and explanation may assist clinicians to better understand the disease and accordingly to choose optimal treatments for improving clinical outcomes. 2. Based on the automated brain tumor-grading platform we propose, multimodal Magnetic Resonance Images (MRIs) have been introduced in our research. A new imaging–tissue correlation based approach called RA-PA-Thomics was proposed to predict the IDH genotype. Inspired by the concept of image fusion, we integrate multimodal MRIs and the scans of histopathological images for indirect, fast, and cost saving IDH genotyping. The proposed model has been verified by multiple evaluation criteria for the integrated data set and compared to the results in the prior art. The experimental data set includes public data sets and image information from two hospitals. Experimental results indicate that the model provided improves the accuracy of glioma grading and genotyping

A Survey on Artificial Intelligence Techniques for Biomedical Image Analysis in Skeleton-Based Forensic Human Identification

This paper represents the first survey on the application of AI techniques for the analysis of biomedical images with forensic human identification purposes. Human identification is of great relevance in today’s society and, in particular, in medico-legal contexts. As consequence, all technological advances that are introduced in this field can contribute to the increasing necessity for accurate and robust tools that allow for establishing and verifying human identity. We first describe the importance and applicability of forensic anthropology in many identification scenarios. Later, we present the main trends related to the application of computer vision, machine learning and soft computing techniques to the estimation of the biological profile, the identification through comparative radiography and craniofacial superimposition, traumatism and pathology analysis, as well as facial reconstruction. The potentialities and limitations of the employed approaches are described, and we conclude with a discussion about methodological issues and future research.Spanish Ministry of Science, Innovation and UniversitiesEuropean Union (EU) PGC2018-101216-B-I00Regional Government of Andalusia under grant EXAISFI P18-FR-4262Instituto de Salud Carlos IIIEuropean Union (EU) DTS18/00136European Commission H2020-MSCA-IF-2016 through the Skeleton-ID Marie Curie Individual Fellowship 746592Spanish Ministry of Science, Innovation and Universities-CDTI, Neotec program 2019 EXP-00122609/SNEO-20191236European Union (EU)Xunta de Galicia ED431G 2019/01European Union (EU) RTI2018-095894-B-I0

Towards Individualized Transcranial Electric Stimulation Therapy through Computer Simulation

Transkranielle Elektrostimulation (tES) beschreibt eine Gruppe von Hirnstimulationstechniken, die einen schwachen elektrischen Strom über zwei nicht-invasiv am Kopf angebrachten Elektroden applizieren. Handelt es sich dabei um einen Gleichstrom, spricht man von transkranieller Gleichstromstimulation, auch tDCS abgekürzt. Die allgemeine Zielstellung aller Hirnstimulationstechniken ist Hirnfunktion durch ein Verstärken oder Dämpfen von Hirnaktivität zu beeinflussen. Unter den Stimulationstechniken wird die transkranielle Gleichstromstimulation als ein adjuvantes Werkzeug zur Unterstützung der mikroskopischen Reorganisation des Gehirnes in Folge von Lernprozessen und besonders der Rehabilitationstherapie nach einem Schlaganfall untersucht. Aktuelle Herausforderungen dieser Forschung sind eine hohe Variabilität im erreichten Stimulationseffekt zwischen den Probanden sowie ein unvollständiges Verständnis des Zusammenspiels der der Stimulation zugrundeliegenden Mechanismen. Als Schlüsselkomponente für das Verständnis der Stimulationsmechanismen wird das zwischen den Elektroden im Kopf des Probanden aufgebaute elektrische Feld erachtet. Einem grundlegenden Konzept folgend wird angenommen, dass Hirnareale, die einer größeren elektrischen Feldstärke ausgesetzt sind, ebenso einen höheren Stimulationseffekt erfahren. Damit kommt der Positionierung der Elektroden eine entscheidende Rolle für die Stimulation zu. Allerdings verteilt sich das elektrische Feld wegen des heterogenen elektrischen Leitfähigkeitsprofil des menschlichen Kopfes nicht uniform im Gehirn der Probanden. Außerdem ist das Verteilungsmuster auf Grund anatomischer Unterschiede zwischen den Probanden verschieden. Die triviale Abschätzung der Ausbreitung des elektrischen Feldes anhand der bloßen Position der Stimulationselektroden ist daher nicht ausreichend genau für eine zielgerichtete Stimulation. Computerbasierte, biophysikalische Simulationen der transkraniellen Elektrostimulation ermöglichen die individuelle Approximation des Verteilungsmusters des elektrischen Feldes in Probanden basierend auf deren medizinischen Bildgebungsdaten. Sie werden daher zunehmend verwendet, um tDCS-Anwendungen zu planen und verifizieren, und stellen ein wesentliches Hilfswerkzeug auf dem Weg zu individualisierter Schlaganfall-Rehabilitationstherapie dar. Softwaresysteme, die den dahinterstehenden individualisierten Verarbeitungsprozess erleichtern und für ein breites Feld an Forschern zugänglich machen, wurden in den vergangenen Jahren für den Anwendungsfall in gesunden Erwachsenen entwickelt. Jedoch bleibt die Simulation von Patienten mit krankhaftem Hirngewebe und strukturzerstörenden Läsionen eine nicht-triviale Aufgabe. Daher befasst sich das hier vorgestellte Projekt mit dem Aufbau und der praktischen Anwendung eines Arbeitsablaufes zur Simulation transkranieller Elektrostimulation. Dabei stand die Anforderung im Vordergrund medizinische Bildgebungsdaten insbesondere neurologischer Patienten mit krankhaft verändertem Hirngewebe verarbeiten zu können. Der grundlegende Arbeitsablauf zur Simulation wurde zunächst für gesunde Erwachsene entworfen und validiert. Dies umfasste die Zusammenstellung medizinischer Bildverarbeitungsalgorithmen zu einer umfangreichen Verarbeitungskette, um elektrisch relevante Strukturen in den Magnetresonanztomographiebildern des Kopfes und des Oberkörpers der Probanden zu identifizieren und zu extrahieren. Die identifizierten Strukturen mussten in Computermodelle überführt werden und das zugrundeliegende, physikalische Problem der elektrischen Volumenleitung in biologischen Geweben mit Hilfe numerischer Simulation gelöst werden. Im Verlauf des normalen Alterns ist das Gehirn strukturellen Veränderungen unterworfen, unter denen ein Verlust des Hirnvolumens sowie die Ausbildung mikroskopischer Veränderungen seiner Nervenfaserstruktur die Bedeutendsten sind. In einem zweiten Schritt wurde der Arbeitsablauf daher erweitert, um diese Phänomene des normalen Alterns zu berücksichtigen. Die vordergründige Herausforderung in diesem Teilprojekt war die biophysikalische Modellierung der veränderten Hirnmikrostruktur, da die resultierenden Veränderungen im Leitfähigkeitsprofil des Gehirns bisher noch nicht in der Literatur quantifiziert wurden. Die Erweiterung des Simulationsablauf zeichnete sich vorrangig dadurch aus, dass mit unsicheren elektrischen Leitfähigkeitswerten gearbeitet werden konnte. Damit war es möglich den Einfluss der ungenau bestimmbaren elektrischen Leitfähigkeit der verschiedenen biologischen Strukturen des menschlichen Kopfes auf das elektrische Feld zu ermitteln. In einer Simulationsstudie, in der Bilddaten von 88 Probanden einflossen, wurde die Auswirkung der veränderten Hirnfaserstruktur auf das elektrische Feld dann systematisch untersucht. Es wurde festgestellt, dass sich diese Gewebsveränderungen hochgradig lokal und im Allgemeinen gering auswirken. Schließlich wurden in einem dritten Schritt Simulationen für Schlaganfallpatienten durchgeführt. Ihre großen, strukturzerstörenden Läsionen wurden dabei mit einem höheren Detailgrad als in bisherigen Arbeiten modelliert und physikalisch abermals mit unsicheren Leitfähigkeiten gearbeitet, was zu unsicheren elektrischen Feldabschätzungen führte. Es wurden individuell berechnete elektrische Felddaten mit der Hirnaktivierung von 18 Patienten in Verbindung gesetzt, unter Berücksichtigung der inhärenten Unsicherheit in der Bestimmung der elektrischen Felder. Das Ziel war zu ergründen, ob die Hirnstimulation einen positiven Einfluss auf die Hirnaktivität der Patienten im Kontext von Rehabilitationstherapie ausüben und so die Neuorganisierung des Gehirns nach einem Schlaganfall unterstützen kann. Während ein schwacher Zusammenhang hergestellt werden konnte, sind weitere Untersuchungen nötig, um diese Frage abschließend zu klären.:Kurzfassung Abstract Contents 1 Overview 2 Anatomical structures in magnetic resonance images 2 Anatomical structures in magnetic resonance images 2.1 Neuroanatomy 2.2 Magnetic resonance imaging 2.3 Segmentation of MR images 2.4 Image morphology 2.5 Summary 3 Magnetic resonance image processing pipeline 3.1 Introduction to human body modeling 3.2 Description of the processing pipeline 3.3 Intermediate and final outcomes in two subjects 3.4 Discussion, limitations & future work 3.5 Conclusion 4 Numerical simulation of transcranial electric stimulation 4.1 Electrostatic foundations 4.2 Discretization of electrostatic quantities 4.3 The numeric solution process 4.4 Spatial discretization by volume meshing 4.5 Summary 5 Simulation workflow 5.1 Overview of tES simulation pipelines 5.2 My implementation of a tES simulation workflow 5.3 Verification & application examples 5.4 Discussion & Conclusion 6 Transcranial direct current stimulation in the aging brain 6.1 Handling age-related brain changes in tES simulations 6.2 Procedure of the simulation study 6.3 Results of the uncertainty analysis 6.4 Findings, limitations and discussion 7 Transcranial direct current stimulation in stroke patients 7.1 Bridging the gap between simulated electric fields and brain activation in stroke patients 7.2 Methodology for relating simulated electric fields to functional MRI data 7.3 Evaluation of the simulation study and correlation analysis 7.4 Discussion & Conclusion 8 Outlooks for simulations of transcranial electric stimulation List of Figures List of Tables Glossary of Neuroscience Terms Glossary of Technical Terms BibliographyTranscranial electric current stimulation (tES) denotes a group of brain stimulation techniques that apply a weak electric current over two or more non-invasively, head-mounted electrodes. When employing a direct-current, this method is denoted transcranial direct current stimulation (tDCS). The general aim of all tES techniques is the modulation of brain function by an up- or downregulation of brain activity. Among these, transcranial direct current stimulation is investigated as an adjuvant tool to promote processes of the microscopic reorganization of the brain as a consequence of learning and, more specifically, rehabilitation therapy after a stroke. Current challenges of this research are a high variability in the achieved stimulation effects across subjects and an incomplete understanding of the interplay between its underlying mechanisms. A key component to understanding the stimulation mechanism is considered the electric field, which is exerted by the electrodes and distributes in the subjects' heads. A principle concept assumes that brain areas exposed to a higher electric field strength likewise experience a higher stimulation. This attributes the positioning of the electrodes a decisive role for the stimulation. However, the electric field distributes non-uniformly across subjects' brains due to the heterogeneous electrical conductivity profile of the human head. Moreover, the distribution pattern is variable between subjects due to their individual anatomy. A trivial estimation of the distribution of the electric field solely based on the position of the stimulating electrodes is, therefore, not precise enough for a well-targeted stimulation. Computer-based biophysical simulations of transcranial electric stimulation enable the individual approximation of the distribution pattern of the electric field in subjects based on their medical imaging data. They are, thus, increasingly employed for the planning and verification of tDCS applications and constitute an essential tool on the way to individualized stroke rehabilitation therapy. Software pipelines facilitating the underlying individualized processing for a wide range of researchers have been developed for use in healthy adults over the past years, but, to date, the simulation of patients with abnormal brain tissue and structure disrupting lesions remains a non-trivial task. Therefore, the presented project was dedicated to establishing and practically applying a tES simulation workflow. The processing of medical imaging data of neurological patients with abnormal brain tissue was a central requirement in this process. The basic simulation workflow was first designed and validated for the simulation of healthy adults. This comprised compiling medical image processing algorithms into a comprehensive workflow to identify and extract electrically relevant physiological structures of the human head and upper torso from magnetic resonance images. The identified structures had to be converted to computational models. The underlying physical problem of electric volume conduction in biological tissue was solved by means of numeric simulation. Over the course of normal aging, the brain is subjected to structural alterations, among which a loss of brain volume and the development of microscopic alterations of its fiber structure are the most relevant. In a second step, the workflow was, thus, extended to incorporate these phenomena of normal aging. The main challenge in this subproject was the biophysical modeling of the altered brain microstructure as the resulting alterations to the conductivity profile of the brain were so far not quantified in the literature. Therefore, the augmentation of the workflow most notably included the modeling of uncertain electrical properties. With this, the influence of the uncertain electrical conductivity of the biological structures of the human head on the electric field could be assessed. In a simulation study, including imaging data of 88 subjects, the influence of the altered brain fiber structure on the electric field was then systematically investigated. These tissue alterations were found to exhibit a highly localized and generally low impact. Finally, in a third step, tDCS simulations of stroke patients were conducted. Their large, structure-disrupting lesions were modeled in a more detailed manner than in previous stroke simulation studies, and they were physically, again, modeled by uncertain electrical conductivity resulting in uncertain electric field estimates. Individually simulated electric fields were related to the brain activation of 18 patients, considering the inherently uncertain electric field estimations. The goal was to clarify whether the stimulation exerts a positive influence on brain function in the context of rehabilitation therapy supporting brain reorganization following a stroke. While a weak correlation could be established, further investigation will be necessary to answer that research question.:Kurzfassung Abstract Contents 1 Overview 2 Anatomical structures in magnetic resonance images 2 Anatomical structures in magnetic resonance images 2.1 Neuroanatomy 2.2 Magnetic resonance imaging 2.3 Segmentation of MR images 2.4 Image morphology 2.5 Summary 3 Magnetic resonance image processing pipeline 3.1 Introduction to human body modeling 3.2 Description of the processing pipeline 3.3 Intermediate and final outcomes in two subjects 3.4 Discussion, limitations & future work 3.5 Conclusion 4 Numerical simulation of transcranial electric stimulation 4.1 Electrostatic foundations 4.2 Discretization of electrostatic quantities 4.3 The numeric solution process 4.4 Spatial discretization by volume meshing 4.5 Summary 5 Simulation workflow 5.1 Overview of tES simulation pipelines 5.2 My implementation of a tES simulation workflow 5.3 Verification & application examples 5.4 Discussion & Conclusion 6 Transcranial direct current stimulation in the aging brain 6.1 Handling age-related brain changes in tES simulations 6.2 Procedure of the simulation study 6.3 Results of the uncertainty analysis 6.4 Findings, limitations and discussion 7 Transcranial direct current stimulation in stroke patients 7.1 Bridging the gap between simulated electric fields and brain activation in stroke patients 7.2 Methodology for relating simulated electric fields to functional MRI data 7.3 Evaluation of the simulation study and correlation analysis 7.4 Discussion & Conclusion 8 Outlooks for simulations of transcranial electric stimulation List of Figures List of Tables Glossary of Neuroscience Terms Glossary of Technical Terms Bibliograph

Construction of Physics-based brain atlas and its application

Ph.DDOCTOR OF PHILOSOPH

Bioengineering Analysis of Traumatic Brain Injury

Traumatic brain injury (TBI) is a serious health concern affecting over a million people in the UK. Brain shift and herniation, which are closely related to severe disability or death, are important signs of abnormally elevated intracranial pressure (ICP) or space-occupying intracranial mass after trauma. This research aims to use medical image computing and biomechanical modelling techniques to characterise the specific deformation field of brain tissues under various TBI scenarios and strengthen the biomechanical understanding across the full spectrum of TBI. Medical image computing provides the research with a solid clinical grounding. To better interpret the neuro-images, three computational tools have been developed, including a CT preprocessing pipeline, an automatic mid-sagittal plane detector and an automatic brain extractor. Using these tools, a novel concept of midplane shift (MPS) is developed to quantitatively evaluate the brain herniation condition across the mid-sagittal plane. In the meantime, a lesion heatmap is generated to quantify the asymmetric haematoma volumes across the mid-sagittal plane. The MPS heatmaps generated for 33 TBI patients with heterogeneous brain pathologies demonstrate highly similar shift patterns. Together with the lesion heatmap, a brain deformation mechanism has been presented: the brain will not deform randomly in response to trauma, instead, it will only deform in a regulated mechanism so that the deformation is directed and restricted to the soft ventricular region, thanks to the anatomic structures of the head such as the falx. The MPS heatmap, the lesion heatmap, together with the novel CT parameters derived from them, provide a rich abundance of information on intracranial brain herniation, for a more complete overview of TBI from medical images. Biomechanical modelling, being one of the most important tools in trauma biomechanics, has been used to quantitatively simulate the brain shift and herniation condition caused by various intracranial lesions and increasing ICP. Preliminary finite element models reconstructed from the Virtual Human Project have demonstrated some limitations. To resolve the observed deficiencies, an advanced high-fidelity patient-specific FE brain model is constructed and explicitly assessed to optimise its injury simulation performance with the help of the developed medical image computing tools. During simulation, the patient-specific traumatic injuries have been reconstructed by imposing both the primary lesion and the secondary injury. The primary lesion simulation is achieved mechanically by ``indenting" a rigid lesion surface simulating the shape of the haematoma to the brain model. While the secondary swelling is modelled with a thermal-expansion-based method to simulate the bulging brain. Using this approach, the observed brain herniation can be decomposed into a deformation due to pure mass effect of space-occupying primary lesion and a shift as a result of secondary swelling. The head injuries of six different TBI patients have been reconstructed and simulated using the prescribed method. The realistic case study suggested that the subdural haematoma patients, as compared to the epidural haematoma patients, were exposed to more significant secondary swelling, which agrees well with the historical clinical findings. In addition to the realistic TBI case studies, an idealised traumatic lesion simulation is performed to investigate the role of lesion morphology and the lesion locations of onsets, in brain herniations during TBI. It is suggested by the idealised TBI cases that the brain is more sensitive to lesion that is more concentrated spatially, if lesion volumes and lesion locations were exactly the same. Moreover, in terms of lesion locations, lesions that strikes on the temporal region and the anterior region are more likely to lead to greater brain deformation, if other lesion morphologies were equal and no secondary swelling considered. Ultimately, the developed tools are expected to help clinicians better understand and predict the brain behaviour after the onset of TBI and during subsequent injury evolution.WD Armstrong Trus