Interactive Visualization of Multimodal Brain Connectivity: Applications in Clinical and Cognitive Neuroscience

Magnetic resonance imaging (MRI) has become a readily available prognostic and diagnostic method, providing invaluable information for the clinical treatment of neurological diseases. Multimodal neuroimaging allows integration of complementary data from various aspects such as functional and anatomical properties; thus, it has the potential to overcome the limitations of each individual modality. Specifically, functional and diffusion MRI are two non-invasive neuroimaging techniques customized to capture brain activity and microstructural properties, respectively. Data from these two modalities is inherently complex, and interactive visualization can assist with data comprehension. The current thesis presents the design, development, and validation of visualization and computation approaches that address the need for integration of brain connectivity from functional and structural domains. Two contexts were considered to develop these approaches: neuroscience exploration and minimally invasive neurosurgical planning. The goal was to provide novel visualization algorithms and gain new insights into big and complex data (e.g., brain networks) by visual analytics. This goal was achieved through three steps: 3D Graphical Collision Detection: One of the primary challenges was the timely rendering of grey matter (GM) regions and white matter (WM) fibers based on their 3D spatial maps. This challenge necessitated pre-scanning those objects to generate a memory array containing their intersections with memory units. This process helped faster retrieval of GM and WM virtual models during the user interactions. Neuroscience Enquiry (MultiXplore): A software interface was developed to display and react to user inputs by means of a connectivity matrix. This matrix displays connectivity information and is capable to accept selections from users and display the relevant ones in 3D anatomical view (with associated anatomical elements). In addition, this package can load multiple matrices from dynamic connectivity methods and annotate brain fibers. Neurosurgical Planning (NeuroPathPlan): A computational method was provided to map the network measures to GM and WM; thus, subject-specific eloquence metric can be derived from related resting state networks and used in objective assessment of cortical and subcortical tissue. This metric was later compared to apriori knowledge based decisions from neurosurgeons. Preliminary results show that eloquence metric has significant similarities with expert decisions

Rapid development of applications for the interactive visual analysis of multimodal medical data

Multimodale medizinische Volumendaten gewinnen zunehmend an Verbreitung. Wir diskutieren verschiedene interaktive Applikationen welche den Nutzer bei der Analyse dieser Daten unterstützen. Alle Applikationen basieren auf Erweiterungen des Voreen Frameworks, welche ebenfalls in dieser Dissertation diskutiert werden. With multimodal volumetric medical data sets becoming more common due to the increasing availability of scanning hardware, software for the visualization and analysis of such data sets needs to become more efficient as well in order to prevent overloading the user with data. This dissertation presents several interactive techniques for the visual analysis of medical volume data. All applications are based on extensions to the Voreen volume rendering framework, which we will discuss first. Since visual analysis applications are interactive by definition, we propose a general-purpose navigation technique for volume data. Next, we discuss our concepts for the interactive planning of brain tumor resections. Finally, we present two systems designed to work with images of vasculature. First, we discuss an interactive vessel segmentation system enabling an efficient, visually supported workflow. Second, we propose an application for the visual analysis of PET tracer uptake along vessels

Structure-Function Relationships in the Brain: Applications in Neurosurgery

Multimodal brain imaging allows the study of structure-function relationships of the brain at the individual level, a key subject in basic neuroscience with important applications in neurosurgery. The current thesis aims to better understand these relationships by (1) examining how cortical morphology metrics influence measures of brain function, (2) their visualization in augmented reality (AR), and (3) their application in neurosurgical planning. To achieve these objectives, we made use of multimodal magnetic resonance imaging (MRI) data: diffusion weighted imaging, resting-state functional MRI (rs-fMRI), task-based fMRI, and T1-weighted images. Various metrics were calculated: cortical thickness (CT), blood oxygen level dependent signal variability (BOLDSD), structural connectivity (SC), functional connectivity (FC), etc.. We found that BOLDSD measures are confounded by CT, developed an application to visualize SC and FC in AR, and used rs-fMRI to map language for epilepsy surgery. Overall, these studies provided a better understanding of structure-function relationships in the brain

Quantitative MRI correlates of hippocampal and neocortical pathology in intractable temporal lobe epilepsy

Intractable or drug-resistant epilepsy occurs in over 30% of epilepsy patients, with many of these patients undergoing surgical excision of the affected brain region to achieve seizure control. Advances in MRI have the potential to improve surgical treatment of epilepsy through improved identification and delineation of lesions. However, validation is currently needed to investigate histopathological correlates of these new imaging techniques. The purpose of this work is to investigate histopathological correlates of quantitative relaxometry and DTI from hippocampal and neocortical specimens of intractable TLE patients. To achieve this goal I developed and evaluated a pipeline for histology to in-vivo MRI image registration, which finds dense spatial correspondence between both modalities. This protocol was divided in two steps whereby sparsely sectioned histology from temporal lobe specimens was first registered to the intermediate ex-vivo MRI which is then registered to the in-vivo MRI, completing a pipeline for histology to in-vivo MRI registration. When correlating relaxometry and DTI with neuronal density and morphology in the temporal lobe neocortex, I found T1 to be a predictor of neuronal density in the neocortical GM and demonstrated that employing multi-parametric MRI (combining T1 and FA together) provided a significantly better fit than each parameter alone in predicting density of neurons. This work was the first to relate in-vivo T1 and FA values to the proportion of neurons in GM. When investigating these quantitative multimodal parameters with histological features within the hippocampal subfields, I demonstrated that MD correlates with neuronal density and size, and can act as a marker for neuron integrity within the hippocampus. More importantly, this work was the first to highlight the potential of subfield relaxometry and diffusion parameters (mainly T2 and MD) as well as volumetry in predicting the extent of cell loss per subfield pre-operatively, with a precision so far unachievable. These results suggest that high-resolution quantitative MRI sequences could impact clinical practice for pre-operative evaluation and prediction of surgical outcomes of intractable epilepsy

What scans we will read: imaging instrumentation trends in clinical oncology

Oncological diseases account for a significant portion of the burden on public healthcare systems with associated costs driven primarily by complex and long-lasting therapies. Through the visualization of patient-specific morphology and functional-molecular pathways, cancerous tissue can be detected and characterized non- invasively, so as to provide referring oncologists with essential information to support therapy management decisions. Following the onset of stand-alone anatomical and functional imaging, we witness a push towards integrating molecular image information through various methods, including anato-metabolic imaging (e.g., PET/ CT), advanced MRI, optical or ultrasound imaging. This perspective paper highlights a number of key technological and methodological advances in imaging instrumentation related to anatomical, functional, molecular medicine and hybrid imaging, that is understood as the hardware-based combination of complementary anatomical and molecular imaging. These include novel detector technologies for ionizing radiation used in CT and nuclear medicine imaging, and novel system developments in MRI and optical as well as opto-acoustic imaging. We will also highlight new data processing methods for improved non-invasive tissue characterization. Following a general introduction to the role of imaging in oncology patient management we introduce imaging methods with well-defined clinical applications and potential for clinical translation. For each modality, we report first on the status quo and point to perceived technological and methodological advances in a subsequent status go section. Considering the breadth and dynamics of these developments, this perspective ends with a critical reflection on where the authors, with the majority of them being imaging experts with a background in physics and engineering, believe imaging methods will be in a few years from now. Overall, methodological and technological medical imaging advances are geared towards increased image contrast, the derivation of reproducible quantitative parameters, an increase in volume sensitivity and a reduction in overall examination time. To ensure full translation to the clinic, this progress in technologies and instrumentation is complemented by progress in relevant acquisition and image-processing protocols and improved data analysis. To this end, we should accept diagnostic images as “data”, and – through the wider adoption of advanced analysis, including machine learning approaches and a “big data” concept – move to the next stage of non-invasive tumor phenotyping. The scans we will be reading in 10 years from now will likely be composed of highly diverse multi- dimensional data from multiple sources, which mandate the use of advanced and interactive visualization and analysis platforms powered by Artificial Intelligence (AI) for real-time data handling by cross-specialty clinical experts with a domain knowledge that will need to go beyond that of plain imaging

Diffusion Tensor Imaging in Pediatric Brain Tumor Patients

In this dissertation, we outline our efforts to introduce an advanced MRI imaging technique called Diffusion Tensor Imaging (DTI) to the pediatric brain tumor population. We discuss the theory and application of DTI as it was performed in a series of translational investigations at St. Jude Children’s Research Hospital. We present evidence of how the introduction of this technique impacted diagnosis, and treatment. And finally, we demonstrate how DTI was used to investigate cognitive morbidities associated with cancer treatment and how this research provided insight into the underlying pathophysiology involved in the development of these treatment sequela. This research has generated important insights into the fundamental causes of neuroanatomical and cognitive deficits associated with cancer and cancer therapy. The use of DTI has permitted us to identify potential targets for improved radiological and surgical techniques as well as targets for pharmacological and behavioral interventions that might improve cognitive function in cancer survivors. The discoveries here afford us an opportunity to reduce the negative effects of cancer therapy on patients treated in the future while maintaining successful survival rates

Modern Developments in Transcranial Magnetic Stimulation (TMS) – Applications and Perspectives in Clinical Neuroscience

Transcranial magnetic stimulation (TMS) is being increasingly used in neuroscience and clinics. Modern advances include but are not limited to the combination of TMS with precise neuronavigation as well as the integration of TMS into a multimodal environment, e.g., by guiding the TMS application using complementary techniques such as functional magnetic resonance imaging (fMRI), electroencephalography (EEG), diffusion tensor imaging (DTI), or magnetoencephalography (MEG). Furthermore, the impact of stimulation can be identified and characterized by such multimodal approaches, helping to shed light on the basic neurophysiology and TMS effects in the human brain. Against this background, the aim of this Special Issue was to explore advancements in the field of TMS considering both investigations in healthy subjects as well as patients