Interactive Visualization of Multimodal Brain Connectivity: Applications in Clinical and Cognitive Neuroscience

Magnetic resonance imaging (MRI) has become a readily available prognostic and diagnostic method, providing invaluable information for the clinical treatment of neurological diseases. Multimodal neuroimaging allows integration of complementary data from various aspects such as functional and anatomical properties; thus, it has the potential to overcome the limitations of each individual modality. Specifically, functional and diffusion MRI are two non-invasive neuroimaging techniques customized to capture brain activity and microstructural properties, respectively. Data from these two modalities is inherently complex, and interactive visualization can assist with data comprehension. The current thesis presents the design, development, and validation of visualization and computation approaches that address the need for integration of brain connectivity from functional and structural domains. Two contexts were considered to develop these approaches: neuroscience exploration and minimally invasive neurosurgical planning. The goal was to provide novel visualization algorithms and gain new insights into big and complex data (e.g., brain networks) by visual analytics. This goal was achieved through three steps: 3D Graphical Collision Detection: One of the primary challenges was the timely rendering of grey matter (GM) regions and white matter (WM) fibers based on their 3D spatial maps. This challenge necessitated pre-scanning those objects to generate a memory array containing their intersections with memory units. This process helped faster retrieval of GM and WM virtual models during the user interactions. Neuroscience Enquiry (MultiXplore): A software interface was developed to display and react to user inputs by means of a connectivity matrix. This matrix displays connectivity information and is capable to accept selections from users and display the relevant ones in 3D anatomical view (with associated anatomical elements). In addition, this package can load multiple matrices from dynamic connectivity methods and annotate brain fibers. Neurosurgical Planning (NeuroPathPlan): A computational method was provided to map the network measures to GM and WM; thus, subject-specific eloquence metric can be derived from related resting state networks and used in objective assessment of cortical and subcortical tissue. This metric was later compared to apriori knowledge based decisions from neurosurgeons. Preliminary results show that eloquence metric has significant similarities with expert decisions

Zifazah: A Scientific Visualization Language for Tensor Field Visualizations

This thesis presents the design and prototype implementation of a scientific visualization language called Zifazah for composing and exploring 3D visualizations of diffusion tensor magnetic resonance imaging (DT-MRI or DTI) data. Unlike existing tools allowing flexible customization of data visualizations that are programmer-oriented, Zifazah focuses on domain scientists as end users in order to enable them to freely compose visualizations of their scientific data set. Verbal descriptions of end users about how they would build and explore DTI visualizations are analyzed to collect syntax, semantics, and control structures of the language. Zifazah makes use of the initial set of lexical terms and semantical patterns to provide a declarative language in the spirit of intuitive syntax and usage. Along with sample scripts representative of the main language design features, some new DTI visualizations created by end users using the novel language have also been presented

Real-time Interactive Tractography Analysis for Multimodal Brain Visualization Tool: MultiXplore

Most debilitating neurological disorders can have anatomical origins. Yet unlike other body organs, the anatomy alone cannot easily provide an understanding of brain functionality. In fact, addressing the challenge of linking structural and functional connectivity remains in the frontiers of neuroscience. Aggregating multimodal neuroimaging datasets may be critical for developing theories that span brain functionality, global neuroanatomy and internal microstructures. Functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI) are main such techniques that are employed to investigate the brain under normal and pathological conditions. FMRI records blood oxygenation level of the grey matter (GM), whereas DTI is able to reveal the underlying structure of the white matter (WM). Brain global activity is assumed to be an integration of GM functional hubs and WM neural pathways that serve to connect them. In this study we developed and evaluated a two-phase algorithm. This algorithm is employed in a 3D interactive connectivity visualization framework and helps to accelerate clustering of virtual neural pathways. In this paper, we will detail an algorithm that makes use of an index-based membership array formed for a whole brain tractography file and corresponding parcellated brain atlas. Next, we demonstrate efficiency of the algorithm by measuring required times for extracting a variety of fiber clusters, which are chosen in such a way to resemble all sizes probable output data files that algorithm will generate. The proposed algorithm facilitates real-time visual inspection of neuroimaging data to further the discovery in structure-function relationship of the brain networks

Interactive Segmentation and Visualization of DTI Data Using a Hierarchical Watershed Representation

Magnetic resonance diffusion tensor imaging (DTI) measures diffusion of water molecules and is used to characterize orientation of white matter fibers and connectivity of neurological structures. Segmentation and visualization of DT images is challenging, because of low data quality and complexity of anatomical structures. In this paper, we propose an interactive segmentation approach, based on a hierarchical representation of the input DT image through a tree structure. The tree is obtained by successively merging watershed regions, based on the morphological waterfall approach, hence the name watershed tree. Region merging is done according to a combined similarity and homogeneity criterion. We introduce filters that work on the proposed tree representation, and that enable region-based attribute filtering of DTI data. Linked views between the visualizations of the simplified DT image and the tree enable a user to visually explore both data and tree at interactive rates. The coupling of filtering, semiautomatic segmentation by labeling nodes in the tree, and various interaction mechanisms support the segmentation task. Our method is robust against noise, which we demonstrate on synthetic and real DTI data