Real-time volume rendering and tractography visualization on the web

In the field of computer graphics, Volume Rendering techniques allow the visualization of 3D datasets, and specifically, Volume Ray-Casting renders images from volumetric datasets, typically used in some scientific areas, such as medical imaging -- This article aims to describe the development of a combined visualization of tractography and volume rendering of brain T1 MRI images in an integrated way -- An innovative web viewer for interactive visualization of neuro-imaging data has been developed based on WebGL -- This recently developed standard enables the clients to use the web viewer on a wide range of devices, with the only requirement of a compliant web-browser -- As the majority of the rendering tasks take place in the client machine, the effect of bottlenecks and server overloading are minimized -- The web application presented is able to compete with desktop tools, even supporting high graphical demands and facing challenges regarding performance and scalability -- The developed software modules are available as open source code and include MRI volume data and tractography generated by the Diffusion Toolkit, and connectivity data from the Connectome Mapping Toolkit -- Our contribution for the Volume Web Viewer implements early ray termination step according to the tractography depthmap, combining volume images and estimated white matter fibers -- Furthermore, the depthmap system extension can be used for visualization of other types of data, where geometric and volume elements are displayed simultaneousl

Improving the Tractography Pipeline: on Evaluation, Segmentation, and Visualization

Recent advances in tractography allow for connectomes to be constructed in vivo. These have applications for example in brain tumor surgery and understanding of brain development and diseases. The large size of the data produced by these methods lead to a variety problems, including how to evaluate tractography outputs, development of faster processing algorithms for tractography and clustering, and the development of advanced visualization methods for verification and exploration. This thesis presents several advances in these fields. First, an evaluation is presented for the robustness to noise of multiple commonly used tractography algorithms. It employs a Monte–Carlo simulation of measurement noise on a constructed ground truth dataset. As a result of this evaluation, evidence for obustness of global tractography is found, and algorithmic sources of uncertainty are identified. The second contribution is a fast clustering algorithm for tractography data based on k–means and vector fields for representing the flow of each cluster. It is demonstrated that this algorithm can handle large tractography datasets due to its linear time and memory complexity, and that it can effectively integrate interrupted fibers that would be rejected as outliers by other algorithms. Furthermore, a visualization for the exploration of structural connectomes is presented. It uses illustrative rendering techniques for efficient presentation of connecting fiber bundles in context in anatomical space. Visual hints are employed to improve the perception of spatial relations. Finally, a visualization method with application to exploration and verification of probabilistic tractography is presented, which improves on the previously presented Fiber Stippling technique. It is demonstrated that the method is able to show multiple overlapping tracts in context, and correctly present crossing fiber configurations

Interactive Visualization of Multimodal Brain Connectivity: Applications in Clinical and Cognitive Neuroscience

Magnetic resonance imaging (MRI) has become a readily available prognostic and diagnostic method, providing invaluable information for the clinical treatment of neurological diseases. Multimodal neuroimaging allows integration of complementary data from various aspects such as functional and anatomical properties; thus, it has the potential to overcome the limitations of each individual modality. Specifically, functional and diffusion MRI are two non-invasive neuroimaging techniques customized to capture brain activity and microstructural properties, respectively. Data from these two modalities is inherently complex, and interactive visualization can assist with data comprehension. The current thesis presents the design, development, and validation of visualization and computation approaches that address the need for integration of brain connectivity from functional and structural domains. Two contexts were considered to develop these approaches: neuroscience exploration and minimally invasive neurosurgical planning. The goal was to provide novel visualization algorithms and gain new insights into big and complex data (e.g., brain networks) by visual analytics. This goal was achieved through three steps: 3D Graphical Collision Detection: One of the primary challenges was the timely rendering of grey matter (GM) regions and white matter (WM) fibers based on their 3D spatial maps. This challenge necessitated pre-scanning those objects to generate a memory array containing their intersections with memory units. This process helped faster retrieval of GM and WM virtual models during the user interactions. Neuroscience Enquiry (MultiXplore): A software interface was developed to display and react to user inputs by means of a connectivity matrix. This matrix displays connectivity information and is capable to accept selections from users and display the relevant ones in 3D anatomical view (with associated anatomical elements). In addition, this package can load multiple matrices from dynamic connectivity methods and annotate brain fibers. Neurosurgical Planning (NeuroPathPlan): A computational method was provided to map the network measures to GM and WM; thus, subject-specific eloquence metric can be derived from related resting state networks and used in objective assessment of cortical and subcortical tissue. This metric was later compared to apriori knowledge based decisions from neurosurgeons. Preliminary results show that eloquence metric has significant similarities with expert decisions

3D Polarized Light Imaging Portrayed: Visualization of Fiber Architecture Derived from 3D-PLI

3D polarized light imaging (3D-PLI) is a neuroimaging technique that has recently opened up new avenues to study the complex architecture of nerve fibers in postmortem brains at microscopic scales. In a specific voxel-based analysis, each voxel is assigned a single 3D fiber orientation vector. This leads to comprehensive 3D vector fields. In order to inspect and analyze such high-resolution fiber orientation vector field, also in combination with complementary microscopy measurements, appropriate visualization techniques are essential to overcome several challenges, such as the massive data sizes, the large amount of both unique and redundant information at different scales, or the occlusion issues of inner structures by outer layers. Here, we introduce a comprehensive software tool that is able to visualize all information of a typical 3D-PLI dataset in an adequate and sophisticated manner. This includes the visualization of (i) anatomic structural and fiber architectonic data in one representation, (ii) a large-scale fiber orientation vector field, and (iii) a clustered version of the field. Alignment of a 3D-PLI dataset to an appropriate brain atlas provides expert-based delineation, segmentation, and, ultimately, visualization of selected anatomical structures. By means of these techniques, a detailed analysis of the complex fiber architecture in 3D is feasible

Fluorescence microscopy tensor imaging representations for large-scale dataset analysis

Understanding complex biological systems requires the system-wide characterization of cellular and molecular features. Recent advances in optical imaging technologies and chemical tissue clearing have facilitated the acquisition of whole-organ imaging datasets, but automated tools for their quantitative analysis and visualization are still lacking. We have here developed a visualization technique capable of providing whole-organ tensor imaging representations of local regional descriptors based on fluorescence data acquisition. This method enables rapid, multiscale, analysis and virtualization of large-volume, high-resolution complex biological data while generating 3D tractographic representations. Using the murine heart as a model, our method allowed us to analyze and interrogate the cardiac microvasculature and the tissue resident macrophage distribution and better infer and delineate the underlying structural network in unprecedented detail