Medical Visualization and Simulation for Customizable Surgical Guides

This thesis revolves around the development of medical visualization tools for the planning of CSG-based surgery. To this end, we performed an extensive computerassisted surgery (CAS) literature study, developed a novel optimization technique for customizable surgical guides (CSG), and introduce three visualization techniques to make the planning more realistic and allow for remote visualization. In Chapter 2 we document the results of an extensive overview study, in which the use of visualization in CAS is analysed. We collected a comprehensive database of visualization relevant CAS publications, and analyse the visualization techniques that are used. We also classify important CAS-related surgical tasks and explain how and why visualization is used. Further, we analysed how surgical plans are transferred to the operating theater. Finally, we discuss how visualization is used in the four most prominent application areas of CAS. Based on this review, we were able to pinpoint interesting new research directions. One of these is the apparent lack of proper tools for CSG-based surgery, a challenge that we addressed in Chapter 3. The optimization of CSG parameters such that the CSG can be docked on bone in an accurate and stable way, is important in the planning of CSG-based surgery. The adjustable nature of the CSG, which allows it to become patient-specific, unfortunately also makes it inherently unstable. Optimizing the configuration by hand leads to poor results as we demonstrated with experiments. In Chapter 3, we therefore solve the problem in sillico. We described a novel planning tool that is able to automatically optimize a CSG for an arbitrary patient. We established this by combining a physical simulator, which models the physical interaction between the CSG and the bone, with a genetic optimization process. With experiments, we were able to prove that our optimization tool produces CSG configurations that lead to accurate and stable docking. In Chapter 4, we address the challenge of enhancing the planning environment with appropriate visualization techniques that help to understand how a CSG is connected to the bone. The state-of-the-art rendering tools in CAS applications are not able to accurately and effectively communicate how the CSG attaches to the bone. However, ambient occlusion (AO) is an illumination technique that is particularly effective at depicting contact between objects, but is generally computationally expensive. Therefore, we developed an efficient version of this algorithm such, that it can be used in the planning pipeline to effectively depict CSG-bone contact. We took the visualization one step further by introducing photo-realistic and physically based volume rendering. Chapter 5 describes Exposure Render, a complete volume rendering framework based on stochastic raytracing, and is able to incorporate a host of otherwise difficult to obtain photorealistic camera, light, and material effects. It is a well known fact that these help to understand shape, depth and size. Therefore, we employed Exposure Render to build a prototype doctor-patient communication system. With this remote visualization system, a doctor can counsel a patient from a distance, or a patient can perform self health management by uploading their tomographic data. In Chapter 6 we optimize the performance of Exposure Render. We introduce visibility sweeps, an efficient method to compute and store visibility information in volume data sets. With this method, it becomes possible to efficiently query approximate global visibility information in a volume data set. We demonstrate that this visibility information can be harnessed to improve the efficiency of the ray sampling processes in Exposure Render, which results in faster convergence. Though we demonstrate the effectiveness of visibility sweeps in the context of stochastic volume rendering, its use stretches beyond this application. Many areas of medical visualization and CAS rely on visibility information, such as automatic view finding in volume data and in various areas of CAS e.g., access, resection and implant planning. In our project it is also relevant because the visibility information can be used to make the physical simulator more realistic, for instance by avoiding docking trajectories that are associated with high risk of tissue damage. The research described in this thesis was part of the project Novel pre-operative planning and intraoperative guidance system for shoulder replacement surgery (10812), funded by the Dutch Technology Foundation.Intelligent SystemsElectrical Engineering, Mathematics and Computer Scienc

Numerical optimization of alignment reproducibility for customizable surgical guides

Purpose Computer-assisted orthopedic surgery aims at minimizing invasiveness, postoperative pain, and morbidity with computer-assisted preoperative planning and intra-operative guidance techniques, of which camera-based navigation and patient-specific templates (PST) are the most common. PSTs are one-time templates that guide the surgeon initially in cutting slits or drilling holes. This method can be extended to reusable and customizable surgical guides (CSG), which can be adapted to the patients’ bone. Determining the right set of CSG input parameters by hand is a challenging task, given the vast amount of input parameter combinations and the complex physical interaction between the PST/CSG and the bone. Methods This paper introduces a novel algorithm to solve the problem of choosing the right set of input parameters. Our approach predicts how well a CSG instance is able to reproduce the planned alignment based on a physical simulation and uses a genetic optimization algorithm to determine optimal configurations. We validate our technique with a prototype of a pin-based CSG and nine rapid prototyped distal femora. Results The proposed optimization technique has been compared to manual optimization by experts, as well as participants with domain experience. Using the optimization technique, the alignment errors remained within practical boundaries of 1.2 mm translation and 0.9? rotation error. In all cases, the proposed method outperformed manual optimization. Conclusions Manually optimizing CSG parameters turns out to be a counterintuitive task. Even after training, subjects with and without anatomical background fail in choosing appropriate CSG configurations. Our optimization algorithm ensures that the CSG is configured correctly, and we could demonstrate that the intended alignment of the CSG is accurately reproduced on all tested bone geometries.Intelligent SystemsElectrical Engineering, Mathematics and Computer Scienc

Visibility sweeps for joint-hierarchical importance sampling of direct lighting for stochastic volume rendering

Physically-based light transport in heterogeneous volumetric data is computationally expensive because the rendering integral (particularly visibility) has to be stochastically solved. We present a visibility estimation method in concert with an importance-sampling technique for efficient and unbiased stochastic volume rendering. Our solution relies on a joint strategy, which involves the environmental illumination and visibility inside of the volume. A major contribution of our method is a fast sweeping-plane algorithm to progressively estimate partial occlusions at discrete locations, where we store the result using an octahedral representation. We then rely on a quadtree-based hierarchy to perform a joint importance sampling. Our technique is unbiased, requires little precomputation, is highly parallelizable, and is applicable to a various volume data sets, dynamic transfer functions, and changing environmental lighting.Intelligent SystemsElectrical Engineering, Mathematics and Computer Scienc

Visual Analysis of RIS Data for Endmember Selection

Reflectance Imaging Spectroscopy (RIS) is a hyperspectral imaging technique used for investigating the molecular composition of materials. It can help identify pigments used in a painting, which are relevant information for art conservation and history. For every scanned pixel, a reflectance spectrum is obtained and domain experts look for pure representative spectra, called endmembers, which could indicate the presence of particular pigments. However, the identification of endmembers can be a lengthy process, which requires domain experts to manually select pixels and visually inspect multiple spectra in order to find accurate endmembers that belong to the historical context of an investigated painting. We propose an integrated interactive visual-analysis workflow, that combines dimensionality reduction and linked visualizations to identify and inspect endmembers. Here, we present initial results, obtained in collaboration with domain experts.Team Matthias AlfeldComputer Graphics and Visualisatio

SpaceWalker enables interactive gradient exploration for spatial transcriptomics data

In spatial transcriptomics (ST) data, biologically relevant features such as tissue compartments or cell-state transitions are reflected by gene expression gradients. Here, we present SpaceWalker, a visual analytics tool for exploring the local gradient structure of 2D and 3D ST data. The user can be guided by the local intrinsic dimensionality of the high-dimensional data to define seed locations, from which a flood-fill algorithm identifies transcriptomically similar cells on the fly, based on the high-dimensional data topology. In several use cases, we demonstrate that the spatial projection of these flooded cells highlights tissue architectural features and that interactive retrieval of gene expression gradients in the spatial and transcriptomic domains confirms known biology. We also show that SpaceWalker generalizes to several different ST protocols and scales well to large, multi-slice, 3D whole-brain ST data while maintaining real-time interaction performance.Pattern Recognition and BioinformaticsComputer Graphics and Visualisatio

ManiVault: A Flexible and Extensible Visual Analytics Framework for High-Dimensional Data

Exploration and analysis of high-dimensional data are important tasks in many fields that produce large and complex data, like the financial sector, systems biology, or cultural heritage. Tailor-made visual analytics software is developed for each specific application, limiting their applicability in other fields. However, as diverse as these fields are, their characteristics and requirements for data analysis are conceptually similar. Many applications share abstract tasks and data types and are often constructed with similar building blocks. Developing such applications, even when based mostly on existing building blocks, requires significant engineering efforts. We developed ManiVault, a flexible and extensible open-source visual analytics framework for analyzing high-dimensional data. The primary objective of ManiVault is to facilitate rapid prototyping of visual analytics workflows for visualization software developers and practitioners alike. ManiVault is built using a plugin-based architecture that offers easy extensibility. While our architecture deliberately keeps plugins self-contained, to guarantee maximum flexibility and re-usability, we have designed and implemented a messaging API for tight integration and linking of modules to support common visual analytics design patterns. We provide several visualization and analytics plugins, and ManiVault's API makes the integration of new plugins easy for developers. ManiVault facilitates the distribution of visualization and analysis pipelines and results for practitioners through saving and reproducing complete application states. As such, ManiVault can be used as a communication tool among researchers to discuss workflows and results. A copy of this paper and all supplemental material is available at osf.io/9k6jw, and source code at github.com/ManiVaultStudio.Computer Graphics and VisualisationPattern Recognition and Bioinformatic