An Empirical Evaluation of Visual Cues for 3D Flow Field Perception

Three-dimensional vector fields are common datasets throughout the sciences. They often represent physical phenomena that are largely invisible to us in the real world, like wind patterns and ocean currents. Computer-aided visualization is a powerful tool that can represent data in any way we choose through digital graphics. Visualizing 3D vector fields is inherently difficult due to issues such as visual clutter, self-occlusion, and the difficulty of providing depth cues that adequately support the perception of flow direction in 3D space. Cutting planes are often used to overcome these issues by presenting slices of data that are more cognitively manageable. The existing literature provides many techniques for visualizing the flow through these cutting planes; however, there is a lack of empirical studies focused on the underlying perceptual cues that make popular techniques successful. The most valuable depth cue for the perception of other kinds of 3D data, notably 3D networks and 3D point clouds, is structure-from-motion (also called the Kinetic Depth Effect); another powerful depth cue is stereoscopic viewing, but none of these cues have been fully examined in the context of flow visualization. This dissertation presents a series of quantitative human factors studies that evaluate depth and direction cues in the context of cutting plane glyph designs for exploring and analyzing 3D flow fields. The results of the studies are distilled into a set of design guidelines to improve the effectiveness of 3D flow field visualizations, and those guidelines are implemented as an immersive, interactive 3D flow visualization proof-of-concept application

Illustrative Flow Visualization of 4D PC-MRI Blood Flow and CFD Data

Das zentrale Thema dieser Dissertation ist die Anwendung illustrativer Methoden auf zwei bisher ungelöste Probleme der Strömungsvisualisierung. Das Ziel der Strömungsvisualisierung ist die Bereitstellung von Software, die Experten beim Auswerten ihrer Strömungsdaten und damit beim Erkenntnisgewinn unterstützt. Bei der illustrativen Visualisierung handelt es sich um einen Zweig der Visualisierung, der sich an der künstlerischen Arbeit von Illustratoren orientiert. Letztere sind darauf spezialisiert komplizierte Zusammenhänge verständlich und ansprechend zu vermitteln. Die angewendeten Techniken werden in der illustrativen Visualisierung auf reale Daten übertragen, um die Effektivität der Darstellung zu erhöhen. Das erste Problem, das im Rahmen dieser Dissertation bearbeitet wurde, ist die eingeschränkte Verständlichkeit von komplexen Stromflächen. Selbstverdeckungen oder Aufrollungen behindern die Form- und Strömungswahrnehmung und machen diese Flächen gerade in interessanten Strömungssituationen wenig nützlich. Auf Basis von handgezeichneten Strömungsdarstellungen haben wir ein Flächenrendering entwickelt, das Silhouetten, nicht-photorealistische Beleuchtung und illustrative Stromlinien verwendet. Interaktive Flächenschnitte erlauben die Exploration der Flächen und der Strömungen, die sie repräsentieren. Angewendet auf verschiedene Stromflächen ließ sich zeigen, dass die Methoden die Verständlichkeit erhöhen, v.a. in Bereichen komplexer Strömung mit Aufwicklungen oder Singularitäten. Das zweite Problem ist die Strömungsanalyse des Blutes aus 4D PC-MRI-Daten. An diese relativ neue Datenmodalität werden hohe Erwartungen für die Erforschung und Behandlung kardiovaskulärer Krankheiten geknüpft, da sie erstmals ein dreidimensionales, zeitlich aufgelöstes Abbild der Hämodynamik liefert. Bisher werden 4D PC-MRI-Daten meist mit Werkzeugen der klassischen Strömungsvisualisierung verarbeitet. Diese werden den besonderen Ansprüchen der medizinischen Anwender jedoch nicht gerecht, die in kurzer Zeit eine übersichtliche Darstellung der relevanten Strömungsaspekte erhalten möchten. Wir haben ein Werkzeug zur visuellen Analyse der Blutströmung entwickelt, welches eine einfache Detektion von markanten Strömungsmustern erlaubt, wie z.B. Jets, Wirbel oder Bereiche mit hoher Blutverweildauer. Die Grundidee ist hierbei aus vorberechneten Integrallinien mit Hilfe speziell definierter Linienprädikate die relevanten, d.h. am gefragten Strömungsmuster, beteiligten Linien ausgewählt werden. Um eine intuitive Darstellung der Resultate zu erreichen, haben wir uns von Blutflußillustrationen inspirieren lassen und präsentieren eine abstrakte Linienbündel- und Wirbeldarstellung. Die Linienprädikatmethode sowie die abstrakte Darstellung der Strömungsmuster wurden an 4D PC-MRI-Daten von gesunden und pathologischen Aorten- und Herzdaten erfolgreich getestet. Auch die Evaluierung durch Experten zeigt die Nützlichkeit der Methode und ihr Potential für den Einsatz in der Forschung und der Klinik.This thesis’ central theme is the use of illustrative methods to solve flow visualization problems. The goal of flow visualization is to provide users with software tools supporting them analyzing and extracting knowledge from their fluid dynamics data. This fluid dynamics data is produced in large amounts by simulations or measurements to answer diverse questions in application fields like engineering or medicine. This thesis deals with two unsolved problems in flow visualization and tackles them with methods of illustrative visualization. The latter is a subbranch of visualization whose methods are inspired by the art work of professional illustrators. They are specialized in the comprehensible and esthetic representation of complex knowledge. With illustrative visualization, their techniques are applied to real data to enhance their representation. The first problem dealt with in this thesis is the limited shape and flow perception of complex stream surfaces. Self-occlusion and wrap-ups hinder their effective use in the most interesting flow situations. On the basis of hand-drawn flow illustrations, a surface rendering method was designed that uses silhouettes, non-photorealistic shading, and illustrative surface stream lines. Additionally, geometrical and flow-based surface cuts allow the user an interactive exploration of the surface and the flow it represents. By applying this illustrative technique to various stream surfaces and collecting expert feedback, we could show that the comprehensibility of the stream surfaces was enhanced – especially in complex areas with surface wrap-ups and singularities. The second problem tackled in this thesis is the analysis of blood flow from 4D PC-MRI data. From this rather young data modality, medical experts expect many advances in the research of cardiovascular diseases because it delivers a three-dimensional and time-resolved image of the hemodynamics. However, 4D PC-MRI data are mainly processed with standard flow visualizaton tools, which do not fulfill the requirements of medical users. They need a quick and easy-to-understand display of the relevant blood flow aspects. We developed a tool for the visual analysis of blood flow that allows a fast detection of distinctive flow patterns, such as high-velocity jets, vortices, or areas with high residence times. The basic idea is to precalculate integral lines and use specifically designed line predicates to select and display only lines involved in the pattern of interest. Traditional blood flow illustrations inspired us to an abstract and comprehensible depiction of the resulting line bundles and vortices. The line predicate method and the illustrative flow pattern representation were successfully tested with 4D PC-MRI data of healthy and pathological aortae and hearts. Also, the feedback of several medical experts confirmed the usefulness of our methods and their capabilities for a future application in the clinical research and routine

Innovative techniques to devise 3D-printed anatomical brain phantoms for morpho-functional medical imaging

Introduction. The Ph.D. thesis addresses the development of innovative techniques to create 3D-printed anatomical brain phantoms, which can be used for quantitative technical assessments on morpho-functional imaging devices, providing simulation accuracy not obtainable with currently available phantoms. 3D printing (3DP) technology is paving the way for advanced anatomical modelling in biomedical applications. Despite the potential already expressed by 3DP in this field, it is still little used for the realization of anthropomorphic phantoms of human organs with complex internal structures. Making an anthropomorphic phantom is very different from making a simple anatomical model and 3DP is still far from being plug-and-print. Hence, the need to develop ad-hoc techniques providing innovative solutions for the realization of anatomical phantoms with unique characteristics, and greater ease-of-use. Aim. The thesis explores the entire workflow (brain MRI images segmentation, 3D modelling and materialization) developed to prototype a new complex anthropomorphic brain phantom, which can simulate three brain compartments simultaneously: grey matter (GM), white matter (WM) and striatum (caudate nucleus and putamen, known to show a high uptake in nuclear medicine studies). The three separate chambers of the phantom will be filled with tissue-appropriate solutions characterized by different concentrations of radioisotope for PET/SPECT, para-/ferro-magnetic metals for MRI, and iodine for CT imaging. Methods. First, to design a 3D model of the brain phantom, it is necessary to segment MRI images and to extract an error-less STL (Standard Tessellation Language) description. Then, it is possible to materialize the prototype and test its functionality. - Image segmentation. Segmentation is one of the most critical steps in modelling. To this end, after demonstrating the proof-of-concept, a multi-parametric segmentation approach based on brain relaxometry was proposed. It includes a pre-processing step to estimate relaxation parameter maps (R1 = longitudinal relaxation rate, R2 = transverse relaxation rate, PD = proton density) from the signal intensities provided by MRI sequences of routine clinical protocols (3D-GrE T1-weighted, FLAIR and fast-T2-weighted sequences with ≤ 3 mm slice thickness). In the past, maps of R1, R2, and PD were obtained from Conventional Spin Echo (CSE) sequences, which are no longer suitable for clinical practice due to long acquisition times. Rehabilitating the multi-parametric segmentation based on relaxometry, the estimation of pseudo-relaxation maps allowed developing an innovative method for the simultaneous automatic segmentation of most of the brain structures (GM, WM, cerebrospinal fluid, thalamus, caudate nucleus, putamen, pallidus, nigra, red nucleus and dentate). This method allows the segmentation of higher resolution brain images for future brain phantom enhancements. - STL extraction. After segmentation, the 3D model of phantom is described in STL format, which represents the shapes through the approximation in manifold mesh (i.e., collection of triangles, which is continuous, without holes and with a positive – not zero – volume). For this purpose, we developed an automatic procedure to extract a single voxelized surface, tracing the anatomical interface between the phantom's compartments directly on the segmented images. Two tubes were designed for each compartment (one for filling and the other to facilitate the escape of air). The procedure automatically checks the continuity of the surface, ensuring that the 3D model could be exported in STL format, without errors, using a common image-to-STL conversion software. Threaded junctions were added to the phantom (for the hermetic closure) using a mesh processing software. The phantom's 3D model resulted correct and ready for 3DP. Prototyping. Finally, the most suitable 3DP technology is identified for the materialization. We investigated the material extrusion technology, named Fused Deposition Modeling (FDM), and the material jetting technology, named PolyJet. FDM resulted the best candidate for our purposes. It allowed materializing the phantom's hollow compartments in a single print, without having to print them in several parts to be reassembled later. FDM soluble internal support structures were completely removable after the materialization, unlike PolyJet supports. A critical aspect, which required a considerable effort to optimize the printing parameters, was the submillimetre thickness of the phantom walls, necessary to avoid distorting the imaging simulation. However, 3D printer manufacturers recommend maintaining a uniform wall thickness of at least 1 mm. The optimization of printing path made it possible to obtain strong, but not completely waterproof walls, approximately 0.5 mm thick. A sophisticated technique, based on the use of a polyvinyl-acetate solution, was developed to waterproof the internal and external phantom walls (necessary requirement for filling). A filling system was also designed to minimize the residual air bubbles, which could result in unwanted hypo-intensity (dark) areas in phantom-based imaging simulation. Discussions and conclusions. The phantom prototype was scanned trough CT and PET/CT to evaluate the realism of the brain simulation. None of the state-of-the-art brain phantoms allow such anatomical rendering of three brain compartments. Some represent only GM and WM, others only the striatum. Moreover, they typically have a poor anatomical yield, showing a reduced depth of the sulci and a not very faithful reproduction of the cerebral convolutions. The ability to simulate the three brain compartments simultaneously with greater accuracy, as well as the possibility of carrying out multimodality studies (PET/CT, PET/MRI), which represent the frontier of diagnostic imaging, give this device cutting-edge prospective characteristics. The effort to further customize 3DP technology for these applications is expected to increase significantly in the coming years

Revealing the Invisible: On the Extraction of Latent Information from Generalized Image Data

The desire to reveal the invisible in order to explain the world around us has been a source of impetus for technological and scientific progress throughout human history. Many of the phenomena that directly affect us cannot be sufficiently explained based on the observations using our primary senses alone. Often this is because their originating cause is either too small, too far away, or in other ways obstructed. To put it in other words: it is invisible to us. Without careful observation and experimentation, our models of the world remain inaccurate and research has to be conducted in order to improve our understanding of even the most basic effects. In this thesis, we1 are going to present our solutions to three challenging problems in visual computing, where a surprising amount of information is hidden in generalized image data and cannot easily be extracted by human observation or existing methods. We are able to extract the latent information using non-linear and discrete optimization methods based on physically motivated models and computer graphics methodology, such as ray tracing, real-time transient rendering, and image-based rendering

Improving the Tractography Pipeline: on Evaluation, Segmentation, and Visualization

Recent advances in tractography allow for connectomes to be constructed in vivo. These have applications for example in brain tumor surgery and understanding of brain development and diseases. The large size of the data produced by these methods lead to a variety problems, including how to evaluate tractography outputs, development of faster processing algorithms for tractography and clustering, and the development of advanced visualization methods for verification and exploration. This thesis presents several advances in these fields. First, an evaluation is presented for the robustness to noise of multiple commonly used tractography algorithms. It employs a Monte–Carlo simulation of measurement noise on a constructed ground truth dataset. As a result of this evaluation, evidence for obustness of global tractography is found, and algorithmic sources of uncertainty are identified. The second contribution is a fast clustering algorithm for tractography data based on k–means and vector fields for representing the flow of each cluster. It is demonstrated that this algorithm can handle large tractography datasets due to its linear time and memory complexity, and that it can effectively integrate interrupted fibers that would be rejected as outliers by other algorithms. Furthermore, a visualization for the exploration of structural connectomes is presented. It uses illustrative rendering techniques for efficient presentation of connecting fiber bundles in context in anatomical space. Visual hints are employed to improve the perception of spatial relations. Finally, a visualization method with application to exploration and verification of probabilistic tractography is presented, which improves on the previously presented Fiber Stippling technique. It is demonstrated that the method is able to show multiple overlapping tracts in context, and correctly present crossing fiber configurations

Connectivity Influences on Nonlinear Dynamics in Weakly-Synchronized Networks: Insights from Rössler Systems, Electronic Chaotic Oscillators, Model and Biological Neurons

Natural and engineered networks, such as interconnected neurons, ecological and social networks, coupled oscillators, wireless terminals and power loads, are characterized by an appreciable heterogeneity in the local connectivity around each node. For instance, in both elementary structures such as stars and complex graphs having scale-free topology, a minority of elements are linked to the rest of the network disproportionately strongly. While the effect of the arrangement of structural connections on the emergent synchronization pattern has been studied extensively, considerably less is known about its influence on the temporal dynamics unfolding within each node. Here, we present a comprehensive investigation across diverse simulated and experimental systems, encompassing star and complex networks of Rössler systems, coupled hysteresis-based electronic oscillators, microcircuits of leaky integrate-and-fire model neurons, and finally recordings from in-vitro cultures of spontaneously-growing neuronal networks. We systematically consider a range of dynamical measures, including the correlation dimension, nonlinear prediction error, permutation entropy, and other information-theoretical indices. The empirical evidence gathered reveals that under situations of weak synchronization, wherein rather than a collective behavior one observes significantly differentiated dynamics, denser connectivity tends to locally promote the emergence of stronger signatures of nonlinear dynamics. In deterministic systems, transition to chaos and generation of higher-dimensional signals were observed; however, when the coupling is stronger, this relationship may be lost or even inverted. In systems with a strong stochastic component, the generation of more temporally-organized activity could be induced. These observations have many potential implications across diverse fields of basic and applied science, for example, in the design of distributed sensing systems based on wireless coupled oscillators, in network identification and control, as well as in the interpretation of neuroscientific and other dynamical data

On Improving Generalization of CNN-Based Image Classification with Delineation Maps Using the CORF Push-Pull Inhibition Operator

Deployed image classification pipelines are typically dependent on the images captured in real-world environments. This means that images might be affected by different sources of perturbations (e.g. sensor noise in low-light environments). The main challenge arises by the fact that image quality directly impacts the reliability and consistency of classification tasks. This challenge has, hence, attracted wide interest within the computer vision communities. We propose a transformation step that attempts to enhance the generalization ability of CNN models in the presence of unseen noise in the test set. Concretely, the delineation maps of given images are determined using the CORF push-pull inhibition operator. Such an operation transforms an input image into a space that is more robust to noise before being processed by a CNN. We evaluated our approach on the Fashion MNIST data set with an AlexNet model. It turned out that the proposed CORF-augmented pipeline achieved comparable results on noise-free images to those of a conventional AlexNet classification model without CORF delineation maps, but it consistently achieved significantly superior performance on test images perturbed with different levels of Gaussian and uniform noise