Doctor of Philosophy

dissertationIn this dissertation, we advance the theory and practice of verifying visualization algorithms. We present techniques to assess visualization correctness through testing of important mathematical properties. Where applicable, these techniques allow us to distinguish whether anomalies in visualization features can be attributed to the underlying physical process or to artifacts from the implementation under verification. Such scientific scrutiny is at the heart of verifiable visualization - subjecting visualization algorithms to the same verification process that is used in other components of the scientific pipeline. The contributions of this dissertation are manifold. We derive the mathematical framework for the expected behavior of several visualization algorithms, and compare them to experimentally observed results in the selected codes. In the Computational Science & Engineering community CS&E, this technique is know as the Method of Manufactured Solution (MMS). We apply MMS to the verification of geometrical and topological properties of isosurface extraction algorithms, and direct volume rendering. We derive the convergence of geometrical properties of isosurface extraction techniques, such as function value and normals. For the verification of topological properties, we use stratified Morse theory and digital topology to design algorithms that verify topological invariants. In the case of volume rendering algorithms, we provide the expected discretization errors for three different error sources. The results of applying the MMS is another important contribution of this dissertation. We report unexpected behavior for almost all implementations tested. In some cases, we were able to find and fix bugs that prevented the correctness of the visualization algorithm. In particular, we address an almost 2 0 -year-old bug with the core disambiguation procedure of Marching Cubes 33, one of the first algorithms intended to preserve the topology of the trilinear interpolant. Finally, an important by-product of this work is a range of responses practitioners can expect to encounter with the visualization technique under verification

From Fully-Supervised Single-Task to Semi-Supervised Multi-Task Deep Learning Architectures for Segmentation in Medical Imaging Applications

Medical imaging is routinely performed in clinics worldwide for the diagnosis and treatment of numerous medical conditions in children and adults. With the advent of these medical imaging modalities, radiologists can visualize both the structure of the body as well as the tissues within the body. However, analyzing these high-dimensional (2D/3D/4D) images demands a significant amount of time and effort from radiologists. Hence, there is an ever-growing need for medical image computing tools to extract relevant information from the image data to help radiologists perform efficiently. Image analysis based on machine learning has pivotal potential to improve the entire medical imaging pipeline, providing support for clinical decision-making and computer-aided diagnosis. To be effective in addressing challenging image analysis tasks such as classification, detection, registration, and segmentation, specifically for medical imaging applications, deep learning approaches have shown significant improvement in performance. While deep learning has shown its potential in a variety of medical image analysis problems including segmentation, motion estimation, etc., generalizability is still an unsolved problem and many of these successes are achieved at the cost of a large pool of datasets. For most practical applications, getting access to a copious dataset can be very difficult, often impossible. Annotation is tedious and time-consuming. This cost is further amplified when annotation must be done by a clinical expert in medical imaging applications. Additionally, the applications of deep learning in the real-world clinical setting are still limited due to the lack of reliability caused by the limited prediction capabilities of some deep learning models. Moreover, while using a CNN in an automated image analysis pipeline, it’s critical to understand which segmentation results are problematic and require further manual examination. To this extent, the estimation of uncertainty calibration in a semi-supervised setting for medical image segmentation is still rarely reported. This thesis focuses on developing and evaluating optimized machine learning models for a variety of medical imaging applications, ranging from fully-supervised, single-task learning to semi-supervised, multi-task learning that makes efficient use of annotated training data. The contributions of this dissertation are as follows: (1) developing a fully-supervised, single-task transfer learning for the surgical instrument segmentation from laparoscopic images; and (2) utilizing supervised, single-task, transfer learning for segmenting and digitally removing the surgical instruments from endoscopic/laparoscopic videos to allow the visualization of the anatomy being obscured by the tool. The tool removal algorithms use a tool segmentation mask and either instrument-free reference frames or previous instrument-containing frames to fill in (inpaint) the instrument segmentation mask; (3) developing fully-supervised, single-task learning via efficient weight pruning and learned group convolution for accurate left ventricle (LV), right ventricle (RV) blood pool and myocardium localization and segmentation from 4D cine cardiac MR images; (4) demonstrating the use of our fully-supervised memory-efficient model to generate dynamic patient-specific right ventricle (RV) models from cine cardiac MRI dataset via an unsupervised learning-based deformable registration field; and (5) integrating a Monte Carlo dropout into our fully-supervised memory-efficient model with inherent uncertainty estimation, with the overall goal to estimate the uncertainty associated with the obtained segmentation and error, as a means to flag regions that feature less than optimal segmentation results; (6) developing semi-supervised, single-task learning via self-training (through meta pseudo-labeling) in concert with a Teacher network that instructs the Student network by generating pseudo-labels given unlabeled input data; (7) proposing largely-unsupervised, multi-task learning to demonstrate the power of a simple combination of a disentanglement block, variational autoencoder (VAE), generative adversarial network (GAN), and a conditioning layer-based reconstructor for performing two of the foremost critical tasks in medical imaging — segmentation of cardiac structures and reconstruction of the cine cardiac MR images; (8) demonstrating the use of 3D semi-supervised, multi-task learning for jointly learning multiple tasks in a single backbone module – uncertainty estimation, geometric shape generation, and cardiac anatomical structure segmentation of the left atrial cavity from 3D Gadolinium-enhanced magnetic resonance (GE-MR) images. This dissertation summarizes the impact of the contributions of our work in terms of demonstrating the adaptation and use of deep learning architectures featuring different levels of supervision to build a variety of image segmentation tools and techniques that can be used across a wide spectrum of medical image computing applications centered on facilitating and promoting the wide-spread computer-integrated diagnosis and therapy data science

Master of Science

thesisIt is common to extract isosurfaces from simulation eld data to visualize and gain understanding of the underlying physical phenomenon being simulated. As the input parameters of the simulation change, the resulting isosurface varies, and there has been increased interest in quantifying and visualization of these variations as part of the larger interest in uncertainty quantification. In this thesis, we propose an analysis and visualization pipeline for examining the intrinsic variation in isosurfaces caused by simulation parameter perturbation. Drawing inspiration from the shape modeling community, we incorporate the use of heat-kernel signatures (HKS) with a simple nite-difference approach for quantifying the degree to which a region (or even a point) on an isosurface has undergone intrinsic change. Coupled with a clustering technique and the use of color maps, our pipeline allows the user to select the level of fidelity with which they wish to evaluate and visualize the amount of intrinsic change. The pipeline is described with a simple example to walk the reader through the different steps, and experimental validation of parameter choices in the pipeline is provided to justify our design. Then we present canonical and simulation examples to demonstrate the pipeline's use in different applications

Smart FRP Composite Sandwich Bridge Decks in Cold Regions

INE/AUTC 12.0

Application of Uncertainty Modeling Frameworks to Uncertain Isosurface Extraction

Abstract. Proper characterization of uncertainty is a challenging task. Depend-ing on the sources of uncertainty, various uncertainty modeling frameworks have been proposed and studied in the uncertainty quantification literature. This pa-per applies various uncertainty modeling frameworks, namely possibility theory, Dempster-Shafer theory and probability theory to isosurface extraction from un-certain scalar fields. It proposes an uncertainty-based marching cubes template as an abstraction of the conventional marching cubes algorithm with a flexible uncertainty measure. The applicability of the template is demonstrated using 2D simulation data in weather forecasting and computational fluid dynamics and a synthetic 3D dataset

Verifying volume rendering using discretization error analysis

pre-printWe propose an approach for verification of volume rendering correctness based on an analysis of the volume rendering integral, the basis of most DVR algorithms. With respect to the most common discretization of this continuous model (Riemann summation), we make assumptions about the impact of parameter changes on the rendered results and derive convergence curves describing the expected behavior. Specifically, we progressively refine the number of samples along the ray, the grid size, and the pixel size, and evaluate how the errors observed during refinement compare against the expected approximation errors. We derive the theoretical foundations of our verification approach, explain how to realize it in practice, and discuss its limitations. We also report the errors identified by our approach when applied to two publicly available volume rendering packages

Uncertainty visualization in 3D scalar data

One problem in most three-dimensional (3D) scalar data visualization techniques is that they often overlook to depict uncertainty that comes with the 3D scalar data and thus fail to faithfully present the 3D scalar data and have risks which may mislead users’ interpretations, conclusions or even decisions. Therefore this thesis focuses on the study of uncertainty visualization in 3D scalar data and we seek to create better uncertainty visualization techniques, as well as to find out the advantages/disadvantages of those state-of-the-art uncertainty visualization techniques. To do this, we address three specific hypotheses: (1) the proposed Texture uncertainty visualization technique enables users to better identify scalar/error data, and provides reduced visual overload and more appropriate brightness than four state-of-the-art uncertainty visualization techniques, as demonstrated using a perceptual effectiveness user study. (2) The proposed Linked Views and Interactive Specification (LVIS) uncertainty visualization technique enables users to better search max/min scalar and error data than four state-of-the-art uncertainty visualization techniques, as demonstrated using a perceptual effectiveness user study. (3) The proposed Probabilistic Query uncertainty visualization technique, in comparison to traditional Direct Volume Rendering (DVR) methods, enables radiologists/physicians to better identify possible alternative renderings relevant to a diagnosis and the classification probabilities associated to the materials appeared on these renderings; this leads to improved decision support for diagnosis, as demonstrated in the domain of medical imaging. For each hypothesis, we test it by following/implementing a unified framework that consists of three main steps: the first main step is uncertainty data modeling, which clearly defines and generates certainty types of uncertainty associated to given 3D scalar data. The second main step is uncertainty visualization, which transforms the 3D scalar data and their associated uncertainty generated from the first main step into two-dimensional (2D) images for insight, interpretation or communication. The third main step is evaluation, which transforms the 2D images generated from the second main step into quantitative scores according to specific user tasks, and statistically analyzes the scores. As a result, the quality of each uncertainty visualization technique is determined