Immersive Visualization for Enhanced Computational Fluid Dynamics Analysis

Modern biomedical computer simulations produce spatiotemporal results that are often viewed at a single point in time on standard 2D displays. An immersive visualization environment (IVE) with 3D stereoscopic capability can mitigate some shortcomings of 2D displays via improved depth cues and active movement to further appreciate the spatial localization of imaging data with temporal computational fluid dynamics (CFD) results. We present a semi-automatic workflow for the import, processing, rendering, and stereoscopic visualization of high resolution, patient-specific imaging data, and CFD results in an IVE. Versatility of the workflow is highlighted with current clinical sequelae known to be influenced by adverse hemodynamics to illustrate potential clinical utility

Advanced Visualization and Intuitive User Interface Systems for Biomedical Applications

Modern scientific research produces data at rates that far outpace our ability to comprehend and analyze it. Such sources include medical imaging data and computer simulations, where technological advancements and spatiotemporal resolution generate increasing amounts of data from each scan or simulation. A bottleneck has developed whereby medical professionals and researchers are unable to fully use the advanced information available to them. By integrating computer science, computer graphics, artistic ability and medical expertise, scientific visualization of medical data has become a new field of study. The objective of this thesis is to develop two visualization systems that use advanced visualization, natural user interface technologies and the large amount of biomedical data available to produce results that are of clinical utility and overcome the data bottleneck that has developed. Computational Fluid Dynamics (CFD) is a tool used to study the quantities associated with the movement of blood by computer simulation. We developed methods of processing spatiotemporal CFD data and displaying it in stereoscopic 3D with the ability to spatially navigate through the data. We used this method with two sets of display hardware: a full-scale visualization environment and a small-scale desktop system. The advanced display and data navigation abilities provide the user with the means to better understand the relationship between the vessel\u27s form and function. Low-cost 3D, depth-sensing cameras capture and process user body motion to recognize motions and gestures. Such devices allow users to use hand motions as an intuitive interface to computer applications. We developed algorithms to process and prepare the biomedical and scientific data for use with a custom control application. The application interprets user gestures as commands to a visualization tool and allows the user to control the visualization of multi-dimensional data. The intuitive interface allows the user to control the visualization of data without manual contact with an interaction device. In developing these methods and software tools we have leveraged recent trends in advanced visualization and intuitive interfaces in order to efficiently visualize biomedical data in such a way that provides meaningful information that can be used to further appreciate it

Virtual Reality applied to biomedical engineering

Actualment, la realitat virtual esta sent tendència i s'està expandint a l'àmbit mèdic, fent possible l'aparició de nombroses aplicacions dissenyades per entrenar metges i tractar pacients de forma més eficient, així com optimitzar els processos de planificació quirúrgica. La necessitat mèdica i objectiu d'aquest projecte és fer òptim el procés de planificació quirúrgica per a cardiopaties congènites, que compren la reconstrucció en 3D del cor del pacient i la seva integració en una aplicació de realitat virtual. Seguint aquesta línia s’ha combinat un procés de modelat 3D d’imatges de cors obtinguts gracies al Hospital Sant Joan de Déu i el disseny de l’aplicació mitjançant el software Unity 3D gracies a l’empresa VISYON. S'han aconseguit millores en quant al software emprat per a la segmentació i reconstrucció, i s’han assolit funcionalitats bàsiques a l’aplicació com importar, moure, rotar i fer captures de pantalla en 3D de l'òrgan cardíac i així, entendre millor la cardiopatia que s’ha de tractar. El resultat ha estat la creació d'un procés òptim, en el que la reconstrucció en 3D ha aconseguit ser ràpida i precisa, el mètode d’importació a l’app dissenyada molt senzill, i una aplicació que permet una interacció atractiva i intuïtiva, gracies a una experiència immersiva i realista per ajustar-se als requeriments d'eficiència i precisió exigits en el camp mèdic

Immersive Visualization in Biomedical Computational Fluid Dynamics and Didactic Teaching and Learning

Virtual reality (VR) can stimulate active learning, critical thinking, decision making and improved performance. It requires a medium to show virtual content, which is called a virtual environment (VE). The MARquette Visualization Lab (MARVL) is an example of a VE. Robust processes and workflows that allow for the creation of content for use within MARVL further increases the userbase for this valuable resource. A workflow was created to display biomedical computational fluid dynamics (CFD) and complementary data in a wide range of VE’s. This allows a researcher to study the simulation in its natural three-dimensional (3D) morphology. In addition, it is an exciting way to extract more information from CFD results by taking advantage of improved depth cues, a larger display canvas, custom interactivity, and an immersive approach that surrounds the researcher. The CFD to VR workflow was designed to be basic enough for a novice user. It is also used as a tool to foster collaboration between engineers and clinicians. The workflow aimed to support results from common CFD software packages and across clinical research areas. ParaView, Blender and Unity were used in the workflow to take standard CFD files and process them for viewing in VR. Designated scripts were written to automate the steps implemented in each software package. The workflow was successfully completed across multiple biomedical vessels, scales and applications including: the aorta with application to congenital cardiovascular disease, the Circle of Willis with respect to cerebral aneurysms, and the airway for surgical treatment planning. The workflow was completed by novice users in approximately an hour. Bringing VR further into didactic teaching within academia allows students to be fully immersed in their respective subject matter, thereby increasing the students’ sense of presence, understanding and enthusiasm. MARVL is a space for collaborative learning that also offers an immersive, virtual experience. A workflow was created to view PowerPoint presentations in 3D using MARVL. A resulting Immersive PowerPoint workflow used PowerPoint, Unity and other open-source software packages to display the PowerPoint presentations in 3D. The Immersive PowerPoint workflow can be completed in under thirty minutes

Contributions to the Development of Objective Techniques for Presence Measurement in Virtual Environments by means of Brain Activity Analysis

En esta tesis, se propone el uso de la técnica de Doppler transcraneal (DTC) para monitorizar la actividad cerebral durante la exposición a entornos virtuales (EV) y así poder analizar los correlatos cerebrales del sentido de presencia. Las hipótesis de partida son las siguientes: 1) DTC se podrá utilizar fácilmente en combinación con sistemas de realidad virtual. 2) Los datos de velocidad de flujo sanguíneo medidos por DTC se podrán utilizar para analizar cambios de actividad cerebral durante la exposición a EV. 3) Habrá diferencias en la velocidad del flujo sanguíneo asociadas a distintos niveles de presencia. 4) Habrá correlación entre el grado de presencia medido por cuestionarios y parámetros de la velocidad de flujo sanguíneo. 5) Cada componente de la experiencia virtual tendrá una influencia en las variaciones de velocidad observadas. Para analizar las hipótesis planteadas, se realizaron cuatro experimentos distintos, en los que se analizó la velocidad del flujo sanguíneo durante: 1) distintas condiciones de navegación, 2) distintas condiciones de inmersión, 3) una tarea de percepción visual y 4) tareas motoras para manejo de un joystick. Durante la tesis, se han propuesto distintas técnicas de procesado de señal basadas en análisis espectral y en la obtención parámetros no lineales de la señal, que no habían sido utilizadas previamente en experimentos psicofisiológicos con DTC. Se ha observado que existe un incremento en la velocidad del flujo sanguíneo durante la exposición a un EV, el cual puede deberse a distintos factores que intervienen en la experiencia: tareas de interacción visuoespacial, tareas de atención, la creación y ejecución de un plan motor, cambios emocionales Los análisis han mostrado que existen correlaciones significativas entre la velocidad media de flujo sanguíneo en las arterias cerebrales medias durante la exposición al EV y respuestas a los cuestionarios de presencia utilizados.Rey Solaz, B. (2010). Contributions to the Development of Objective Techniques for Presence Measurement in Virtual Environments by means of Brain Activity Analysis [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/8505Palanci

Visual Perception and Cognition in Image-Guided Intervention

Surgical image visualization and interaction systems can dramatically affect the efficacy and efficiency of surgical training, planning, and interventions. This is even more profound in the case of minimally-invasive surgery where restricted access to the operative field in conjunction with limited field of view necessitate a visualization medium to provide patient-specific information at any given moment. Unfortunately, little research has been devoted to studying human factors associated with medical image displays and the need for a robust, intuitive visualization and interaction interfaces has remained largely unfulfilled to this day. Failure to engineer efficient medical solutions and design intuitive visualization interfaces is argued to be one of the major barriers to the meaningful transfer of innovative technology to the operating room. This thesis was, therefore, motivated by the need to study various cognitive and perceptual aspects of human factors in surgical image visualization systems, to increase the efficiency and effectiveness of medical interfaces, and ultimately to improve patient outcomes. To this end, we chose four different minimally-invasive interventions in the realm of surgical training, planning, training for planning, and navigation: The first chapter involves the use of stereoendoscopes to reduce morbidity in endoscopic third ventriculostomy. The results of this study suggest that, compared with conventional endoscopes, the detection of the basilar artery on the surface of the third ventricle can be facilitated with the use of stereoendoscopes, increasing the safety of targeting in third ventriculostomy procedures. In the second chapter, a contour enhancement technique is described to improve preoperative planning of arteriovenous malformation interventions. The proposed method, particularly when combined with stereopsis, is shown to increase the speed and accuracy of understanding the spatial relationship between vascular structures. In the third chapter, an augmented-reality system is proposed to facilitate the training of planning brain tumour resection. The results of our user study indicate that the proposed system improves subjects\u27 performance, particularly novices\u27, in formulating the optimal point of entry and surgical path independent of the sensorimotor tasks performed. In the last chapter, the role of fully-immersive simulation environments on the surgeons\u27 non-technical skills to perform vertebroplasty procedure is investigated. Our results suggest that while training surgeons may increase their technical skills, the introduction of crisis scenarios significantly disturbs the performance, emphasizing the need of realistic simulation environments as part of training curriculum

Realtime design and analysis of 3D structures using Finite Element Analysis within Virtual Reality environments

Structural analysis is a 3-dimensional concept that has traditionally been taught with 2-dimensional mediums, such as whiteboards and computer screens. This often leads to a cognitive disconnect as students are forced to rely on their individual imaginations to form complete visualizations of the topic. Understanding how structures are affected by different loading conditions is not a trivial skill for students learning the concepts. While laboratory classes can provide some real world perspective into how structures deform, they require expensive testing equipment setups and training which take away from their experiential learning capabilities. Virtual reality is an emerging technology that can be leveraged to conduct structural analysis while intuitively teaching how it is done. Using the fundamental equations and geometric principles behind finite element analysis, a virtual environment can be created where students can experiment with creating and editing their own real-time deformable structures. To provide a true classroom experience, this simulation could run as both a local and multi-user shared experience. This will require leveraging two different virtual reality software development platforms, such as Unity and A-Frame, to create the simulation and output it to a user wearing a virtual reality headset. Tools will need to be scripted for these experiences to allow a user to define a structure, either in pre-processing or in real time, within the experience. The user will then have the capability to modify the structure and its material properties. This will require seamless integration between the user's interactions and the finite element analysis solver to update the results with minimal latency. The integrity of the finite element analysis results from within the simulation will be numerically validated against a trusted commercial software to confirm accuracy. After analysis, the simulation will need to visualize the long slender members of the deformed structure. The current implementation will accurately render truss elements, but will need further improvements to visualize frame elements correctly. Multiple use case scenarios will be defined as an extension of this work to directly benefit students in the classroom as well as professionals who are conducting structural analysis

Simulating Developmental Cardiac Morphology in Virtual Reality Using a Deformable Image Registration Approach

While virtual reality (VR) has potential in enhancing cardiovascular diagnosis and treatment, prerequisite labor-intensive image segmentation remains an obstacle for seamlessly simulating 4-dimensional (4-D, 3-D + time) imaging data in an immersive, physiological VR environment. We applied deformable image registration (DIR) in conjunction with 3-D reconstruction and VR implementation to recapitulate developmental cardiac contractile function from light-sheet fluorescence microscopy (LSFM). This method addressed inconsistencies that would arise from independent segmentations of time-dependent data, thereby enabling the creation of a VR environment that fluently simulates cardiac morphological changes. By analyzing myocardial deformation at high spatiotemporal resolution, we interfaced quantitative computations with 4-D VR. We demonstrated that our LSFM-captured images, followed by DIR, yielded average dice similarity coefficients of 0.92 ± 0.05 (n = 510) and 0.93 ± 0.06 (n = 240) when compared to ground truth images obtained from Otsu thresholding and manual segmentation, respectively. The resulting VR environment simulates a wide-angle zoomed-in view of motion in live embryonic zebrafish hearts, in which the cardiac chambers are undergoing structural deformation throughout the cardiac cycle. Thus, this technique allows for an interactive micro-scale VR visualization of developmental cardiac morphology to enable high resolution simulation for both basic and clinical science