Catheter insertion simulation with co-registered direct volume rendering and haptic feedback

We have developed an experimental catheter insertion simulation system supporting head-tracked stereoscopic viewing of volumetric anatomic reconstructions registered with direct haptic 3D interaction. The system takes as input data acquired with standard medical imaging modalities and regards it as a visual and haptic environment whose parameters are interactively defined using look-up tables. The system's display, positioned like a surgical table, provide a realistic impression of looking down at the patient. Measuring head motion via a six degrees-of-freedom head tracker, good positions to observe the anatomy and identify the catheter insertion point are quickly established with simple head motion. By generating appropriate stereoscopic images and co-registering physical and virtual spaces beforehand, volumes appear at fixed physical positions and it is possible to control catheter insertion via direct interaction with a PHANToM haptic device. During the insertion procedure, the system provides perception of the effort of penetration and deviation inside the traversed tissues. Semi-transparent volumetric rendering augment the sensory feedback with the visual indication of the inserted catheter position inside the body.96-9

Catheter insertion simulation with combined visual and haptic feedback

We have developed an experimental catheter insertion system supporting head-tracked stereoscopic viewing of volumetric reconstruction registered with direct haptic 3D interaction. The system takes as input patient data acquired with standard medical imaging modalities and regards it as a visual and haptic environment whose parameters are defined using look-up tables. By means of a mirror, the screen seems to be positioned like a surgical table providing the impression of looking down at the patient in a natural way. Co-registering physical and virtual spaces beforehand means that the patient appears at a fixed physical positionj on the surgical table and inside the workspace of the PHANToM device which controls catheter insertion. During the insertion procedure the system provides perception of the force of penetration and positional deviation of the inserted catheter

Multi-touch 3D Exploratory Analysis of Ocean Flow Models

Modern ocean flow simulations are generating increasingly complex, multi-layer 3D ocean flow models. However, most researchers are still using traditional 2D visualizations to visualize these models one slice at a time. Properly designed 3D visualization tools can be highly effective for revealing the complex, dynamic flow patterns and structures present in these models. However, the transition from visualizing ocean flow patterns in 2D to 3D presents many challenges, including occlusion and depth ambiguity. Further complications arise from the interaction methods required to navigate, explore, and interact with these 3D datasets. We present a system that employs a combination of stereoscopic rendering, to best reveal and illustrate 3D structures and patterns, and multi-touch interaction, to allow for natural and efficient navigation and manipulation within the 3D environment. Exploratory visual analysis is facilitated through the use of a highly-interactive toolset which leverages a smart particle system. Multi-touch gestures allow users to quickly position dye emitting tools within the 3D model. Finally, we illustrate the potential applications of our system through examples of real world significance

A framework for realistic 3D tele-immersion

Meeting, socializing and conversing online with a group of people using teleconferencing systems is still quite differ- ent from the experience of meeting face to face. We are abruptly aware that we are online and that the people we are engaging with are not in close proximity. Analogous to how talking on the telephone does not replicate the experi- ence of talking in person. Several causes for these differences have been identified and we propose inspiring and innova- tive solutions to these hurdles in attempt to provide a more realistic, believable and engaging online conversational expe- rience. We present the distributed and scalable framework REVERIE that provides a balanced mix of these solutions. Applications build on top of the REVERIE framework will be able to provide interactive, immersive, photo-realistic ex- periences to a multitude of users that for them will feel much more similar to having face to face meetings than the expe- rience offered by conventional teleconferencing systems

Unwind: Interactive Fish Straightening

The ScanAllFish project is a large-scale effort to scan all the world's 33,100 known species of fishes. It has already generated thousands of volumetric CT scans of fish species which are available on open access platforms such as the Open Science Framework. To achieve a scanning rate required for a project of this magnitude, many specimens are grouped together into a single tube and scanned all at once. The resulting data contain many fish which are often bent and twisted to fit into the scanner. Our system, Unwind, is a novel interactive visualization and processing tool which extracts, unbends, and untwists volumetric images of fish with minimal user interaction. Our approach enables scientists to interactively unwarp these volumes to remove the undesired torque and bending using a piecewise-linear skeleton extracted by averaging isosurfaces of a harmonic function connecting the head and tail of each fish. The result is a volumetric dataset of a individual, straight fish in a canonical pose defined by the marine biologist expert user. We have developed Unwind in collaboration with a team of marine biologists: Our system has been deployed in their labs, and is presently being used for dataset construction, biomechanical analysis, and the generation of figures for scientific publication

Artificial Intelligence in Radiotherapy Treatment Planning: Present and Future.

Treatment planning is an essential step of the radiotherapy workflow. It has become more sophisticated over the past couple of decades with the help of computer science, enabling planners to design highly complex radiotherapy plans to minimize the normal tissue damage while persevering sufficient tumor control. As a result, treatment planning has become more labor intensive, requiring hours or even days of planner effort to optimize an individual patient case in a trial-and-error fashion. More recently, artificial intelligence has been utilized to automate and improve various aspects of medical science. For radiotherapy treatment planning, many algorithms have been developed to better support planners. These algorithms focus on automating the planning process and/or optimizing dosimetric trade-offs, and they have already made great impact on improving treatment planning efficiency and plan quality consistency. In this review, the smart planning tools in current clinical use are summarized in 3 main categories: automated rule implementation and reasoning, modeling of prior knowledge in clinical practice, and multicriteria optimization. Novel artificial intelligence-based treatment planning applications, such as deep learning-based algorithms and emerging research directions, are also reviewed. Finally, the challenges of artificial intelligence-based treatment planning are discussed for future works