Visual Perception and Cognition in Image-Guided Intervention

Surgical image visualization and interaction systems can dramatically affect the efficacy and efficiency of surgical training, planning, and interventions. This is even more profound in the case of minimally-invasive surgery where restricted access to the operative field in conjunction with limited field of view necessitate a visualization medium to provide patient-specific information at any given moment. Unfortunately, little research has been devoted to studying human factors associated with medical image displays and the need for a robust, intuitive visualization and interaction interfaces has remained largely unfulfilled to this day. Failure to engineer efficient medical solutions and design intuitive visualization interfaces is argued to be one of the major barriers to the meaningful transfer of innovative technology to the operating room. This thesis was, therefore, motivated by the need to study various cognitive and perceptual aspects of human factors in surgical image visualization systems, to increase the efficiency and effectiveness of medical interfaces, and ultimately to improve patient outcomes. To this end, we chose four different minimally-invasive interventions in the realm of surgical training, planning, training for planning, and navigation: The first chapter involves the use of stereoendoscopes to reduce morbidity in endoscopic third ventriculostomy. The results of this study suggest that, compared with conventional endoscopes, the detection of the basilar artery on the surface of the third ventricle can be facilitated with the use of stereoendoscopes, increasing the safety of targeting in third ventriculostomy procedures. In the second chapter, a contour enhancement technique is described to improve preoperative planning of arteriovenous malformation interventions. The proposed method, particularly when combined with stereopsis, is shown to increase the speed and accuracy of understanding the spatial relationship between vascular structures. In the third chapter, an augmented-reality system is proposed to facilitate the training of planning brain tumour resection. The results of our user study indicate that the proposed system improves subjects\u27 performance, particularly novices\u27, in formulating the optimal point of entry and surgical path independent of the sensorimotor tasks performed. In the last chapter, the role of fully-immersive simulation environments on the surgeons\u27 non-technical skills to perform vertebroplasty procedure is investigated. Our results suggest that while training surgeons may increase their technical skills, the introduction of crisis scenarios significantly disturbs the performance, emphasizing the need of realistic simulation environments as part of training curriculum

Immersive analytics with abstract 3D visualizations: A survey

After a long period of scepticism, more and more publications describe basic research but also practical approaches to how abstract data can be presented in immersive environments for effective and efficient data understanding. Central aspects of this important research question in immersive analytics research are concerned with the use of 3D for visualization, the embedding in the immersive space, the combination with spatial data, suitable interaction paradigms and the evaluation of use cases. We provide a characterization that facilitates the comparison and categorization of published works and present a survey of publications that gives an overview of the state of the art, current trends, and gaps and challenges in current research

Is Embodied Interaction Beneficial? A Study on Navigating Network Visualizations

Network visualizations are commonly used to analyze relationships in various contexts. To efficiently explore a network visualization, the user needs to quickly navigate to different parts of the network and analyze local details. Recent advancements in display and interaction technologies inspire new visions for improved visualization and interaction design. Past research into network design has identified some key benefits to visualizing networks in 3D versus 2D. However, little work has been done to study the impact of varying levels of embodied interaction on network analysis. We present a controlled user study that compared four environments featuring conditions and hardware that leveraged different amounts of embodiment and visual perception ranging from a 2D visualization desktop environment with a standard mouse to a 3D visualization virtual reality environment. We measured the accuracy, speed, perceived workload, and preferences of 20 participants as they completed three network analytic tasks, each of which required unique navigation and substantial effort. For the task that required participants to iterate over the entire visualization rather than focus on a specific area, we found that participants were more accurate using a VR and a trackball mouse than conventional desktop settings. From a workload perspective, VR was generally considered the least mentally demanding and least frustrating in two of our three tasks. It was also preferred and ranked as the most effective and visually appealing condition overall. However, using VR to compare two side-by-side networks was difficult, and it was similar to or slower than other conditions in two of the three tasks. Overall, the accuracy and workload advantages of conditions with greater embodiment in specific tasks suggest promising opportunities to create more effective environments in which to analyze network visualizations.Comment: Accepted by the Information Visualization journa

3D Cadastres Best Practices, Chapter 5: Visualization and New Opportunities

This paper proposes a discussion on opportunities offered by 3D visualization to improve the understanding and the analysis of cadastre data. It first introduce the rationale of having 3D visualization functionalities in the context of cadastre applications. Second the publication outline some basic concepts in 3D visualization. This section specially addresses the visualization pipeline as a driven classification schema to understand the steps leading to 3D visualization. In this section is also presented a brief review of current 3D standards and technologies. Next is proposed a summary of progress made in the last years in 3D cadastral visualization. For instance, user’s requirement, data and semiotics, and platforms are highlighted as main actions performed in the development of 3D cadastre visualization. This review could be perceived as an attempt to structure and emphasise the best practices in the domain of 3D cadastre visualization and as an inventory of issues that still need to be tackled. Finally, by providing a review on advances and trends in 3D visualization, the paper initiates a discussion and a critical analysis on the benefit of applying these new developments to cadastre domain. This final section discusses about enhancing 3D techniques as dynamic transparency and cutaway, 3D generalization, 3D visibility model, 3D annotation, 3D data and web platform, augmented reality, immersive virtual environment, 3D gaming, interaction techniques and time

Modeling Three-Dimensional Interaction Tasks for Desktop Virtual Reality

A virtual environment is an interactive, head-referenced computer display that gives a user the illusion of presence in real or imaginary worlds. Two most significant differences between a virtual environment and a more traditional interactive 3D computer graphics system are the extent of the user's sense of presence and the level of user participation that can be obtained in the virtual environment. Over the years, advances in computer display hardware and software have substantially progressed the realism of computer-generated images, which dramatically enhanced user’s sense of presence in virtual environments. Unfortunately, such progress of user’s interaction with a virtual environment has not been observed. The scope of the thesis lies in the study of human-computer interaction that occurs in a desktop virtual environment. The objective is to develop/verify 3D interaction models that can be used to quantitatively describe users’ performance for 3D pointing, steering and object pursuit tasks and through the analysis of the interaction models and experimental results to gain a better understanding of users’ movements in the virtual environment. The approach applied throughout the thesis is a modeling methodology that is composed of three procedures, including identifying the variables involved for modeling a 3D interaction task, formulating and verifying the interaction model through user studies and statistical analysis, and applying the model to the evaluation of interaction techniques and input devices and gaining an insight into users’ movements in the virtual environment. In the study of 3D pointing tasks, a two-component model is used to break the tasks into a ballistic phase and a correction phase, and comparison is made between the real-world and virtual-world tasks in each phase. The results indicate that temporal differences arise in both phases, but the difference is significantly greater in the correction phase. This finding inspires us to design a methodology with two-component model and Fitts’ law, which decomposes a pointing task into the ballistic and correction phase and decreases the index of the difficulty of the task during the correction phase. The methodology allows for the development and evaluation of interaction techniques for 3D pointing tasks. For 3D steering tasks, the steering law, which was proposed to model 2D steering tasks, is adapted to 3D tasks by introducing three additional variables, i.e., path curvature, orientation and haptic feedback. The new model suggests that a 3D ball-and-tunnel steering movement consists of a series of small and jerky sub-movements that are similar to the ballistic/correction movements observed in the pointing movements. An interaction model is originally proposed and empirically verified for 3D object pursuit tasks, making use of Stevens’ power law. The results indicate that the power law can be used to model all three common interaction tasks, which may serve as a general law for modeling interaction tasks, and also provides a way to quantitatively compare the tasks

Diverse approaches to learning with immersive Virtual Reality identified from a systematic review

To investigate how learning in immersive Virtual Reality was designed in contemporary educational studies, this systematic literature review identified nine design features and analysed 219 empirical studies on the designs of learning activities with immersive Virtual Reality. Overall, the technological features for physical presence were more readily implemented and investigated than pedagogical features for learning engagement. Further analysis with k-means clustering revealed five approaches with varying levels of interactivity and openness in learning tasks, from watching virtual worlds passively to responding to personalised prompts. Such differences in the design appeared to stem from different practical and educational priorities, such as accessibility, interactivity, and engagement. This review highlights the diversity in the learning task designs in immersive Virtual Reality and illustrates how researchers are navigating practical and educational concerns. We recommend future empirical studies recognise the different approaches and priorities when designing and evaluating learning with immersive Virtual Reality. We also recommend that future systematic reviews investigate immersive Virtual Reality-based learning not only by learning topics or learner demographics, but also by task designs and learning experiences

Diverse approaches to learning with immersive Virtual Reality identified from a systematic review

To investigate how learning in immersive Virtual Reality was designed in contemporary educational studies, this systematic literature review identified nine design features and analysed 219 empirical studies on the designs of learning activities with immersive Virtual Reality. Overall, the technological features for physical presence were more readily implemented and investigated than pedagogical features for learning engagement. Further analysis with k-means clustering revealed five approaches with varying levels of interactivity and openness in learning tasks, from watching virtual worlds passively to responding to personalised prompts. Such differences in the design appeared to stem from different practical and educational priorities, such as accessibility, interactivity, and engagement. This review highlights the diversity in the learning task designs in immersive Virtual Reality and illustrates how researchers are navigating practical and educational concerns. We recommend future empirical studies recognise the different approaches and priorities when designing and evaluating learning with immersive Virtual Reality. We also recommend that future systematic reviews investigate immersive Virtual Reality-based learning not only by learning topics or learner demographics, but also by task designs and learning experiences

Sonic interactions in virtual environments

This book tackles the design of 3D spatial interactions in an audio-centered and audio-first perspective, providing the fundamental notions related to the creation and evaluation of immersive sonic experiences. The key elements that enhance the sensation of place in a virtual environment (VE) are: Immersive audio: the computational aspects of the acoustical-space properties of Virutal Reality (VR) technologies Sonic interaction: the human-computer interplay through auditory feedback in VE VR systems: naturally support multimodal integration, impacting different application domains Sonic Interactions in Virtual Environments will feature state-of-the-art research on real-time auralization, sonic interaction design in VR, quality of the experience in multimodal scenarios, and applications. Contributors and editors include interdisciplinary experts from the fields of computer science, engineering, acoustics, psychology, design, humanities, and beyond. Their mission is to shape an emerging new field of study at the intersection of sonic interaction design and immersive media, embracing an archipelago of existing research spread in different audio communities and to increase among the VR communities, researchers, and practitioners, the awareness of the importance of sonic elements when designing immersive environments