Preparing for the future of cardiothoracic surgery with virtual reality simulation and surgical planning:a narrative review

Background and Objective: Virtual reality (VR) technology in cardiothoracic surgery has been an area of interest for almost three decades, but computational limitations had restricted its implementation. Recent advances in computing power have facilitated the creation of high-fidelity VR simulations and anatomy visualisation tools. We undertook a non-systematic narrative review of literature on VR simulations and preoperative planning tools in cardiothoracic surgery and present the state-of-the-art, and a future outlook. Methods: A comprehensive search through MEDLINE database was performed in November 2022 for all publications that describe the use of VR in cardiothoracic surgery regarding training purposes, education, simulation, and procedural planning. We excluded papers that were not in English or Dutch, and that used two-dimensional (2D) screens, augmented, and simulated reality. Key Content and Findings: Results were categorised as simulators and preoperative planning tools. Current surgical simulators include the lobectomy module in the LapSim for video assisted thorascopic surgery which has been extensively validated, and the more recent robotic assisted lobectomy simulators from Robotix Mentor and Da Vinci SimNow, which are increasingly becoming integrated into the robotic surgery curriculum. Other perioperative simulators include the CardioPulmonary VR Resuscitation simulator for advanced life support after cardiac surgery, and the VR Extracorporeal Circulation (ECC) simulator for perfusionists to simulate the use of a heart-lung machine (HLM). For surgical planning, there are many small-scale tools available, and many case/pilot studies have been published utilising the visualisation possibilities provided by VR, including congenital cardiac, congenital thoracic, adult cardiac, and adult thoracic diseases. Conclusions: There are many promising tools becoming available to leverage the immersive power of VR in cardiothoracic surgery. The path to validate these simulators is well described, but large-scale trials producing high-level evidence for their efficacy are absent as of yet. Our view is that these tools will become increasingly integral parts of daily practice in this field in the coming decade.</p

Augmented reality meeting table: a novel multi-user interface for architectural design

Immersive virtual environments have received widespread attention as providing possible replacements for the media and systems that designers traditionally use, as well as, more generally, in providing support for collaborative work. Relatively little attention has been given to date however to the problem of how to merge immersive virtual environments into real world work settings, and so to add to the media at the disposal of the designer and the design team, rather than to replace it. In this paper we report on a research project in which optical see-through augmented reality displays have been developed together with prototype decision support software for architectural and urban design. We suggest that a critical characteristic of multi user augmented reality is its ability to generate visualisations from a first person perspective in which the scale of rendition of the design model follows many of the conventions that designers are used to. Different scales of model appear to allow designers to focus on different aspects of the design under consideration. Augmenting the scene with simulations of pedestrian movement appears to assist both in scale recognition, and in moving from a first person to a third person understanding of the design. This research project is funded by the European Commission IST program (IST-2000-28559)

Application of virtual and mixed reality for 3D visualization in intracranial aneurysm surgery planning: a systematic review

BACKGROUND Precise preoperative anatomical visualization and understanding of an intracranial aneurysm (IA) are fundamental for surgical planning and increased intraoperative confidence. Application of virtual reality (VR) and mixed reality (MR), thus three-dimensional (3D) visualization of IAs could be significant in surgical planning. Authors provide an up-to-date overview of VR and MR applied to IA surgery, with specific focus on tailoring of the surgical treatment. METHODS A systematic analysis of the literature was performed in accordance with the PRISMA guidelines. Pubmed, and Embase were searched to identify studies reporting use of MR and VR 3D visualization in IA surgery during the last 25 years. Type and number of IAs, category of input scan, visualization techniques (screen, glasses or head set), inclusion of haptic feedback, tested population (residents, fellows, attending neurosurgeons), and aim of the study (surgical planning/rehearsal, neurosurgical training, methodological validation) were noted. RESULTS Twenty-eight studies were included. Eighteen studies (64.3%) applied VR, and 10 (35.7%) used MR. A positive impact on surgical planning was documented by 19 studies (67.9%): 17 studies (60.7%) chose the tailoring of the surgical approach as primary outcome of the analysis. A more precise anatomical visualization and understanding with VR and MR was endorsed by all included studies (100%). CONCLUSION Application of VR and MR to perioperative 3D visualization of IAs allowed an improved understanding of the patient-specific anatomy and surgical preparation. This review describes a tendency to utilize mostly VR-platforms, with the primary goals of a more accurate anatomical understanding, surgical planning and rehearsal

Use of extended realities in cardiology

Recent miniaturization of electronic components and advances in image processing software have facilitated the entry of extended reality technology into clinical practice. In the last several years, the number of applications in cardiology has multiplied, with many promising to become standard of care. We review many of these applications in the areas of patient and physician education, cardiac rehabilitation, pre-procedural planning and intraprocedural use. The rapid integration of these approaches into the many facets of cardiology suggests that they will one day become an every-day part of physician practice

INTERFACE DESIGN FOR A VIRTUAL REALITY-ENHANCED IMAGE-GUIDED SURGERY PLATFORM USING SURGEON-CONTROLLED VIEWING TECHNIQUES

Initiative has been taken to develop a VR-guided cardiac interface that will display and deliver information without affecting the surgeons’ natural workflow while yielding better accuracy and task completion time than the existing setup. This paper discusses the design process, the development of comparable user interface prototypes as well as an evaluation methodology that can measure user performance and workload for each of the suggested display concepts. User-based studies and expert recommendations are used in conjunction to es­ tablish design guidelines for our VR-guided surgical platform. As a result, a better understanding of autonomous view control, depth display, and use of virtual context, is attained. In addition, three proposed interfaces have been developed to allow a surgeon to control the view of the virtual environment intra-operatively. Comparative evaluation of the three implemented interface prototypes in a simulated surgical task scenario, revealed performance advantages for stereoscopic and monoscopic biplanar display conditions, as well as the differences between three types of control modalities. One particular interface prototype demonstrated significant improvement in task performance. Design recommendations are made for this interface as well as the others as we prepare for prospective development iterations

Visual Perception and Cognition in Image-Guided Intervention

Surgical image visualization and interaction systems can dramatically affect the efficacy and efficiency of surgical training, planning, and interventions. This is even more profound in the case of minimally-invasive surgery where restricted access to the operative field in conjunction with limited field of view necessitate a visualization medium to provide patient-specific information at any given moment. Unfortunately, little research has been devoted to studying human factors associated with medical image displays and the need for a robust, intuitive visualization and interaction interfaces has remained largely unfulfilled to this day. Failure to engineer efficient medical solutions and design intuitive visualization interfaces is argued to be one of the major barriers to the meaningful transfer of innovative technology to the operating room. This thesis was, therefore, motivated by the need to study various cognitive and perceptual aspects of human factors in surgical image visualization systems, to increase the efficiency and effectiveness of medical interfaces, and ultimately to improve patient outcomes. To this end, we chose four different minimally-invasive interventions in the realm of surgical training, planning, training for planning, and navigation: The first chapter involves the use of stereoendoscopes to reduce morbidity in endoscopic third ventriculostomy. The results of this study suggest that, compared with conventional endoscopes, the detection of the basilar artery on the surface of the third ventricle can be facilitated with the use of stereoendoscopes, increasing the safety of targeting in third ventriculostomy procedures. In the second chapter, a contour enhancement technique is described to improve preoperative planning of arteriovenous malformation interventions. The proposed method, particularly when combined with stereopsis, is shown to increase the speed and accuracy of understanding the spatial relationship between vascular structures. In the third chapter, an augmented-reality system is proposed to facilitate the training of planning brain tumour resection. The results of our user study indicate that the proposed system improves subjects\u27 performance, particularly novices\u27, in formulating the optimal point of entry and surgical path independent of the sensorimotor tasks performed. In the last chapter, the role of fully-immersive simulation environments on the surgeons\u27 non-technical skills to perform vertebroplasty procedure is investigated. Our results suggest that while training surgeons may increase their technical skills, the introduction of crisis scenarios significantly disturbs the performance, emphasizing the need of realistic simulation environments as part of training curriculum

Implementation and Assessment of a Virtual Reality Experiment in the Undergraduate Themo-Fluids Laboratory

Results are presented from an NSF supported project that is geared towards advancing the development and use of virtual reality (VR) laboratories, designed to emulate the learning environment of physical laboratories. As part of this project, an experiment in the undergraduate thermo-fluids laboratory course titled Jet Impact Force was transformed into a 3-D virtual reality experiment using the widely used MAYA R and VIRTOOLS R software. In order to facilitate students\u27 interactions with the newly created 3-D interactive, immersive and stereoscopic virtual laboratory environment, the human computer interfaces (HCI) were programmed and incorporated in the simulation software. Two immersion levels were included in the VR experiment to assess their impact on student learning. The first one namely the desktop virtual reality (DTVR) used a computer and a 3-D TV for display while the CAVE virtual reality (CVR) employed a computer in conjunction with a three-wall CAVE (acronym for Cave Automatic Virtual Environment) for visualizing the simulation. The above said VR experiment was embedded in the thermo-fluids laboratory course in the mechanical engineering curriculum at Old Dominion University (ODU) so that it could be used in the supplementation mode for the pre-lab practice sessions prior to the physical experiment sessions. To test the efficacy of this supplementation pedagogy for enhancement of student learning, both quantitative (quiz) as well as qualitative (direct observation and student survey) assessment instruments were used. Of the three objectives set for this study two, namely the development and implementation of VR experiment and the assessment of impact of immersion levels on student learning were fully achieved. Assessment results also showed that the CVR module resulted in a higher level of student learning when compared with the DTVR module. The third objective, namely the assessment of the VR experiment in enhancing student learning in the supplementation mode was met only partially since the quantitative and qualitative assessments produced divergent results. The statistical analysis of the quiz scores of the experimental group, consisting of students who used the VR experiment for supplementation, and the control group (without supplementation) showed that the supplementation produced improvements in student learning that were statistically insignificant. In contrast the direct observation of both the experimental and the control groups during the physical experiment pointed to student learning gains for the experimental group. Student surveys showed generally positive disposition of students towards the newly introduced VR experiment