Performance Factors in Neurosurgical Simulation and Augmented Reality Image Guidance

Virtual reality surgical simulators have seen widespread adoption in an effort to provide safe, cost-effective and realistic practice of surgical skills. However, the majority of these simulators focus on training low-level technical skills, providing only prototypical surgical cases. For many complex procedures, this approach is deficient in representing anatomical variations that present clinically, failing to challenge users’ higher-level cognitive skills important for navigation and targeting. Surgical simulators offer the means to not only simulate any case conceivable, but to test novel approaches and examine factors that influence performance. Unfortunately, there is a void in the literature surrounding these questions. This thesis was motivated by the need to expand the role of surgical simulators to provide users with clinically relevant scenarios and evaluate human performance in relation to image guidance technologies, patient-specific anatomy, and cognitive abilities. To this end, various tools and methodologies were developed to examine cognitive abilities and knowledge, simulate procedures, and guide complex interventions all within a neurosurgical context. The first chapter provides an introduction to the material. The second chapter describes the development and evaluation of a virtual anatomical training and examination tool. The results suggest that learning occurs and that spatial reasoning ability is an important performance predictor, but subordinate to anatomical knowledge. The third chapter outlines development of automation tools to enable efficient simulation studies and data management. In the fourth chapter, subjects perform abstract targeting tasks on ellipsoid targets with and without augmented reality guidance. While the guidance tool improved accuracy, performance with the tool was strongly tied to target depth estimation – an important consideration for implementation and training with similar guidance tools. In the fifth chapter, neurosurgically experienced subjects were recruited to perform simulated ventriculostomies. Results showed anatomical variations influence performance and could impact outcome. Augmented reality guidance showed no marked improvement in performance, but exhibited a mild learning curve, indicating that additional training may be warranted. The final chapter summarizes the work presented. Our results and novel evaluative methodologies lay the groundwork for further investigation into simulators as versatile research tools to explore performance factors in simulated surgical procedures

Modelling and simulation of flexible instruments for minimally invasive surgical training in virtual reality

Improvements in quality and safety standards in surgical training, reduction in training hours and constant technological advances have challenged the traditional apprenticeship model to create a competent surgeon in a patient-safe way. As a result, pressure on training outside the operating room has increased. Interactive, computer based Virtual Reality (VR) simulators offer a safe, cost-effective, controllable and configurable training environment free from ethical and patient safety issues. Two prototype, yet fully-functional VR simulator systems for minimally invasive procedures relying on flexible instruments were developed and validated. NOViSE is the first force-feedback enabled VR simulator for Natural Orifice Transluminal Endoscopic Surgery (NOTES) training supporting a flexible endoscope. VCSim3 is a VR simulator for cardiovascular interventions using catheters and guidewires. The underlying mathematical model of flexible instruments in both simulator prototypes is based on an established theoretical framework – the Cosserat Theory of Elastic Rods. The efficient implementation of the Cosserat Rod model allows for an accurate, real-time simulation of instruments at haptic-interactive rates on an off-the-shelf computer. The behaviour of the virtual tools and its computational performance was evaluated using quantitative and qualitative measures. The instruments exhibited near sub-millimetre accuracy compared to their real counterparts. The proposed GPU implementation further accelerated their simulation performance by approximately an order of magnitude. The realism of the simulators was assessed by face, content and, in the case of NOViSE, construct validity studies. The results indicate good overall face and content validity of both simulators and of virtual instruments. NOViSE also demonstrated early signs of construct validity. VR simulation of flexible instruments in NOViSE and VCSim3 can contribute to surgical training and improve the educational experience without putting patients at risk, raising ethical issues or requiring expensive animal or cadaver facilities. Moreover, in the context of an innovative and experimental technique such as NOTES, NOViSE could potentially facilitate its development and contribute to its popularization by keeping practitioners up to date with this new minimally invasive technique.Open Acces

Patient Specific Systems for Computer Assisted Robotic Surgery Simulation, Planning, and Navigation

The evolving scenario of surgery: starting from modern surgery, to the birth of medical imaging and the introduction of minimally invasive techniques, has seen in these last years the advent of surgical robotics. These systems, making possible to get through the difficulties of endoscopic surgery, allow an improved surgical performance and a better quality of the intervention. Information technology contributed to this evolution since the beginning of the digital revolution: providing innovative medical imaging devices and computer assisted surgical systems. Afterwards, the progresses in computer graphics brought innovative visualization modalities for medical datasets, and later the birth virtual reality has paved the way for virtual surgery. Although many surgical simulators already exist, there are no patient specific solutions. This thesis presents the development of patient specific software systems for preoperative planning, simulation and intraoperative assistance, designed for robotic surgery: in particular for bimanual robots that are becoming the future of single port interventions. The first software application is a virtual reality simulator for this kind of surgical robots. The system has been designed to validate the initial port placement and the operative workspace for the potential application of this surgical device. Given a bimanual robot with its own geometry and kinematics, and a patient specific 3D virtual anatomy, the surgical simulator allows the surgeon to choose the optimal positioning of the robot and the access port in the abdominal wall. Additionally, it makes possible to evaluate in a virtual environment if a dexterous movability of the robot is achievable, avoiding unwanted collisions with the surrounding anatomy to prevent potential damages in the real surgical procedure. Even if the software has been designed for a specific bimanual surgical robot, it supports any open kinematic chain structure: as far as it can be described in our custom format. The robot capabilities to accomplish specific tasks can be virtually tested using the deformable models: interacting directly with the target virtual organs, trying to avoid unwanted collisions with the surrounding anatomy not involved in the intervention. Moreover, the surgical simulator has been enhanced with algorithms and data structures to integrate biomechanical parameters into virtual deformable models (based on mass-spring-damper network) of target solid organs, in order to properly reproduce the physical behaviour of the patient anatomy during the interactions. The main biomechanical parameters (Young's modulus and density) have been integrated, allowing the automatic tuning of some model network elements, such as: the node mass and the spring stiffness. The spring damping coefficient has been modeled using the Rayleigh approach. Furthermore, the developed method automatically detect the external layer, allowing the usage of both the surface and internal Young's moduli, in order to model the main parts of dense organs: the stroma and the parenchyma. Finally the model can be manually tuned to represent lesion with specific biomechanical properties. Additionally, some software modules of the simulator have been properly extended to be integrated in a patient specific computer guidance system for intraoperative navigation and assistance in robotic single port interventions. This application provides guidance functionalities working in three different modalities: passive as a surgical navigator, assistive as a guide for the single port placement and active as a tutor preventing unwanted collision during the intervention. The simulation system has beed tested by five surgeons: simulating the robot access port placemen, and evaluating the robot movability and workspace inside the patient abdomen. The tested functionalities, rated by expert surgeons, have shown good quality and performance of the simulation. Moreover, the integration of biomechanical parameters into deformable models has beed tested with various material samples. The results have shown a good visual realism ensuring the performance required by an interactive simulation. Finally, the intraoperative navigator has been tested performing a cholecystectomy on a synthetic patient mannequin, in order to evaluate: the intraoperative navigation accuracy, the network communications latency and the overall usability of the system. The tests performed demonstrated the effectiveness and the usability of the software systems developed: encouraging the introduction of the proposed solution in the clinical practice, and the implementation of further improvements. Surgical robotics will be enhanced by an advanced integration of medical images into software systems: allowing the detailed planning of surgical interventions by means of virtual surgery simulation based on patient specific biomechanical parameters. Furthermore, the advanced functionalities offered by these systems, enable surgical robots to improve the intraoperative surgical assistance: benefitting of the knowledge of the virtual patient anatomy

Exploration, design and application of simulation based technology in interventional cardiology

Medical education is undergoing a vast change from the traditional apprenticeship model to technology driven delivery of training to meet the demands of the new generation of doctors. With the reduction in the training hours of junior doctors, technology driven education can compensate for the time deficit in training. Each new technology arrives on a wave of great expectations; sometimes our expectations of true change are met and sometimes the new technology remains as a passing fashion only. The aim of the thesis is to explore, design and apply simulation based applications in interventional cardiology for educating the doctors and the public. Chapters 1and 2 present an overview of the current practice of education delivery and the evidence concerning simulation based education in interventional cardiology. Introduction of any new technology into an established system is often met with resistance. Hence Chapters 3 and 4 explore the attitudes and perceptions of consultants and trainees in cardiology towards the integration of a simulation based education into the cardiology curriculum. Chapters 5 and 6 present the “i-health project,” introduction of an electronic form for clinical information transfer from the ambulance crew to the hospital, enactment of case scenarios of myocardial infarction of varied levels of difficulty in a simulated environment and preliminary evaluation of the simulation. Chapter 7 focuses on educating the public in cardiovascular diseases and in coronary interventional procedures through simulation technology. Finally, Chapter 8 presents an overview of my findings, limitations and the future research that needs to be conducted which will enable the successful adoption of simulation based education into the cardiology curriculum.Open Acces

History of virtual reality and its use in medicine

Virtual Reality can be defined as an advanced interface between man and machine, that simulates a realistic environment and allows participants to interact with such ambience. This Science has its origins in the military academies and in the entertainment industry, and experienced great impulse in the second half of twentieth century. Using techniques like stereoscopy, it is possible to use human senses and to emulate experiences with much fidelity. Models of Virtual Reality became more and more important for medicine. Today, we can find many applications in this field, such as preoperative planning, assistance, surgical training and teaching. For the study of neuroanatomy and neurosurgery in particular, the technology of Virtual Reality has shown great effectiveness and utility, despite of high costs and low quality of visual information. Virtual Reality is now definitively included in the context of medical education.Realidade Virtual pode ser definida como uma avançada interface homem-máquina que simula um ambiente realístico e permite que participantes interajam com ele. Essa ciência tem suas origens nas academias militares e na indústria do entretenimento, com grande impulso na segunda metade do século XX. Através de técnicas como a estereoscopia, é possível usar os sentidos humanos e emular experiências com grande fidelidade. Modelos de Realidade Virtual ganham importância cada vez maior para a medicina. Hoje é possível encontrar diversas aplicações nessa área, como planejamento pré-operatório, assistência, treinamento cirúrgico e ensino. Para o estudo da neuroanatomia e da neurocirurgia especificamente, embora os custos proibitivos e a experiência visual limitada ainda sejam realidade, a tecnologia da Realidade Virtual tem demonstrado grande eficiência e utilidade, assumindo definitivamente seu papel no contexto da educação médica

Surgical Subtask Automation for Intraluminal Procedures using Deep Reinforcement Learning

Intraluminal procedures have opened up a new sub-field of minimally invasive surgery that use flexible instruments to navigate through complex luminal structures of the body, resulting in reduced invasiveness and improved patient benefits. One of the major challenges in this field is the accurate and precise control of the instrument inside the human body. Robotics has emerged as a promising solution to this problem. However, to achieve successful robotic intraluminal interventions, the control of the instrument needs to be automated to a large extent. The thesis first examines the state-of-the-art in intraluminal surgical robotics and identifies the key challenges in this field, which include the need for safe and effective tool manipulation, and the ability to adapt to unexpected changes in the luminal environment. To address these challenges, the thesis proposes several levels of autonomy that enable the robotic system to perform individual subtasks autonomously, while still allowing the surgeon to retain overall control of the procedure. The approach facilitates the development of specialized algorithms such as Deep Reinforcement Learning (DRL) for subtasks like navigation and tissue manipulation to produce robust surgical gestures. Additionally, the thesis proposes a safety framework that provides formal guarantees to prevent risky actions. The presented approaches are evaluated through a series of experiments using simulation and robotic platforms. The experiments demonstrate that subtask automation can improve the accuracy and efficiency of tool positioning and tissue manipulation, while also reducing the cognitive load on the surgeon. The results of this research have the potential to improve the reliability and safety of intraluminal surgical interventions, ultimately leading to better outcomes for patients and surgeons

Patient-specific simulation environment for surgical planning and preoperative rehearsal

Surgical simulation is common practice in the fields of surgical education and training. Numerous surgical simulators are available from commercial and academic organisations for the generic modelling of surgical tasks. However, a simulation platform is still yet to be found that fulfils the key requirements expected for patient-specific surgical simulation of soft tissue, with an effective translation into clinical practice. Patient-specific modelling is possible, but to date has been time-consuming, and consequently costly, because data preparation can be technically demanding. This motivated the research developed herein, which addresses the main challenges of biomechanical modelling for patient-specific surgical simulation. A novel implementation of soft tissue deformation and estimation of the patient-specific intraoperative environment is achieved using a position-based dynamics approach. This modelling approach overcomes the limitations derived from traditional physically-based approaches, by providing a simulation for patient-specific models with visual and physical accuracy, stability and real-time interaction. As a geometrically- based method, a calibration of the simulation parameters is performed and the simulation framework is successfully validated through experimental studies. The capabilities of the simulation platform are demonstrated by the integration of different surgical planning applications that are found relevant in the context of kidney cancer surgery. The simulation of pneumoperitoneum facilitates trocar placement planning and intraoperative surgical navigation. The implementation of deformable ultrasound simulation can assist surgeons in improving their scanning technique and definition of an optimal procedural strategy. Furthermore, the simulation framework has the potential to support the development and assessment of hypotheses that cannot be tested in vivo. Specifically, the evaluation of feedback modalities, as a response to user-model interaction, demonstrates improved performance and justifies the need to integrate a feedback framework in the robot-assisted surgical setting.Open Acces

Trends and features of virtual reality: a literature review between 2017 and 2018

Virtual Reality has greatly evolved since the 1960´s to the present. It has been implemented in multiple research and knowledge areas, the most recognized being the entertainment industry. In order to establish the state of Virtual Reality and get an idea of its prospects, 537 scientific documents have been reviewed, applying search criteria, more specifically, Virtual Reality applied to education. A bibliometric analysis was realized based on summarizing each document, as well as its keywords and trends, then proceeding to categorize each one, according to the field of application. It was found that Virtual Reality is having relevance in medicine and training area, due to the ability to simulate difficult situations and above all, specific conditions raised by instructors. The future of Virtual Reality as a tool to train professionals in multiples areas is promising.La realidad virtual ha tenido una gran evolución desde la década de 1960 hasta el día de hoy. Se ha implementado en múltiples áreas, tanto de la investigación como del conocimiento, siendo más reconocida en la industria del entretenimiento. Con el propósito de establecer el estado de la realidad virtual y obtener una idea de su futuro, 537 documentos científicos han sido revisados aplicando criterios de búsqueda específicos, como realidad virtual aplicada a la educación. Un análisis bibliométrico fue realizado, teniendo como base un resumen y descripción de cada artículo, así como sus palabras claves y tendencias, procediendo a categorizar cada documento de acuerdo con su campo de aplicación. Se encontró que la realidad virtual tiene una gran relevancia en la medicina, industria militar y en el entrenamiento de personas, debido a su capacidad de simular situaciones difíciles y, sobre todo, condiciones específicas requeridas por instructores. El futuro de la realidad virtual como herramienta de entrenamiento para los profesionales de múltiples áreas es promisori

Tendencias y características de la realidad virtual: Una revisión de la literatura

Virtual Reality has greatly evolved between 1940s and 1990s. Since then, it has been implemented in multiple research and knowledge areas, the most recognized being the entertainment industry. In order to establish the state of Virtual Reality and get an idea of its prospects, 537 scientific documents have been reviewed, applying search criteria, more specifically, Virtual Reality applied to education. A bibliometric analysis was realized based on summarizing each document, as well as its keywords and trends, then proceeding to categorize each one, according to the field of application. It was found that Virtual Reality is having relevance in medicine and training area, due to the ability to simulate difficult situations and above all, specific conditions raised by instructors. The future of Virtual Reality as a tool to train professionals in multiples areas is promising.La realidad virtual ha tenido una gran avance entre la década 1940 y 1990. Desde entonces, se ha implementado en múltiples áreas, tanto de la investigación como del conocimiento, siendo más reconocida en la industria del entretenimiento. Con el propósito de establecer el estado de la realidad virtual y obtener una idea de su futuro, 537 documentos científicos han sido revisados aplicando criterios de búsqueda específicos, como realidad virtual aplicada a la educación. Un análisis bibliométrico fue realizado, teniendo como base un resumen y descripción de cada artículo, así como sus palabras claves y tendencias, procediendo a categorizar cada documento de acuerdo con su campo de aplicación. Se encontró que la realidad virtual tiene una gran relevancia en la medicina y en el entrenamiento de personas, debido a su capacidad de simular situaciones difíciles y, sobre todo, condiciones específicas requeridas por instructores. El futuro de la realidad virtual como herramienta de entrenamiento para los profesionales de múltiples áreas es promisori