Patient Specific Systems for Computer Assisted Robotic Surgery Simulation, Planning, and Navigation

The evolving scenario of surgery: starting from modern surgery, to the birth of medical imaging and the introduction of minimally invasive techniques, has seen in these last years the advent of surgical robotics. These systems, making possible to get through the difficulties of endoscopic surgery, allow an improved surgical performance and a better quality of the intervention. Information technology contributed to this evolution since the beginning of the digital revolution: providing innovative medical imaging devices and computer assisted surgical systems. Afterwards, the progresses in computer graphics brought innovative visualization modalities for medical datasets, and later the birth virtual reality has paved the way for virtual surgery. Although many surgical simulators already exist, there are no patient specific solutions. This thesis presents the development of patient specific software systems for preoperative planning, simulation and intraoperative assistance, designed for robotic surgery: in particular for bimanual robots that are becoming the future of single port interventions. The first software application is a virtual reality simulator for this kind of surgical robots. The system has been designed to validate the initial port placement and the operative workspace for the potential application of this surgical device. Given a bimanual robot with its own geometry and kinematics, and a patient specific 3D virtual anatomy, the surgical simulator allows the surgeon to choose the optimal positioning of the robot and the access port in the abdominal wall. Additionally, it makes possible to evaluate in a virtual environment if a dexterous movability of the robot is achievable, avoiding unwanted collisions with the surrounding anatomy to prevent potential damages in the real surgical procedure. Even if the software has been designed for a specific bimanual surgical robot, it supports any open kinematic chain structure: as far as it can be described in our custom format. The robot capabilities to accomplish specific tasks can be virtually tested using the deformable models: interacting directly with the target virtual organs, trying to avoid unwanted collisions with the surrounding anatomy not involved in the intervention. Moreover, the surgical simulator has been enhanced with algorithms and data structures to integrate biomechanical parameters into virtual deformable models (based on mass-spring-damper network) of target solid organs, in order to properly reproduce the physical behaviour of the patient anatomy during the interactions. The main biomechanical parameters (Young's modulus and density) have been integrated, allowing the automatic tuning of some model network elements, such as: the node mass and the spring stiffness. The spring damping coefficient has been modeled using the Rayleigh approach. Furthermore, the developed method automatically detect the external layer, allowing the usage of both the surface and internal Young's moduli, in order to model the main parts of dense organs: the stroma and the parenchyma. Finally the model can be manually tuned to represent lesion with specific biomechanical properties. Additionally, some software modules of the simulator have been properly extended to be integrated in a patient specific computer guidance system for intraoperative navigation and assistance in robotic single port interventions. This application provides guidance functionalities working in three different modalities: passive as a surgical navigator, assistive as a guide for the single port placement and active as a tutor preventing unwanted collision during the intervention. The simulation system has beed tested by five surgeons: simulating the robot access port placemen, and evaluating the robot movability and workspace inside the patient abdomen. The tested functionalities, rated by expert surgeons, have shown good quality and performance of the simulation. Moreover, the integration of biomechanical parameters into deformable models has beed tested with various material samples. The results have shown a good visual realism ensuring the performance required by an interactive simulation. Finally, the intraoperative navigator has been tested performing a cholecystectomy on a synthetic patient mannequin, in order to evaluate: the intraoperative navigation accuracy, the network communications latency and the overall usability of the system. The tests performed demonstrated the effectiveness and the usability of the software systems developed: encouraging the introduction of the proposed solution in the clinical practice, and the implementation of further improvements. Surgical robotics will be enhanced by an advanced integration of medical images into software systems: allowing the detailed planning of surgical interventions by means of virtual surgery simulation based on patient specific biomechanical parameters. Furthermore, the advanced functionalities offered by these systems, enable surgical robots to improve the intraoperative surgical assistance: benefitting of the knowledge of the virtual patient anatomy

Augmented reality (AR) for surgical robotic and autonomous systems: State of the art, challenges, and solutions

Despite the substantial progress achieved in the development and integration of augmented reality (AR) in surgical robotic and autonomous systems (RAS), the center of focus in most devices remains on improving end-effector dexterity and precision, as well as improved access to minimally invasive surgeries. This paper aims to provide a systematic review of different types of state-of-the-art surgical robotic platforms while identifying areas for technological improvement. We associate specific control features, such as haptic feedback, sensory stimuli, and human-robot collaboration, with AR technology to perform complex surgical interventions for increased user perception of the augmented world. Current researchers in the field have, for long, faced innumerable issues with low accuracy in tool placement around complex trajectories, pose estimation, and difficulty in depth perception during two-dimensional medical imaging. A number of robots described in this review, such as Novarad and SpineAssist, are analyzed in terms of their hardware features, computer vision systems (such as deep learning algorithms), and the clinical relevance of the literature. We attempt to outline the shortcomings in current optimization algorithms for surgical robots (such as YOLO and LTSM) whilst providing mitigating solutions to internal tool-to-organ collision detection and image reconstruction. The accuracy of results in robot end-effector collisions and reduced occlusion remain promising within the scope of our research, validating the propositions made for the surgical clearance of ever-expanding AR technology in the future

From Concept to Market: Surgical Robot Development

Surgical robotics and supporting technologies have really become a prime example of modern applied information technology infiltrating our everyday lives. The development of these systems spans across four decades, and only the last few years brought the market value and saw the rising customer base imagined already by the early developers. This chapter guides through the historical development of the most important systems, and provide references and lessons learnt for current engineers facing similar challenges. A special emphasis is put on system validation, assessment and clearance, as the most commonly cited barrier hindering the wider deployment of a system

Robótica en cirugía y neurocirugía, aplicaciones y desafíos, una revisión

The integration of robots in operating rooms aims to improve the performance and efficiency of various procedures, since it offers remarkable advantages over conventional procedures, in particular precision, hand shake filtering and the possibility of executing complex tasks, however, Considerable challenges still prevail affecting massification and maneuverability on the part of surgeons. In the present work a review of the current state of robotic surgery, the challenges and trends is carried out. Specifically, the need for optimal force feedback mechanisms is evidenced, as well as dynamic visualization through augmented reality or virtual reality. It is not yet possible to determine that robotic surgery has reached standards, however, the integration of alternative technologies will allow surgeons to improve not only the efficiency of the robot, but also of its operation by the surgeonLa integración de robots en los quirófanos plantea mejorar el desempeño y eficiencia de variados procedimientos, dado que ofrece ventajas destacables sobre los procedimientos convencionales, en particular la precisión, el filtrado de temblor de mano y la posibilidad de ejecución de tareas complejas, sin embargo, aún prevalecen considerables desafíos que afectan la masificación y la maniobrabilidad por parte de los cirujanos. En el presente trabajo se realiza una revisión del estado actual de la cirugía robótica, los retos y las tendencias. En concreto se evidencia la necesidad de mecanismos de realimentación de fuerza óptimos, así como la visualización dinámica mediante realidad aumentada o realidad virtual. Aun no es posible determinar que la cirugía robótica ha alcanzado estándares, sin embargo, la integración de tecnologías alternas permitirá mejorar no solo la eficiencia en cuanto al robot sino respecto de su operación por parte de los cirujanos

Robótica en cirugía y neurocirugía, aplicaciones y desafíos, una revisión

The integration of robots in operating rooms aims to improve the performance and efficiency of various procedures, since it offers remarkable advantages over conventional procedures, in particular precision, hand shake filtering and the possibility of executing complex tasks, however, Considerable challenges still prevail affecting massification and maneuverability on the part of surgeons. In the present work a review of the current state of robotic surgery, the challenges and trends is carried out. Specifically, the need for optimal force feedback mechanisms is evidenced, as well as dynamic visualization through augmented reality or virtual reality. It is not yet possible to determine that robotic surgery has reached standards, however, the integration of alternative technologies will allow surgeons to improve not only the efficiency of the robot, but also of its operation by the surgeonLa integración de robots en los quirófanos plantea mejorar el desempeño y eficiencia de variados procedimientos, dado que ofrece ventajas destacables sobre los procedimientos convencionales, en particular la precisión, el filtrado de temblor de mano y la posibilidad de ejecución de tareas complejas, sin embargo, aún prevalecen considerables desafíos que afectan la masificación y la maniobrabilidad por parte de los cirujanos. En el presente trabajo se realiza una revisión del estado actual de la cirugía robótica, los retos y las tendencias. En concreto se evidencia la necesidad de mecanismos de realimentación de fuerza óptimos, así como la visualización dinámica mediante realidad aumentada o realidad virtual. Aun no es posible determinar que la cirugía robótica ha alcanzado estándares, sin embargo, la integración de tecnologías alternas permitirá mejorar no solo la eficiencia en cuanto al robot sino respecto de su operación por parte de los cirujanos

Robótica en cirugía y neurocirugía, aplicaciones y desafíos, una revisión

The integration of robots in operating rooms aims to improve the performance and efficiency of various procedures, since it offers remarkable advantages over conventional procedures, in particular precision, hand shake filtering and the possibility of executing complex tasks, however, Considerable challenges still prevail affecting massification and maneuverability on the part of surgeons. In the present work a review of the current state of robotic surgery, the challenges and trends is carried out. Specifically, the need for optimal force feedback mechanisms is evidenced, as well as dynamic visualization through augmented reality or virtual reality. It is not yet possible to determine that robotic surgery has reached standards, however, the integration of alternative technologies will allow surgeons to improve not only the efficiency of the robot, but also of its operation by the surgeonLa integración de robots en los quirófanos plantea mejorar el desempeño y eficiencia de variados procedimientos, dado que ofrece ventajas destacables sobre los procedimientos convencionales, en particular la precisión, el filtrado de temblor de mano y la posibilidad de ejecución de tareas complejas, sin embargo, aún prevalecen considerables desafíos que afectan la masificación y la maniobrabilidad por parte de los cirujanos. En el presente trabajo se realiza una revisión del estado actual de la cirugía robótica, los retos y las tendencias. En concreto se evidencia la necesidad de mecanismos de realimentación de fuerza óptimos, así como la visualización dinámica mediante realidad aumentada o realidad virtual. Aun no es posible determinar que la cirugía robótica ha alcanzado estándares, sin embargo, la integración de tecnologías alternas permitirá mejorar no solo la eficiencia en cuanto al robot sino respecto de su operación por parte de los cirujanos

Robotic implantation of intracerebral electrodes for deep brain stimulation

Dissertação de mestrado integrado em Engenharia BiomédicaThe objective of this dissertation is to develop an initial approach of a robotic system to play an assistive role in Deep Brain Stimulation (DBS) stereotactic neurosurgery. The robot is expected to position and manipulate several surgical instrumentation in a passive or semi-active role according to pre-operative directives and to medical team instructions. The current impact of neurological disorders sensitive to DBS, the underlying knowledge of neurostimulation and neuroanatomy, and practical insight about DBS surgery is studied to understand the ultimate goal of our project. We elaborated a state of the art search on neurosurgery robots to get the picture of what was done and what could be improved. Upon determining the optimal robotic system characteristics for DBS surgery, we conducted a search on industrial robotic manipulators to select the best candidates. The geometric and differential kinematic equations are developed for each robotic manipulator. To test the kinematic equations and the control application in a virtual operating room environment, we used the CoopDynSim simulator. Being this simulator oriented to mobile robots, we introduced the serial manipulator concept and implemented the selected robots with all specifications. We designed a control application to manoeuvre the robot and devised an initial interface towards positioning/manipulation of instrumentation along surgical trajectories, while emphasizing safety procedures. Although it was impossible to assess the robot’s precision in simulation, we studied how and where to place the manipulator to avoid collisions with surrounding equipment without restricting its flexibility.O objectivo desta dissertação é o desenvolvimento de uma abordagem inicial a um sistema robótico para desempenhar um papel de assistência em neurocirurgia estereotáxica de Estimulação Cerebral Profunda (DBS). O robô deve posicionar e manipular variados instrumentos cirúrgicos de uma forma passiva ou semi-ativa de acordo com diretivas pré-operativas ou com as instruções da equipa médica. O impacto atual dos distúrbios neurológicos sensíveis a DBS, o conhecimento subjacente de neuro-estimulação e neuro-anatomia, e conhecimento prático sobre a cirurgia de DBS são estudados para concluir sobre o objectivo final do nosso projeto. Nós elaborámos uma pesquisa sobre o estado da arte em robots neurocirúrgicos para perceber o que tem sido feito e o que pode ser melhorado. Após determinar o conjunto óptimo de características de um sistema robótico para cirurgia de DBS, nós procuramos manipuladores robóticos industriais para escolher os melhores candidatos. As cinemáticas geométricas e diferenciais são desenvolvidas para cada manipulador robótico. Para testar as equações cinemáticas e a aplicação de controlo num ambiente virtual de uma sala de operações, nós usamos o simulador CoopDynSim. Sendo este manipulador orientado a robôs móveis, nós introduzimos o conceito de manipuladores em série e implementamos os robôs selecionados com todas as especificações. Nós projetamos uma aplicação de controlo para manobrar os robôs e desenvolvemos uma interface inicial no sentido do posicionamento/manipulação de instrumentação ao longo de trajetórias cirúrgicas, enfatizando os procedimentos de segurança. Embora não tenha sido possível avaliar a precisão do robô em simulação, nós estudamos como e onde posicionar o manipulador de forma a evitar colisões com o equipamento circundante sem restringir a sua flexibilidade

Medical Robotics

The first generation of surgical robots are already being installed in a number of operating rooms around the world. Robotics is being introduced to medicine because it allows for unprecedented control and precision of surgical instruments in minimally invasive procedures. So far, robots have been used to position an endoscope, perform gallbladder surgery and correct gastroesophogeal reflux and heartburn. The ultimate goal of the robotic surgery field is to design a robot that can be used to perform closed-chest, beating-heart surgery. The use of robotics in surgery will expand over the next decades without any doubt. Minimally Invasive Surgery (MIS) is a revolutionary approach in surgery. In MIS, the operation is performed with instruments and viewing equipment inserted into the body through small incisions created by the surgeon, in contrast to open surgery with large incisions. This minimizes surgical trauma and damage to healthy tissue, resulting in shorter patient recovery time. The aim of this book is to provide an overview of the state-of-art, to present new ideas, original results and practical experiences in this expanding area. Nevertheless, many chapters in the book concern advanced research on this growing area. The book provides critical analysis of clinical trials, assessment of the benefits and risks of the application of these technologies. This book is certainly a small sample of the research activity on Medical Robotics going on around the globe as you read it, but it surely covers a good deal of what has been done in the field recently, and as such it works as a valuable source for researchers interested in the involved subjects, whether they are currently “medical roboticists” or not

Augmented Reality (AR) for Surgical Robotic and Autonomous Systems: State of the Art, Challenges, and Solutions

Despite the substantial progress achieved in the development and integration of augmented reality (AR) in surgical robotic and autonomous systems (RAS), the center of focus in most devices remains on improving end-effector dexterity and precision, as well as improved access to minimally invasive surgeries. This paper aims to provide a systematic review of different types of state-of-the-art surgical robotic platforms while identifying areas for technological improvement. We associate specific control features, such as haptic feedback, sensory stimuli, and human–robot collaboration, with AR technology to perform complex surgical interventions for increased user perception of the augmented world. Current researchers in the field have, for long, faced innumerable issues with low accuracy in tool placement around complex trajectories, pose estimation, and difficulty in depth perception during two-dimensional medical imaging. A number of robots described in this review, such as Novarad and SpineAssist, are analyzed in terms of their hardware features, computer vision systems (such as deep learning algorithms), and the clinical relevance of the literature. We attempt to outline the shortcomings in current optimization algorithms for surgical robots (such as YOLO and LTSM) whilst providing mitigating solutions to internal tool-to-organ collision detection and image reconstruction. The accuracy of results in robot end-effector collisions and reduced occlusion remain promising within the scope of our research, validating the propositions made for the surgical clearance of ever-expanding AR technology in the future

Neurosurgery and brain shift: review of the state of the art and main contributions of robotics

Este artículo presenta una revisión acerca de la neurocirugía, los asistentes robóticos en este tipo de procedimiento, y el tratamiento que se le da al problema del desplazamiento que sufre el tejido cerebral, incluyendo las técnicas para la obtención de imágenes médicas. Se abarca de manera especial el fenómeno del desplazamiento cerebral, comúnmente conocido como brain shift, el cual causa pérdida de referencia entre las imágenes preoperatorias y los volúmenes a tratar durante la cirugía guiada por imágenes médicas. Hipotéticamente, con la predicción y corrección del brain shift sobre el sistema de neuronavegación, se podrían planear y seguir trayectorias de mínima invasión, lo que conllevaría a minimizar el daño a los tejidos funcionales y posiblemente a reducir la morbilidad y mortalidad en estos delicados y exigentes procedimientos médicos, como por ejemplo, en la extirpación de un tumor cerebral. Se mencionan también otros inconvenientes asociados a la neurocirugía y se muestra cómo los sistemas robotizados han ayudado a solventar esta problemática. Finalmente se ponen en relieve las perspectivas futuras de esta rama de la medicina, la cual desde muchas disciplinas busca tratar las dolencias del principal órgano del ser humano.This paper presents a review about neurosurgery, robotic assistants in this type of procedure, and the approach to the problem of brain tissue displacement, including techniques for obtaining medical images. It is especially focused on the phenomenon of brain displacement, commonly known as brain shift, which causes a loss of reference between the preoperative images and the volumes to be treated during image-guided surgery. Hypothetically, with brain shift prediction and correction for the neuronavigation system, minimal invasion trajectories could be planned and shortened. This would reduce damage to functional tissues and possibly lower the morbidity and mortality in delicate and demanding medical procedures such as the removal of a brain tumor. This paper also mentions other issues associated with neurosurgery and shows the way robotized systems have helped solve these problems. Finally, it highlights the future perspectives of neurosurgery, a branch of medicine that seeks to treat the ailments of the main organ of the human body from the perspective of many disciplines