Shared control for natural motion and safety in hands-on robotic surgery

Hands-on robotic surgery is where the surgeon controls the tool's motion by applying forces and torques to the robot holding the tool, allowing the robot-environment interaction to be felt though the tool itself. To further improve results, shared control strategies are used to combine the strengths of the surgeon with those of the robot. One such strategy is active constraints, which prevent motion into regions deemed unsafe or unnecessary. While research in active constraints on rigid anatomy has been well-established, limited work on dynamic active constraints (DACs) for deformable soft tissue has been performed, particularly on strategies which handle multiple sensing modalities. In addition, attaching the tool to the robot imposes the end effector dynamics onto the surgeon, reducing dexterity and increasing fatigue. Current control policies on these systems only compensate for gravity, ignoring other dynamic effects. This thesis presents several research contributions to shared control in hands-on robotic surgery, which create a more natural motion for the surgeon and expand the usage of DACs to point clouds. A novel null-space based optimization technique has been developed which minimizes the end effector friction, mass, and inertia of redundant robots, creating a more natural motion, one which is closer to the feeling of the tool unattached to the robot. By operating in the null-space, the surgeon is left in full control of the procedure. A novel DACs approach has also been developed, which operates on point clouds. This allows its application to various sensing technologies, such as 3D cameras or CT scans and, therefore, various surgeries. Experimental validation in point-to-point motion trials and a virtual reality ultrasound scenario demonstrate a reduction in work when maneuvering the tool and improvements in accuracy and speed when performing virtual ultrasound scans. Overall, the results suggest that these techniques could increase the ease of use for the surgeon and improve patient safety.Open Acces

Complementary Situational Awareness for an Intelligent Telerobotic Surgical Assistant System

Robotic surgical systems have contributed greatly to the advancement of Minimally Invasive Surgeries (MIS). More specifically, telesurgical robots have provided enhanced dexterity to surgeons performing MIS procedures. However, current robotic teleoperated systems have only limited situational awareness of the patient anatomy and surgical environment that would typically be available to a surgeon in an open surgery. Although the endoscopic view enhances the visualization of the anatomy, perceptual understanding of the environment and anatomy is still lacking due to the absence of sensory feedback. In this work, these limitations are addressed by developing a computational framework to provide Complementary Situational Awareness (CSA) in a surgical assistant. This framework aims at improving the human-robot relationship by providing elaborate guidance and sensory feedback capabilities for the surgeon in complex MIS procedures. Unlike traditional teleoperation, this framework enables the user to telemanipulate the situational model in a virtual environment and uses that information to command the slave robot with appropriate admittance gains and environmental constraints. Simultaneously, the situational model is updated based on interaction of the slave robot with the task space environment. However, developing such a system to provide real-time situational awareness requires that many technical challenges be met. To estimate intraoperative organ information continuous palpation primitives are required. Intraoperative surface information needs to be estimated in real-time while the organ is being palpated/scanned. The model of the task environment needs to be updated in near real-time using the estimated organ geometry so that the force-feedback applied on the surgeon's hand would correspond to the actual location of the model. This work presents a real-time framework that meets these requirements/challenges to provide situational awareness of the environment in the task space. Further, visual feedback is also provided for the surgeon/developer to view the near video frame rate updates of the task model. All these functions are executed in parallel and need to have a synchronized data exchange. The system is very portable and can be incorporated to any existing telerobotic platforms with minimal overhead

A Scalable, High-Performance, Real-Time Control Architecture with Application to Semi-Autonomous Teleoperation

A scalable and real-time capable infrastructure is required to enable high-performance control and haptic rendering of systems with many degrees-of-freedom. The specific platform that motivates this thesis work is the open research platform da Vinci ReResearch Kit (dVRK). For the system architecture, we propose a specialized IEEE-1394 (FireWire) broadcast protocol that takes advantage of broadcast and peer-to-peer transfers to minimize the number of transactions, and thus the software overhead, on the control PC, thereby enabling fast real-time control. It has also been extended to Ethernet via a novel Ethernet-to-FireWire bridge protocol. The software architecture consists of a distributed hardware interface layer, a real-time component-based software framework, and integration with the Robot Operating System (ROS). The architecture is scalable to support multiple active manipulators, reconfigurable to enable researchers to partition a full system into multiple independent subsystems, and extensible at all levels of control. This architecture has been applied to two semi-autonomous teleoperation applications. The first application is a suturing task in Robotic Minimally Invasive Surgery (RMIS), that includes the development of virtual fixtures for the needle passing and knot tying sub-tasks, with a multi-user study to verify their effectiveness. The second application concerns time-delayed teleoperation of a robotic arm for satellite servicing. The research contribution includes the development of a line virtual fixture with augmented reality, a test for different time delay configurations and a multi-user study that evaluates the effectiveness of the system

Cable-driven parallel mechanisms for minimally invasive robotic surgery

Minimally invasive surgery (MIS) has revolutionised surgery by providing faster recovery times, less post-operative complications, improved cosmesis and reduced pain for the patient. Surgical robotics are used to further decrease the invasiveness of procedures, by using yet smaller and fewer incisions or using natural orifices as entry point. However, many robotic systems still suffer from technical challenges such as sufficient instrument dexterity and payloads, leading to limited adoption in clinical practice. Cable-driven parallel mechanisms (CDPMs) have unique properties, which can be used to overcome existing challenges in surgical robotics. These beneficial properties include high end-effector payloads, efficient force transmission and a large configurable instrument workspace. However, the use of CDPMs in MIS is largely unexplored. This research presents the first structured exploration of CDPMs for MIS and demonstrates the potential of this type of mechanism through the development of multiple prototypes: the ESD CYCLOPS, CDAQS, SIMPLE, neuroCYCLOPS and microCYCLOPS. One key challenge for MIS is the access method used to introduce CDPMs into the body. Three different access methods are presented by the prototypes. By focusing on the minimally invasive access method in which CDPMs are introduced into the body, the thesis provides a framework, which can be used by researchers, engineers and clinicians to identify future opportunities of CDPMs in MIS. Additionally, through user studies and pre-clinical studies, these prototypes demonstrate that this type of mechanism has several key advantages for surgical applications in which haptic feedback, safe automation or a high payload are required. These advantages, combined with the different access methods, demonstrate that CDPMs can have a key role in the advancement of MIS technology.Open Acces

ISMCR 1994: Topical Workshop on Virtual Reality. Proceedings of the Fourth International Symposium on Measurement and Control in Robotics

This symposium on measurement and control in robotics included sessions on: (1) rendering, including tactile perception and applied virtual reality; (2) applications in simulated medical procedures and telerobotics; (3) tracking sensors in a virtual environment; (4) displays for virtual reality applications; (5) sensory feedback including a virtual environment application with partial gravity simulation; and (6) applications in education, entertainment, technical writing, and animation

Smart Camera Robotic Assistant for Laparoscopic Surgery

The cognitive architecture also includes learning mechanisms to adapt the behavior of the robot to the different ways of working of surgeons, and to improve the robot behavior through experience, in a similar way as a human assistant would do. The theoretical concepts of this dissertation have been validated both through in-vitro experimentation in the labs of medical robotics of the University of Malaga and through in-vivo experimentation with pigs in the IACE Center (Instituto Andaluz de Cirugía Experimental), performed by expert surgeons.In the last decades, laparoscopic surgery has become a daily practice in operating rooms worldwide, which evolution is tending towards less invasive techniques. In this scenario, robotics has found a wide field of application, from slave robotic systems that replicate the movements of the surgeon to autonomous robots able to assist the surgeon in certain maneuvers or to perform autonomous surgical tasks. However, these systems require the direct supervision of the surgeon, and its capacity of making decisions and adapting to dynamic environments is very limited. This PhD dissertation presents the design and implementation of a smart camera robotic assistant to collaborate with the surgeon in a real surgical environment. First, it presents the design of a novel camera robotic assistant able to augment the capacities of current vision systems. This robotic assistant is based on an intra-abdominal camera robot, which is completely inserted into the patient’s abdomen and it can be freely moved along the abdominal cavity by means of magnetic interaction with an external magnet. To provide the camera with the autonomy of motion, the external magnet is coupled to the end effector of a robotic arm, which controls the shift of the camera robot along the abdominal wall. This way, the robotic assistant proposed in this dissertation has six degrees of freedom, which allow providing a wider field of view compared to the traditional vision systems, and also to have different perspectives of the operating area. On the other hand, the intelligence of the system is based on a cognitive architecture specially designed for autonomous collaboration with the surgeon in real surgical environments. The proposed architecture simulates the behavior of a human assistant, with a natural and intuitive human-robot interface for the communication between the robot and the surgeon

Realization of a demonstrator slave for robotic minimally invasive surgery

Robots for Minimally Invasive Surgery (MIS) can improve the surgeon’s work conditions with respect to conventional MIS and to enable MIS with more complex procedures. This requires to provide the surgeon with tactile feedback to feel forces executed on e.g. tissue and sutures, which is partially lost in conventional MIS. Additionally use of a robot should improve the approach possibilities of a target organ by means of instrument degrees of freedom (DoFs) and of the entry points with a compact set-up. These requirements add to the requirements set by the most common commercially available system, the da Vinci which are: (i) dexterity, (ii) natural hand-eye coordination, (iii) a comfortable body posture, (iv) intuitive utilization, and (v) a stereoscopic ’3D’ view of the operation site. The purpose of Sofie (Surgeon’s operating force-feedback interface Eindhoven) is to evaluate the possible benefit of force-feedback and the approach of both patient and target organ. Sofie integrates master, slave, electronic hardware and control. This thesis focusses on the design and realization of a technology demonstrator of the Slave. To provide good accuracy and valuable force-feedback, good dynamic behavior and limited hysteresis are required. To this end the Slave includes (i) a relatively short force-path between its instrument-tips and between tip and patient, and (ii) a passive instrument-support by means of a remote kinematically fixed point of rotation. The incision tissue does not support the instrument. The Slave is connected directly to the table. It provides a 20 DoF highly adaptable stiff frame (pre-surgical set-up) with a short force-path between the instrumenttips and between instrument-tip and patient. During surgery this frame supports three 4 DoF manipulators, two for exchangeable 4 DoF instruments and one for an endoscope. The pre-surgical set-up of the Slave consists of a 5 DoF platform-adjustment with a platform. This platform can hold three 5 DoF manipulator-adjustments in line-up. The set-up is compact to avoid interference with the team, entirely mechanical and allows fast manual adjustment and fixation of the joints. It provides a stiff frame during surgery. A weight-compensation mechanism for the platformadjustment has been proposed. Measurements indicate all natural frequencies are above 25 Hz. The manipulator moves the instrument in 4 DoFs (??, , ?? and Z). Each manipulator passively supports its instrument with a parallelogram mechanism, providing a kinematically fixed point of rotation. Two manipulators have been designed in consecutive order. The first manipulator drives with a worm-wormwheel, the second design uses a ball-screw drive. This ball-screw drive reduces friction, which is preferred for next generations of the manipulator, since the worm-wormwheel drive shows a relatively low coherence at low frequencies. The compact ??Zmanipulator moves the instrument in ?? by rotating a drum. Friction wheels in the drum provide Z. Eventually, the drum will be removable from the manipulator for sterilization. This layout of the manipulator results in a small motion-envelope and least obstructs the team at the table. Force sensors measuring forces executed with the instrument, are integrated in the manipulator instead of at the instrument tip, to avoid all risks of electrical signals being introduced into the patient. Measurements indicate the separate sensors function properly. Integrated in the manipulator the sensors provide a good indication of the force but do suffer from some hysteresis which might be caused by moving wires. The instrument as realized consists of a drive-box, an instrument-tube and a 4 DoF tip. It provides the surgeon with three DoFs additional to the gripper of conventional MIS instruments. These DoFs include two lateral rotations (pitch and pivot) to improve the approach possibilities and the roll DoF will contribute in stitching. Pitch and roll are driven by means of bevelgears, driven with concentric tubes. Cables drive the pivot and close DoFs of the gripper. The transmissions are backdriveable for safety. Theoretical torques that can be achieved with this instrument approximate the requirements closely. Further research needs to reveal the torques achieved in practice and whether the requirements obtained from literature actually are required for these 4 DoF instruments. Force-sensors are proposed and can be integrated. Sofie currently consists of a master prototype with two 5 DoF haptic interfaces, the Slave and an electronic hardware cabinet. The surgeon uses the haptic interfaces of the Master to manipulate the manipulators and instruments of the Slave, while the actuated DoFs of the Master provide the surgeon with force-feedback. This project resulted in a demonstrator of the slave with force sensors incorporated, compact for easy approach of the patient and additional DoFs to increase approach possibilities of the target organ. This slave and master provide a good starting point to implement haptic controllers. These additional features may ultimately benefit both surgeon and patient

Minimally Invasive Expeditionary Surgical Care Using Human-Inspired Robots

This technical report serves as an updated collection of subject matter experts on surgical care using human-inspired robotics for human exploration. It is a summary of the Blue Sky Meeting, organized by the Florida Institute for Human and Machine Cognition (IHMC), Pensacola, Florida, and held on October 2-3, 2018. It contains an executive summary, the final report, all of the presentation materials, and an updated reference list

Physical Diagnosis and Rehabilitation Technologies

The book focuses on the diagnosis, evaluation, and assistance of gait disorders; all the papers have been contributed by research groups related to assistive robotics, instrumentations, and augmentative devices