Virtual Fixture Assistance for Suturing in Robot-Aided Pediatric Endoscopic Surgery

The limited workspace in pediatric endoscopic surgery makes surgical suturing one of the most difficult tasks. During suturing, surgeons have to prevent collisions between tools and also collisions with the surrounding tissues. Surgical robots have been shown to be effective in adult laparoscopy, but assistance for suturing in constrained workspaces has not been yet fully explored. In this letter, we propose guidance virtual fixtures to enhance the performance and the safety of suturing while generating the required task constraints using constrained optimization and Cartesian force feedback. We propose two guidance methods: looping virtual fixtures and a trajectory guidance cylinder, that are based on dynamic geometric elements. In simulations and experiments with a physical robot, we show that the proposed methods achieve a more precise and safer looping in robot-assisted pediatric endoscopy.Comment: Accepted on RA-L/ICRA 2020, 8 Pages. Fixed a few typo

Real-Time Support of Haptic Interaction by Means of Sampling-Based Path Planning

Haptic feedback enables the support of a human during the interaction with an environment. A variety of concepts have been developed to achieve an effective haptic support of the user in specific scenarios, e.g. Virtual Fixtures. However, most of these methods do not enable an adaptive support of the motion from a user within a (real or virtual) environment, which would be desirable in many situations. Especially when dynamical obstacles are involved or when the desired motion of the human is not known beforehand, an online computation of this support is essential, which should be based on a fast and effective determination of feasible motions.In contrast to other methods, sampling-based path planning is applicable to arbitrary interaction scenarios and enables to find a solution if it exists at all. Thus, it seems to be ideally suited for a generic framework that is able to deal with various kinematics, as e.g. a virtual prototyping test bed for the haptic evaluation of mechanisms requires. At such a test bed, the path planner could directly be coupled to the haptic rendering of a virtual scene to assist a user in approaching a target.This motivated the development of SamPP, a sampling-based path planning library with implementations of the most important algorithms. It can be used for nearly arbitrary rigid robots and environments. By performing numerous benchmarks, we prove the effectiveness and efficiency of SamPP. It is shown that a single-threaded version of the path planning can be used for real-time support of the haptic interaction at a novel actuated car door.Furthermore, we enhance the path planning performance for unknown or dynamical environments significantly by the OR-Parallelization of different path planning queries. This Generalized OR-Parallelization is a novel concept that to the best knowledge of the authors has not been proposed beforehand. We show that for the case of dynamic environments the likelihood of a fast path planning result is higher with our approach. Thus, even in dynamic or unknown environments, a real-time support of haptic interaction can be achieved. Finally, we highlight four promising research directions to exploit the concept of Generalized OR-Parallelization: 1) Combination of PRMs and RRTs to achieve a synergy of the advantages of both concepts, 2) concurrent use of different parameter sets of path planning algorithms, 3) online adaptation of these parameter sets and 4) online adaptation of the types and numbers of parallel executed path planning programs

Hybrid Stiff/Compliant Workspace Control for Robotized Minimally Invasive Surgery

Abstract-This paper presents a novel control architecture for hybrid stiff and compliant control for minimally invasive surgery which satisfies the constraints of zero lateral velocity at the entry point for serial manipulators. For minimally invasive surgery it is required that there is no sideways motion at the point where the robots enter the abdomen. This is necessary to avoid any damage to the patient's body when the robot moves. We solve this at a kinematic level, i.e., we find a Jacobian matrix that maps the velocities in joint space to the end-effector velocities and at the same time guarantees that certain velocities at the entry point are zero. Because the new velocity variables are defined in the end-effector workspace we can use these for hybrid motion/force control. The approach is verified experimentally by implementing hybrid stiff and compliant control of the end effector and we show that the insertion point constraints are always satisfied

CO-ROBOTIC ULTRASOUND IMAGING: A COOPERATIVE FORCE CONTROL APPROACH

Ultrasound (US) imaging remains one of the most commonly used imaging modalities in medical practice due to its low cost and safety. However, 63-91% of ultrasonographers develop musculoskeletal disorders due to the effort required to perform imaging tasks. Robotic ultrasound (RUS), the application of robotic systems to assist ultrasonographers in ultrasound scanning procedures, has been proposed in literature and recently deployed in clinical settings using limited degree-of-freedom (DOF) systems. An example of this includes breast-scanning systems, which allow one-DOF translation of a large ultrasound array in order to capture patients’ breast scans and minimize sonographer effort while preserving a desired clinical outcome. Recently, the robotic industry has evolved to provide light-weight, compact, accurate, and cost-effective manipulators. We leverage this new reality in able to provide ultrasonographers with a full 6-DOF system that provides force assistance to facilitate US image acquisition. Admittance robot control allows for smooth human-machine interaction in a desired task. In the case of RUS, force control is capable of assisting sonographers in facilitating and even improving the imaging results of typical procedures. We propose a new system setup for collaborative force control in US applications. This setup consists of the 6-DOF UR5 industrial robot, and a 6-axes force sensor attached to the robot tooltip, which in turn has an US probe attached to it through a custom-designed probe attachment mechanism. Additionally, an independent one-axis load cell is placed inside this attachment device and used to measure the contact force between the probe and the patient’s anatomy in real time and independent of any other forces. As the sonographer guides the US probe, the robot collaborates with the hand motions, following the path of the user. When imaging, the robot can offer assistance to the sonographer by augmenting the forces applied by him or her, thereby lessening the physical effort required as well as the resulting strain. Additional benefits include force and velocity limiting for patient safety and robot motion constraints for particular imaging tasks. Initial results of a conducted user study show the feasibility of implementing the presented robot-assisted system in a clinical setting

Model Driven Robotic Assistance for Human-Robot Collaboration

While robots routinely perform complex assembly tasks in highly structured factory environments, it is challenging to apply completely autonomous robotic systems in less structured manipulation tasks, such as surgery and machine assembly/repair, due to the limitations of machine intelligence, sensor data interpretation and environment modeling. A practical, yet effective approach to accomplish these tasks is through human-robot collaboration, in which the human operator and the robot form a partnership and complement each other in performing a complex task. We recognize that humans excel at determining task goals and recognizing constraints, if given sufficient feedback about the interaction between the tool (e.g., end-effector of the robot) and the environment. Robots are precise, unaffected by fatigue and able to work in environments not suitable for humans. We hypothesize that by providing the operator with adequate information about the task, through visual and force (haptic) feedback, the operator can: (1) define the task model, in terms of task goals and virtual fixture constraints through an interactive, or immersive augmented reality interface, and (2) have the robot actively assist the operator to enhance the execution time, quality and precision of the tasks. We validate our approaches through the implementations of both cooperative (i.e., hands-on) control and telerobotic systems, for image-guided robotic neurosurgery and telerobotic manipulation tasks for satellite servicing under significant time delay

A Scalable, High-Performance, Real-Time Control Architecture with Application to Semi-Autonomous Teleoperation

A scalable and real-time capable infrastructure is required to enable high-performance control and haptic rendering of systems with many degrees-of-freedom. The specific platform that motivates this thesis work is the open research platform da Vinci ReResearch Kit (dVRK). For the system architecture, we propose a specialized IEEE-1394 (FireWire) broadcast protocol that takes advantage of broadcast and peer-to-peer transfers to minimize the number of transactions, and thus the software overhead, on the control PC, thereby enabling fast real-time control. It has also been extended to Ethernet via a novel Ethernet-to-FireWire bridge protocol. The software architecture consists of a distributed hardware interface layer, a real-time component-based software framework, and integration with the Robot Operating System (ROS). The architecture is scalable to support multiple active manipulators, reconfigurable to enable researchers to partition a full system into multiple independent subsystems, and extensible at all levels of control. This architecture has been applied to two semi-autonomous teleoperation applications. The first application is a suturing task in Robotic Minimally Invasive Surgery (RMIS), that includes the development of virtual fixtures for the needle passing and knot tying sub-tasks, with a multi-user study to verify their effectiveness. The second application concerns time-delayed teleoperation of a robotic arm for satellite servicing. The research contribution includes the development of a line virtual fixture with augmented reality, a test for different time delay configurations and a multi-user study that evaluates the effectiveness of the system

Monitoring companion for industrial robotic processes

For system integrators, optimizing complex industrial robotic applications (e.g. robotised welding) is a difficult and time-consuming task. This procedure is rendered tedious and often very hard to achieve when the operator cannot access the robotic system once in operation, perhaps because the installation is far away or because of the operational environment. In these circumstances, as an alternative to physically visiting the installation site, the system integrator may rely on additional nearby sensors to remotely acquire the necessary process information. While it is hard to completely replace this trial and error approach, it is possible to provide a way to gather process information more effectively that can be used in several robotic installations.This thesis investigates the use of a "monitoring robot" in addition to the task robot(s) that belong to the industrial process to be optimized. The monitoring robot can be equipped with several different sensors and can be moved into close proximity of any installed task robot so that it can be used to collect information from that process during and/or after the operation without interfering. The thesis reviews related work in the industry and in the field of teleoperation to identify the most important challenges in remote monitoring and teleoperation. From the background investigation it is clear that two very important issues are: i) the nature of the teleoperator’s interface and; ii) the efficiency of the shared control between the human operator and the monitoring system. In order to investigate these two issues efficiently it was necessary to create experimental scenarios that operate independently from any application scenario, so an abstract problem domain is created. This way the monitoring system's control and interface can be evaluated in a context that presents challenges that are typical of a remote monitoring task but are not application domain specific. Therefore the validity of the proposed approach can be assessed from a generic and, therefore, more powerful and widely applicable perspective. The monitoring framework developed in this thesis is described, both in the shared control design choices based on virtual fixtures (VF) and the implementation in a 3D visualization environment. The monitoring system developed is evaluated with a usability study with user participants. The usability study aims at assessing the system's performance along with its acceptance and ease of use in a static monitoring task, accompanied by user\hyp{}filled TLX questionnaires. Since future work will apply this system in real robotic welding scenarios, this thesis finally reports some preliminary work in such an application