Planning and Real Time Control of a Minimally Invasive Robotic Surgery System

This paper introduces the planning and control software of a teleoperating robotic system for minimally invasive surgery. It addresses the problem of how to organize a complex system with 41 degrees of freedom including robot setup planning, force feedback control and nullspace handling with three robotic arms. The planning software is separated into sequentially executed planning and registration procedures. An optimal setup is first planned in virtual reality and then adapted to variations in the operating room. The real time control system is composed of hierarchical layers. The design is flexible and expandable without losing performance. Structure, functionality and implementation of planning and control are described. The robotic system provides the surgeon with an intuitive hand-eye-coordination and force feedback in teleoperation for both hands

Integration of New Features for Telerobotic Surgery into The Mirosurge System

International audienc

A peer-to-peer Trilateral Passivity Control for delayed collaborative Teleoperation

In this paper a trilateral Multi-Master-Single-Slave-System with control authority allocation between two human operators is proposed. The authority coefficient permits to slide the dominant role between the operators. They can simultaneously execute a task in a collaborative way or a trainee might haptically only observe the task, while an expert is in full control. The master devices are connected with each other and the slave robot peer to peer without a central processing unit in a equitable way. The system design is general in that it allows delayed communication and different coupling causalities between masters and slave, which can be located far from each other. The Time Domain Passivity Control Approach guarantees passivity of the network in the presence of communication delays. The methods presented are sustained with simulations and experiments using different authority coefficients

Eine Chirurgenkonsole für MiroSurge

In dieser Arbeit wird die neue Eingabekonsole des Chirurgiesystems MiroSurge vorgestellt. Die Konsole integriert zwei haptische Eingabegeräte, Sigma.7, welche von Force Dimension für das DLR entwickelt wurden. Das Sigma.7 verfügt über 7 Motoren um alle Raumfreiheitsgrade und einen Greifer aktiv anzutreiben. Es bietet damit die Möglichkeit zur Rückkopplung von Kräften und Momenten. Die maximalen Kräfte betragen dabei 20 N und die Momente 0.4 Nm im Arbeitsbereich. Durch eine steife Auslegung der Struktur, eine geringe Verkopplung der Trägheiten und eine unterstützende Regelung mit einem integrierten Kraft-/Momentensensor wird eine hohe Transparenz für den Bediener erreicht. Es soll damit ermöglicht werden auch geringe Variationen der Umgebungssteifigkeit zu ertasten, um z.B. einen Tumor zu lokalisieren. Zwei Eingabegeräte, für die linke und rechte Hand, sind ergonomisch günstig angeordnet. Die gesamte Konsole mit haptischen Eingabegeräten und 3D-Bildschirm ist elektrisch höhenverstellbar und kann sitzend oder stehend bedient werden. Die neue Chirurgenkonsole ist voll in das MiroSurge-System am DLR integriert

Planning and control of a teleoperation system for research in minimally invasive robotic surgery

Abstract—This paper introduces the planning and control software of a teleoperation system for research in minimally invasive robotic surgery. It addresses the problem of how to organize a complex system with 41 degrees of freedom as a flexible configurable platform. Robot setup planning, force feedback control and nullspace handling with three robotic arms are considered. The planning software is separated into sequentially executed planning and registration procedures. An optimal setup is first planned in virtual reality and then adapted to variations in the operating room. The real time control system is structured in hierarchical layers. Functions are arranged in the layers with respect to their domain and maximum response time. The design is flexible and expandable while performance is maintained. Structure, functionality and implementation of planning and control are described. The prototypic robotic system provides intuitive bimanual bilateral teleoperation within the planned working space. I

Optical-inertial tracking with active markers and changing visibility

This paper presents an optical-inertial tracking algorithm with explicit and assured optical marker identifica- tion. Active optical markers are sequentially or simultaneously triggered to achieve a maximum in quantity and quality of measurements available for tracking. Markers that appear in a camera image are identified by individual activation and are locally tracked in the 2D-images after initial identification. The 2D-position measurements of the cameras are combined with low latency measurements of acceleration and angular velocity from an inertial measurement unit. An Extended Kalman Filter is used for an ultra-tightly coupled data fusion, that takes advantage of all marker measurements with verified identity. The accurate, low latency tracking is robust with respect to temporary marker occlusions, as needed in applications where a robot is directly controlled with a tracked device. The tracking algorithms are implemented in real-time and verified with a test bed in a medical robotics context

Optical-Inertial Tracking of an Input Device for Real-Time Robot Control

Minimally invasive robotic surgery systems are usually controlled by input devices, that are mechanically linked to the environment. These input devices often have a limited workspace, which makes intuitive operation difficult. This paper presents a tracking algorithm of a handheld input device, which combines inertial and optical measurements to obtain accurate and robust state estimates with high update rates and low latency. It is based on the fusion of inertial and optical data in an error state extended Kalman filter. To achieve a high degree of robustness with respect to partial device occlusions, active optical markers are tracked and their 2D positions in the camera planes are directly forwarded to the fusion process. The algorithm can handle partial occlusions of the device in one or all of the cameras. A quality measure is defined, which indicates if tracking performance is sufficient to control a robot. An exemplary task in a medical robotics context verifies the assumption that the tracking system can be used for real-time robot control despite frequent marker occlusions

An approach to ultra-tightly coupled data fusion for handheld input devices in robotic surgery

This paper introduces an ultra-tightly coupled approach to data fusion of optical and inertial measurements. The two redundant sensor systems complement each other well, with the cameras providing absolute positions and the inertial measurements giving low latency information of derivatives. A targeted application is the tracking of handheld input devices for robotic surgery, where landmarks are not always visible to all cameras. Especially when bi-manual operation is considered, where one hand can move between the other hand and a camera, occlusions occur frequently. The ultra-tighly coupled data fusion uses 2D-camera measurements to correct pose estimations in an extended Kalman filter without an explicit 3D-reconstruction. Therefore marker measurements are used to support the pose estimation, even if the marker is only visible in one camera. Experiments were done with an inertial measurement unit and rectified stereo cameras that show the advantage of the approach