A First Evaluation of a Multi-Modal Learning System to Control Surgical Assistant Robots via Action Segmentation

The next stage for robotics development is to introduce autonomy and cooperation with human agents in tasks that require high levels of precision and/or that exert considerable physical strain. To guarantee the highest possible safety standards, the best approach is to devise a deterministic automaton that performs identically for each operation. Clearly, such approach inevitably fails to adapt itself to changing environments or different human companions. In a surgical scenario, the highest variability happens for the timing of different actions performed within the same phases. This paper presents a cognitive control architecture that uses a multi-modal neural network trained on a cooperative task performed by human surgeons and produces an action segmentation that provides the required timing for actions while maintaining full phase execution control via a deterministic Supervisory Controller and full execution safety by a velocity-constrained Model-Predictive Controller

Technical and Functional Validation of a Teleoperated Multirobots Platform for Minimally Invasive Surgery

Nowadays Robotic assisted Minimally Invasive Surgeries (R-MIS) are the elective procedures for treating highly accurate and scarcely invasive pathologies, thanks to their abil- ity to empower surgeons\u2019 dexterity and skills. The research on new Multi-Robots Surgery (MRS) platform is cardinal to the development of a new SARAS surgical robotic platform, which aims at carrying out autonomously the assistants tasks during R- MIS procedures. In this work, we will present the SARAS MRS platform validation protocol, framed in order to assess: (i) its technical performances in purely dexterity exercises, and (ii) its functional performances. The results obtained show a prototype able to put the users in the condition of accomplishing the tasks requested (both dexterity- and surgical-related), even with rea- sonably lower performances respect to the industrial standard. The main aspects on which further improvements are needed result to be the stability of the end effectors, the depth per- ception and the vision systems, to be enriched with dedicated virtual fixtures. The SARAS\u2019 aim is to reduce the main surgeon\u2019s workload through the automation of assistive tasks which would benefit both surgeons and patients by facilitating the surgery and reducing the operation time

Modeling of Surgical Procedures Using Statecharts for Semi-Autonomous Robotic Surgery

In this paper we propose a new methodology to model surgical procedures that is specifically tailored to semi-autonomous robotic surgery. We propose to use a restricted version of statecharts to merge the bottom-up approach, based on data-driven techniques (eg machine learning), with the top-down approach based on knowledge representation techniques. We consider medical knowledge about the procedure and sensing of the environment in two concurrent regions of the statecharts to facilitate re-usability and adaptability of the modules. Our approach allows producing a well defined procedural model exploiting the hierarchy capability of the statecharts, while machine learning modules act as soft sensors to trigger state transitions. Integrating data driven and prior knowledge techniques provides a robust, modular, flexible and re-configurable methodology to define a surgical procedure which is comprehensible by both humans and machines. We validate our approach on the three surgical phases of a Robot-Assisted Radical Prostatectomy (RARP) that directly involve the assistant surgeon: bladder mobilization, bladder neck transection, and vesicourethral anastomosis, all performed on synthetic manikins

Two-Layer-Based Multiarms Bilateral Teleoperation Architecture

In this article, we propose a novel bilateral tele- operation architecture for a multiarms system based on the two-layer approach. Exploiting the concept of shared energy tank, a passivity layer guarantees the passivity of the overall architecture with respect to destabilizing factors such as time delays in the communication channel. The desired behavior can then be freely designed in the transparency layer. The formulation of the energy tank is first revised, allowing a more efficient use of energy, and then extended, allowing explicitly the use of both admittance and impedance causality robots. A novel framework capable of combining the use of teleoperated and autonomous robots is proposed. The architecture has been tested and validated on a multiarms system in a realistic surgical scenario with the da Vinci research kit (dVRK) and an autonomous arm holding the endoscope

A supervisory controller for semi-autonomous surgical interventions

Nowadays the main research interests in the field of Robotic Minimally Invasive Surgery (R-MIS) are related to robots’ autonomy. Techniques like trajectory planning, collision avoidance, decision making and scene understanding require technical advances in order to be applied to such an environment. In this paper, we propose a deterministic supervisory controller for a surgical semi-autonomous robotic platform

A novel inverse dynamic model for 3-DoF delta robots

Delta Robots belong to a class of parallel robots widely used in industrial production processes, mostly for pick-and-place operations. The most relevant characteristics are the high speed and the extremely favorable ratio between the maximum payload and the weight of the robot itself. A reliable dynamic model is needed to implement torque controllers that reduce unnecessary high accelerations and so mechanical vibrations. The state-of-art inverse dynamic models exploit simplifications in order to facilitate the derivation of the equations of motion and their implementation. In this work, a novel and more rigorous inverse dynamic model is presented which does not rely on simplifications of the kinematic structure. The model has been validated comparing its estimations with real torques data collected moving a Delta Robot D3-1200 by SIPRO Srl; the computational complexity of the algorithm has also been investigated

Gray-Box Model Identification and Payload Estimation for Delta Robots

Delta Robots belong to a class of parallel robots widely used in industrial production processes, mostly for pick-and-place operations. The most relevant characteristics are the high speed and the extremely favorable ratio between the maximum payload and the weight of the robot itself. A reliable dynamic model is needed to implement torque controllers that reduce unnecessary high accelerations and so mechanical vibrations. Moreover, when the mass of the pickable object is unknown, it is crucial to identify with sufficient precision the dynamic contribution of the payload and to accordingly adapt the dynamic model in order to guarantee high performance

A Time-of-Flight Stereoscopic Endoscope for Anatomical 3D Reconstruction

This paper presents a novel endoscope design for laparoscopic surgery that has been specifically tailored to provide both a stereoscopic view to the surgeon and a high-accuracy 3D reconstruction for an advanced visualization of the anatomical environment. The former helps the main surgeon in teleoperating a robotic minimally-invasive system (R-MIS) while the latter provides necessary data to upcoming autonomous surgical procedure implementations in a manner akin to the current development of autonomous driving systems. To this aim, we created an initial prototype that incorporates a pair of high-quality, chip-on-tip RGB cameras with a Time-of-Flight (ToF) 3D sensor in a sufficiently compact design to allow its usage in intra-luminal operations. The combination of these sensors provides a reliable 3D model of the anatomical structures at close and far distances within the workspace to effectively overcome the issues presented by current laparoscopy stereo endoscopes, for which the depth estimation is hindered by the reduced baseline distance between the cameras. Moreover, the application to current robotic platforms presents innate mathematical issues when applying hand-eye calibration techniques for localization. We finally developed a calibration procedure that merges both color and depth information. The endoscope design is fully validated through the reconstruction of a planar surface, achieving a depth, latitudinal, and longitudinal orientation precision of 3.3mm,-0.02rad,-0.025rad respectively