Improved human-robot collaborative control of redundant robot for teleoperated minimally invasive surgery

© 2016 IEEE. An improved human-robot collaborative control scheme is proposed in a teleoperated minimally invasive surgery scenario, based on a hierarchical operational space formulation of a seven-degree-of-freedom redundant robot. Redundancy is exploited to guarantee a remote center of motion (RCM) constraint and to provide a compliant behavior for the medical staff. Based on the implemented hierarchical control framework, an RCM constraint and a safe constraint are applied to the null-space motion to achieve the surgical tasks with human-robot interaction. Due to the physical interactions, safety and accuracy of the surgery may be affected. The control framework integrates an adaptive compensator to enhance the accuracy of the surgical tip and to maintain the RCM constraint in a decoupled way avoiding any physical interactions. The system performance is verified on a patient phantom. Compared with the methods proposed in the literature, results show that the accuracy of both the RCM constraint and the surgical tip is improved. The compliant swivel motion of the robot arm is also constrained in a defined area, and the interaction force on the abdominal wall becomes smaller

Cooperative Object Manipulation with Force Tracking on the da Vinci Research Kit

The da Vinci Surgical System is one of the most established robot-assisted surgery device commended for its dexterity and ergonomics in minimally invasive surgery. Conversely, it inherits disadvantages which are lack of autonomy and haptic feedback. In order to address these issues, this work proposes an industry-inspired solution to the field of force control in medical robotics. This approach contributes to shared autonomy by developing a controller for cooperative object manipulation with force tracking utilizing available manipulators and force feedback. To achieve simultaneous position and force tracking of the object, master and slave manipulators were assigned then controlled with Cartesian position control and impedance control respectively. Because impedance control requires a model-based feedforward compensation, we identified the lumped base parameters of mass, inertias, and frictions of a three degree-of-freedom double four-bar linkage mechanism with least squares and weighted least squares regression methods. Additionally, semidefinite programming was used to constrain the parameters to a feasible physical solution in standard parameter space. Robust stick-slip static friction compensation was applied where linear Viscous and Coulomb friction was inadequate in modeling the prismatic third joint. The Robot Operating System based controller was tested in RViz to check the cooperative kinematics of up to three manipulators. Additionally, simulation with the dynamic engine Gazebo verified the cooperative controller applying a constant tension force on a massless spring-damper virtual object. With adequate model feedback linearization, the cooperative impedance controller tested on the da Vinci Research Kit yielded stable tension force tracking while simultaneously moving in Cartesian space. The maximum force tracking error was +/- 0.5 N for both a compliant and stiff manipulated object

Contact aware robust semi-autonomous teleoperation of mobile manipulators

In the context of human-robot collaboration, cooperation and teaming, the use of mobile manipulators is widespread on applications involving unpredictable or hazardous environments for humans operators, like space operations, waste management and search and rescue on disaster scenarios. Applications where the manipulator's motion is controlled remotely by specialized operators. Teleoperation of manipulators is not a straightforward task, and in many practical cases represent a common source of failures. Common issues during the remote control of manipulators are: increasing control complexity with respect the mechanical degrees of freedom; inadequate or incomplete feedback to the user (i.e. limited visualization or knowledge of the environment); predefined motion directives may be incompatible with constraints or obstacles imposed by the environment. In the latter case, part of the manipulator may get trapped or blocked by some obstacle in the environment, failure that cannot be easily detected, isolated nor counteracted remotely. While control complexity can be reduced by the introduction of motion directives or by abstraction of the robot motion, the real-time constraint of the teleoperation task requires the transfer of the least possible amount of data over the system's network, thus limiting the number of physical sensors that can be used to model the environment. Therefore, it is of fundamental to define alternative perceptive strategies to accurately characterize different interaction with the environment without relying on specific sensory technologies. In this work, we present a novel approach for safe teleoperation, that takes advantage of model based proprioceptive measurement of the robot dynamics to robustly identify unexpected collisions or contact events with the environment. Each identified collision is translated on-the-fly into a set of local motion constraints, allowing the exploitation of the system redundancies for the computation of intelligent control laws for automatic reaction, without requiring human intervention and minimizing the disturbance of the task execution (or, equivalently, the operator efforts). More precisely, the described system consist in two different building blocks. The first, for detecting unexpected interactions with the environment (perceptive block). The second, for intelligent and autonomous reaction after the stimulus (control block). The perceptive block is responsible of the contact event identification. In short, the approach is based on the claim that a sensorless collision detection method for robot manipulators can be extended to the field of mobile manipulators, by embedding it within a statistical learning framework. The control deals with the intelligent and autonomous reaction after the contact or impact with the environment occurs, and consist on an motion abstraction controller with a prioritized set of constrains, where the highest priority correspond to the robot reconfiguration after a collision is detected; when all related dynamical effects have been compensated, the controller switch again to the basic control mode

Shared control for natural motion and safety in hands-on robotic surgery

Hands-on robotic surgery is where the surgeon controls the tool's motion by applying forces and torques to the robot holding the tool, allowing the robot-environment interaction to be felt though the tool itself. To further improve results, shared control strategies are used to combine the strengths of the surgeon with those of the robot. One such strategy is active constraints, which prevent motion into regions deemed unsafe or unnecessary. While research in active constraints on rigid anatomy has been well-established, limited work on dynamic active constraints (DACs) for deformable soft tissue has been performed, particularly on strategies which handle multiple sensing modalities. In addition, attaching the tool to the robot imposes the end effector dynamics onto the surgeon, reducing dexterity and increasing fatigue. Current control policies on these systems only compensate for gravity, ignoring other dynamic effects. This thesis presents several research contributions to shared control in hands-on robotic surgery, which create a more natural motion for the surgeon and expand the usage of DACs to point clouds. A novel null-space based optimization technique has been developed which minimizes the end effector friction, mass, and inertia of redundant robots, creating a more natural motion, one which is closer to the feeling of the tool unattached to the robot. By operating in the null-space, the surgeon is left in full control of the procedure. A novel DACs approach has also been developed, which operates on point clouds. This allows its application to various sensing technologies, such as 3D cameras or CT scans and, therefore, various surgeries. Experimental validation in point-to-point motion trials and a virtual reality ultrasound scenario demonstrate a reduction in work when maneuvering the tool and improvements in accuracy and speed when performing virtual ultrasound scans. Overall, the results suggest that these techniques could increase the ease of use for the surgeon and improve patient safety.Open Acces

Robotic Assistant Systems for Otolaryngology-Head and Neck Surgery

Recently, there has been a significant movement in otolaryngology-head and neck surgery (OHNS) toward minimally invasive techniques, particularly those utilizing natural orifices. However, while these techniques can reduce the risk of complications encountered with classic open approaches such as scarring, infection, and damage to healthy tissue in order to access the surgical site, there remain significant challenges in both visualization and manipulation, including poor sensory feedback, reduced visibility, limited working area, and decreased precision due to long instruments. This work presents two robotic assistance systems which help to overcome different aspects of these challenges. The first is the Robotic Endo-Laryngeal Flexible (Robo-ELF) Scope, which assists surgeons in manipulating flexible endoscopes. Flexible endoscopes can provide superior visualization compared to microscopes or rigid endoscopes by allowing views not constrained by line-of-sight. However, they are seldom used in the operating room due to the difficulty in precisely manually manipulating and stabilizing them for long periods of time. The Robo-ELF Scope enables stable, precise robotic manipulation for flexible scopes and frees the surgeon’s hands to operate bimanually. The Robo-ELF Scope has been demonstrated and evaluated in human cadavers and is moving toward a human subjects study. The second is the Robotic Ear Nose and Throat Microsurgery System (REMS), which assists surgeons in manipulating rigid instruments and endoscopes. There are two main types of challenges involved in manipulating rigid instruments: reduced precision from hand tremor amplified by long instruments, and difficulty navigating through complex anatomy surrounded by sensitive structures. The REMS enables precise manipulation by allowing the surgeon to hold the surgical instrument while filtering unwanted movement such as hand tremor. The REMS also enables augmented navigation by calculating the position of the instrument with high accuracy, and combining this information with registered preoperative imaging data to enforce virtual safety barriers around sensitive anatomy. The REMS has been demonstrated and evaluated in user studies with synthetic phantoms and human cadavers

Improved coordinated automatic voltage control in power grids through complex network analysis

PhD ThesisAutomatic and Co-ordinated Voltage Regulation (CVR) contributes significantly to economy and security of transmission grids. The role of CVR will become more critical as systems are operated closer to their capacity limits due to technical, economic and environmental reasons. CVR has 1 min resolution and owing to the inherent complexity of the task, CVR is enabled through zoning-based Reduced Control Models (RCM) i.e. simplified models of the network suitable for Voltage Control. RCM not only enables CVR bus also affects its performance and robustness. This thesis contributes towards improved CVR through thorough investigation of the RCM. As a starting point, with current power systems structure in mind, this work investigates static RCM schemes (i.e. fixed Reduced Control Model for all network configurations). To that end this thesis develops: (1) a novel generic framework for CVR modelling and evaluation and (2) new zoning-based RCM approaches using Complex Network Analysis. The evaluation of CVR in conjunction with both static and adaptive RCM schemes are based on a novel framework for CVR modelling and evaluation. This framework is generic and can be used to facilitate the selection and design of any of the CVR components. As a next step, due to the fact that a single RCM cannot be optimal for all network configurations, adaptive RCM (i.e. RCM determined in an online event driven fashion) is investigated using the proposed framework. This concerns future transmission grids where RCM is driven by the need for reliability rather than economy of measurement points at a planning phase. Lastly, this thesis examines zone division in an interconnected system ranging from EHV down to MV, and assesses the required degree of co-ordination for the voltage control of these zones. Essentially, this last item extends the scope of this work’s contributions beyond a single transmission-level Independent System Operator (ISO).EPSRC for funding my Research and the Consortium of the “Autonomic Power System” project

Doctor of Philosophy

dissertationIn this dissertation, we present methods for intuitive telemanipulation of manipulators that use piezoelectric stick-slip actuators (PSSAs). Commercial micro/nano-manipulators, which utilize PSSAs to achieve high precision over a large workspace, are typically controlled by a human operator at the joint level, leading to unintuitive and time-consuming telemanipulation. Prior work has considered the use of computer-vision-feedback to close a control loop for improved performance, but computer-vision-feedback is not a viable option for many end users. We discuss how open-loop models of the micro/nano-manipulator can be used to achieve desired end-effector movements, and we explain the process of obtaining open-loop models. We propose a rate-control telemanipulation method that utilizes the obtained model, and we experimentally quantify the effectiveness of the method using a common commercial manipulator (the Kleindiek MM3A). The utility of open-loop control methods for PSSAs with a human in the loop depends directly on the accuracy of the open-loop models of the manipulator. Prior research has shown that modeling of piezoelectric actuators is not a trivial task as they are known to suffer from nonlinearities that degrade their performance. We study the effect of static (non-inertial) loads on a prismatic and a rotary PSSA, and obtain a model relating the step size of the actuator to the load. The actuator-specific parameters of the model are calibrated by taking measurements in specific configurations of the manipulator. Results comparing the obtained model to experimental data are presented. PSSAs have properties that make them desirable over traditional DC-motor actuators for use in retinal surgery. We present a telemanipulation system for retinal surgery that uses a full range of existing disposable instruments. The system uses a PSSA-based manipulator that is compact and light enough that it could reasonably be made head-mounted to passively compensate for head movements. Two mechanisms are presented that enable the system to use existing disposable actuated instruments, and an instrument adapter enables quick-change of instruments during surgery. A custom stylus for a haptic interface enables intuitive and ergonomic telemanipulation of actuated instruments. Experimental results with a force-sensitive phantom eye show that telemanipulated surgery results in reduced forces on the retina compared to manual surgery, and training with the system results in improved performance. Finally, we evaluate operator efficiency with different haptic-interface kinematics for telemanipulated retinal surgery. Surgical procedures of the retina require precise manipulation of instruments inserted through trocars in the sclera. Telemanipulated robotic systems have been developed to improve retinal surgery, but there is not a unique mapping of the motions of the surgeon's hand to the lower-dimensional motions of the instrument through the trocar. We study operator performance during a precision positioning task on a force-sensing phantom retina, reminiscent of telemanipulated retinal surgery, with three common haptic-interface kinematics implemented in software on a PHANTOM Premium 6DOF haptic interface. Results from a study with 12 human subjects show that overall performance is best with the kinematics that represent a compact and inexpensive option, and that subjects' subjective preference agrees with the objective performance results

Augmentation Of Human Skill In Microsurgery

Surgeons performing highly skilled microsurgery tasks can benefit from information and manual assistance to overcome technological and physiological limitations to make surgery safer, efficient, and more successful. Vitreoretinal surgery is particularly difficult due to inherent micro-scale and fragility of human eye anatomy. Additionally, surgeons are challenged by physiological hand tremor, poor visualization, lack of force sensing, and significant cognitive load while executing high-risk procedures inside the eye, such as epiretinal membrane peeling. This dissertation presents the architecture and the design principles for a surgical augmentation environment which is used to develop innovative functionality to address the fundamental limitations in vitreoretinal surgery. It is an inherently information driven modular system incorporating robotics, sensors, and multimedia components. The integrated nature of the system is leveraged to create intuitive and relevant human-machine interfaces and generate a particular system behavior to provide active physical assistance and present relevant sensory information to the surgeon. These include basic manipulation assistance, audio-visual and haptic feedback, intraoperative imaging and force sensing. The resulting functionality, and the proposed architecture and design methods generalize to other microsurgical procedures. The system's performance is demonstrated and evaluated using phantoms and in vivo experiments