Control of Cooperative Haptics-Enabled Teleoperation Systems with Application to Minimally Invasive Surgery

Robot-Assisted Minimally Invasive Surgical (RAMIS) systems frequently have a structure of cooperative teleoperator systems where multiple master-slave pairs are used to collaboratively execute a task. Although multiple studies indicate that haptic feedback improves the realism of tool-tissue interaction to the surgeon and leads to better performance for surgical procedures, current telesurgical systems typically do not provide force feedback, mainly because of the inherent stability issues. The research presented in this thesis is directed towards the development of control algorithms for force reflecting cooperative surgical teleoperator systems with improved stability and transparency characteristics. In the case of cooperative force reflecting teleoperation over networks, conventional passivity based approaches may have limited applicability due to potentially non-passive slave-slave interactions and irregular communication delays imposed by the network. In this thesis, an alternative small gain framework for the design of cooperative network-based force reflecting teleoperator systems is developed. Using the small gain framework, control algorithms for cooperative force-reflecting teleoperator systems are designed that guarantee stability in the presence of multiple network-induced communication constraints. Furthermore, the design conservatism typically associated with the small-gain approach is eliminated by using the Projection-Based Force Reflection (PBFR) algorithms. Stability results are established for networked cooperative teleoperator systems under different types of force reflection algorithms in the presence of irregular communication delays. The proposed control approach is consequently implemented on a dual-arm (two masters/two slaves) robotic MIS testbed. The testbed consists of two Haptic Wand devices as masters and two PA10-7C robots as the slave manipulators equipped with da Vinci laparoscopic surgical instruments. The performance of the proposed control approach is evaluated in three different cooperative surgical tasks, which are knot tightening, pegboard transfer, and object manipulation. The experimental results obtained indicate that the PBFR algorithms demonstrate statistically significant performance improvement in comparison with the conventional direct force reflection algorithms. One possible shortcoming of using PBFR algorithms is that implementation of these algorithms may lead to attenuation of the high-frequency component of the contact force which is important, in particular, for haptic perception of stiff surfaces. In this thesis, a solution to this problem is proposed which is based on the idea of separating the different frequency bands in the force reflection signal and consequently applying the projection-based principle to the low-frequency component, while reflecting the high-frequency component directly. The experimental results demonstrate that substantial improvement in transient fidelity of the force feedback is achieved using the proposed method without negative effects on the stability of the system

Posture-Dependent Projection-Based Force Reflection Algorithms for Bilateral Teleoperators

It was previously established that the projection-based force reflection (PBFR) algorithms improve the overall stability of a force reflecting teleoperation system. The idea behind the PBFR algorithms is to identify the component of the reflected force which is compensated by interaction with the operator\u27s hand, and subsequently attenuate the residual component of the reflected force. If there is no a priori information regarding the behaviour of the human operator, the PBFR gain is selected equal to sufficiently small constant in order to guarantee stability for a wide range of human operator responses. Small PBRF gains, however, may deteriorate the transparency of a teleoperator system. In this thesis, a new method for selecting the PBFR gain is introduced which depends on human postures. Using an online human posture estimation, the introduced posture-dependent PBFR algorithm has been applied to a teleoperation system with force feedback. It is experimentally demonstrated that the developed method for selection of the PBFR gain based on human postures improves the transparency of the teleoperator system while the stability is preserved. Finally, preliminary results that deal with an extension of the developed methods towards a more realistic model of the human arm with 4 degrees of freedom and three dimensional movements are presented

Generalized Scattering-Based Stabilization of Nonlinear Interconnected Systems

The research presented in this thesis is aimed at development of new methods and techniques for stability analysis and stabilization of interconnections of nonlinear systems, in particular, in the presence of communication delays. Based on the conic systems\u27 formalism, we extend the notion of conicity for the non-planar case where the dimension of the cone\u27s central subspace may be greater than one. One of the advantages of the notion of non-planar conicity is that any dissipative system with a quadratic supply rate can be represented as a non-planar conic system; specifically, its central subspace and radius can be calculated using an algorithm developed in this thesis. For a feedback interconnection of two non-planar conic systems, a graph separation condition for finite-gain L2-stability is established in terms of central subspaces and radii of the subsystems\u27 non-planar cones. Subsequently, a generalized version of the scattering transformation is developed which is applicable to non-planar conic systems. The transformation allows for rendering the dynamics of a non-planar conic system into a prescribed cone with compatible dimensions; the corresponding design algorithm is presented. The ability of the generalized scattering transformation to change the parameters of a system\u27s cone can be used for stabilization of interconnections of non-planar conic systems. For interconnections without communication delays, stabilization is achieved through the design of a scattering transformation that guarantees the fulfilment of the graph separation stability condition. For interconnected systems with communication delays, scattering transformations are designed on both sides of communication channel in a way that guarantees the overall stability through fulfilment of the small gain stability condition. Application to stabilization of bilateral teleoperators with multiple heterogeneous communication delays is briefly discussed. Next, the coupled stability problem is addressed based on the proposed scattering based stabilization techniques. The coupled stability problem is one of the most fundamental problems in robotics. It requires to guarantee stability of a controlled manipulator in contact with an environment whose dynamics are unknown, or at least not known precisely. We present a scattering-based design procedure that guarantees coupled stability while at the same time does not affect the robot\u27s trajectory tracking performance in free space. A detailed design example is presented that demonstrates the capabilities of the scattering-based design approach, as well as its advantages in comparison with more conventional passivity-based approaches. Finally, the generalized scattering-based technique is applied to the problem of stabilization of complex interconnections of dissipative systems with quadratic supply rates in the presence of multiple heterogeneous constant time delays. Our approach is to design local scattering transformations that guarantee the fulfilment of a multi-dimensional small-gain stability condition for the interconnected system. A numerical example is presented that illustrates the capabilities of the proposed design method

Kinesthetic Haptics Sensing and Discovery with Bilateral Teleoperation Systems

In the mechanical engineering field of robotics, bilateral teleoperation is a classic but still increasing research topic. In bilateral teleoperation, a human operator moves the master manipulator, and a slave manipulator is controlled to follow the motion of the master in a remote, potentially hostile environment. This dissertation focuses on kinesthetic perception analysis in teleoperation systems. Design of the controllers of the systems is studied as the influential factor of this issue. The controllers that can provide different force tracking capability are compared using the same experimental protocol. A 6 DOF teleoperation system is configured as the system testbed. An innovative master manipulator is developed and a 7 DOF redundant manipulator is used as the slave robot. A singularity avoidance inverse kinematics algorithm is developed to resolve the redundancy of the slave manipulator. An experimental protocol is addressed and three dynamics attributes related to kineshtetic feedback are investigated: weight, center of gravity and inertia. The results support our hypothesis: the controller that can bring a better force feedback can improve the performance in the experiments

The Shape of Damping: Optimizing Damping Coefficients to Improve Transparency on Bilateral Telemanipulation

This thesis presents a novel optimization-based passivity control algorithm for hapticenabled bilateral teleoperation systems involving multiple degrees of freedom. In particular, in the context of energy-bounding control, the contribution focuses on the implementation of a passivity layer for an existing time-domain scheme, ensuring optimal transparency of the interaction along subsets of the environment space which are preponderant for the given task, while preserving the energy bounds required for passivity. The involved optimization problem is convex and amenable to real-time implementation. The effectiveness of the proposed design is validated via an experiment performed on a virtual teleoperated environment. The interplay between transparency and stability is a critical aspect in haptic-enabled bilateral teleoperation control. While it is important to present the user with the true impedance of the environment, destabilizing factors such as time delays, stiff environments, and a relaxed grasp on the master device may compromise the stability and safety of the system. Passivity has been exploited as one of the the main tools for providing sufficient conditions for stable teleoperation in several controller design approaches, such as the scattering algorithm, timedomain passivity control, energy bounding algorithm, and passive set position modulation. In this work it is presented an innovative energy-based approach, which builds upon existing time-domain passivity controllers, improving and extending their effectiveness and functionality. The set of damping coefficients are prioritized in each degree of freedom, the resulting transparency presents a realistic force feedback in comparison to the other directions. Thus, the prioritization takes effect using a quadratic programming algorithm to find the optimal values for the damping. Finally, the energy tanks approach on passivity control is a solution used to ensure stability in a system for robotics bilateral manipulation. The bilateral telemanipulation must maintain the principle of passivity in all moments to preserve the system\u2019s stability. This work presents a brief introduction to haptic devices as a master component on the telemanipulation chain; the end effector in the slave side is a representation of an interactive object within an environment having a force sensor as feedback signal. The whole interface is designed into a cross-platform framework named ROS, where the user interacts with the system. Experimental results are presented

Control of Networked Robotic Systems

With the infrastructure of ubiquitous networks around the world, the study of robotic systems over communication networks has attracted widespread attention. This area is denominated as networked robotic systems. By exploiting the fruitful technological developments in networking and computing, networked robotic systems are endowed with potential and capabilities for several applications. Robots within a network are capable of connecting with control stations, human operators, sensors, and other robots via digital communication over possibly noisy channels/media. The issues of time delays in communication and data losses have emerged as a pivotal issue that have stymied practical deployment. The aim of this dissertation is to develop control algorithms and architectures for networked robotic systems that guarantee stability with improved overall performance in the presence of time delays in communication. The first topic addressed in this dissertation is controlled synchronization that is utilized for networked robotic systems to achieve collective behaviors. Exploiting passivity property of individual robotic systems, the proposed control schemes and interconnections are shown to ensure stability and convergence of synchronizing errors. The robustness of the control algorithms to constant and time-varying communication delays is also studied. In addition to time delays, the number of communication links, which prevents scalability of networked robotic systems, is another challenging issue. Thus, a synchronizing control with practically feasible constraints of network topology is developed. The problem of networked robotic systems interacting with human operators is then studied subsequently. This research investigates a teleoperation system with heterogeneous robots under asymmetric and unknown communication delays. Sub-task controllers are proposed for redundant slave robot to autonomously achieve additional tasks, such as singularity avoidance, joint angle limits, and collision avoidance. The developed control algorithms can enhance the efficiency of teleoperation systems, thereby ameliorating the performance degradation due to cognitive limitations of human operator and incomplete information about the environment. Compared to traditional robotic systems, control of robotic manipulators over networks has significant advantages; for example, increased flexibility and ease of maintenance. With the utilization of scattering variables, this research demonstrates that transmitting scattering variables over delayed communications can stabilize an otherwise unstable system. An architecture utilizing delayed position feedback in conjunction with scattering variables is developed for the case of time-varying communication delays. The proposed control architecture improves tracking performance and stabilizes robotic manipulators with input-output communication delays. The aforementioned control algorithms and architectures for networked robotic systems are validated via numerical examples and experiments

An Adaptive Regulator for Space Teleoperation System in Task Space

The problem of the gravity information which can not be obtained in advance for bilateral teleoperation is studied. In outer space exploration, the gravity term changes with the position changing of the slave manipulator. So it is necessary to design an adaptive regulator controller to compensate for the unknown gravity signal. Moreover, to get a more accurate position tracking performance, the controller is designed in the task space instead of the joint space. Additionally, the time delay considered in this paper is not only time varying but also unsymmetrical. Finally, simulations are presented to show the effectiveness of the proposed approach

Passivity-Based adaptive bilateral teleoperation control for uncertain manipulators without jerk measurements

In this work, we consider the bilateral teleoperation problem of cooperative robotic systems in a Single-Master Multi-Slave (SM/MS) configuration, which is able to perform load transportation tasks in the presence of parametric uncertainty in the robot kinematic and dynamic models. The teleoperation architecture is based on the two-layer approach placed in a hierarchical structure, whose top and bottom layers are responsible for ensuring the transparency and stability properties respectively. The load transportation problem is tackled by using the formation control approach wherein the desired translational velocity and interaction force are provided to the master robot by the user, while the object is manipulated with a bounded constant force by the slave robots. Firstly, we develop an adaptive kinematic-based control scheme based on a composite adaptation law to solve the cooperative control problem for robots with uncertain kinematics. Secondly, the dynamic adaptive control for cooperative robots is implemented by means of a cascade control strategy, which does not require the measurement of the time derivative of force (which requires jerk measurements). The combination of the Lyapunov stability theory and the passivity formalism are used to establish the stability and convergence property of the closed-loop control system. Simulations and experimental results illustrate the performance and feasibility of the proposed control scheme.No presente trabalho, considera-se o problema de teleoperação bilateral de um sistema robótico cooperativo do tipo single-master e multiple-slaves (SM/MS) capaz de realizar tarefas de transporte de carga na presença de incertezas paramétricas no modelo cinemático e dinâmico dos robôs. A arquitetura de teleoperação está baseada na abordagem de duas camadas em estrutura hierárquica, onde as camadas superior e inferior são responsáveis por assegurar as propriedades de transparência e estabilidade respectivamente. O problema de transporte de carga é formulado usando a abordagem de controle de formação onde a velocidade de translação desejada e a força de interação são fornecidas ao robô mestre pelo operador, enquanto o objeto é manipulado pelos robôs escravos com uma força constante limitada. Primeiramente, desenvolve-se um esquema de controle adaptativo cinemático baseado em uma lei de adaptação composta para solucionar o problema de controle cooperativo de robôs com cinemática incerta. Em seguida, o controle adaptativo dinâmico de robôs cooperativos é implementado por meio de uma estratégia de controle em cascata, que não requer a medição da derivada da força (o qual requer a derivada da aceleração ou jerk). A teoria de estabilidade de Lyapunov e o formalismo de passividade são usados para estabelecer as propriedades de estabilidade e a convergência do sistema de controle em malha-fechada. Resultados de simulações numéricas ilustram o desempenho e viabilidade da estratégia de controle proposta

A Framework for Stable Robot-Environment Interaction Based on the Generalized Scattering Transformation

This thesis deals with development and experimental evaluation of control algorithms for stabilization of robot-environment interaction based on the conic systems formalism and scattering transformation techniques. A framework for stable robot-environment interaction is presented and evaluated on a real physical system. The proposed algorithm fundamentally generalizes the conventional passivity-based approaches to the coupled stability problem. In particular, it allows for stabilization of not necessarily passive robot-environment interaction. The framework is based on the recently developed non-planar conic systems formalism and generalized scattering-based stabilization methods. A comprehensive theoretical background on the scattering transformation techniques, planar and non-planar conic systems is presented. The dynamics of the robot are estimated using data-driven techniques, which allows the equations for the dynamics of a robot to be obtained in an explicit form. The generalized scattering transformation is used in combination with the Lyapunov-based adaptive trajectory tracking control. It is shown that the original interconnected system is not stable due to its non-passive nature; however, the application of the proposed stabilization algorithm allows stability to be achieved, without affecting the robot’s trajectory tracking performance in free space