Control of Cooperative Haptics-Enabled Teleoperation Systems with Application to Minimally Invasive Surgery

Robot-Assisted Minimally Invasive Surgical (RAMIS) systems frequently have a structure of cooperative teleoperator systems where multiple master-slave pairs are used to collaboratively execute a task. Although multiple studies indicate that haptic feedback improves the realism of tool-tissue interaction to the surgeon and leads to better performance for surgical procedures, current telesurgical systems typically do not provide force feedback, mainly because of the inherent stability issues. The research presented in this thesis is directed towards the development of control algorithms for force reflecting cooperative surgical teleoperator systems with improved stability and transparency characteristics. In the case of cooperative force reflecting teleoperation over networks, conventional passivity based approaches may have limited applicability due to potentially non-passive slave-slave interactions and irregular communication delays imposed by the network. In this thesis, an alternative small gain framework for the design of cooperative network-based force reflecting teleoperator systems is developed. Using the small gain framework, control algorithms for cooperative force-reflecting teleoperator systems are designed that guarantee stability in the presence of multiple network-induced communication constraints. Furthermore, the design conservatism typically associated with the small-gain approach is eliminated by using the Projection-Based Force Reflection (PBFR) algorithms. Stability results are established for networked cooperative teleoperator systems under different types of force reflection algorithms in the presence of irregular communication delays. The proposed control approach is consequently implemented on a dual-arm (two masters/two slaves) robotic MIS testbed. The testbed consists of two Haptic Wand devices as masters and two PA10-7C robots as the slave manipulators equipped with da Vinci laparoscopic surgical instruments. The performance of the proposed control approach is evaluated in three different cooperative surgical tasks, which are knot tightening, pegboard transfer, and object manipulation. The experimental results obtained indicate that the PBFR algorithms demonstrate statistically significant performance improvement in comparison with the conventional direct force reflection algorithms. One possible shortcoming of using PBFR algorithms is that implementation of these algorithms may lead to attenuation of the high-frequency component of the contact force which is important, in particular, for haptic perception of stiff surfaces. In this thesis, a solution to this problem is proposed which is based on the idea of separating the different frequency bands in the force reflection signal and consequently applying the projection-based principle to the low-frequency component, while reflecting the high-frequency component directly. The experimental results demonstrate that substantial improvement in transient fidelity of the force feedback is achieved using the proposed method without negative effects on the stability of the system

Consensus control in robot networks and cooperative teleoperation : an operational space approach

An interesting approach in cooperative control is to design distributed control strategies which use only local information so that a multi-agent system achieves specified behaviors. A basic behavior in cooperative control is the consensus. Given a multi-agent system, like a multiple robot network, it is said that the agents reach a consensus if the state of each agent converges to a common state. Examples of cooperative tasks in which consensus algorithms are employed include formation control, flocking theory, rendezvous problems and synchronization. These cooperative tasks have several possible applications, like: transportation systems (intelligent highways, air-traffic control); military systems (formation flight, surveillance, reconnaissance, cooperative attack and rendezvous) and mobile sensor networks (space-based interferometers, environmental sampling). The solution to the consensus problems involves the design of control algorithms such that the agents can reach an agreement on their states. There are two main problems that are studied in consensus, the leader-follower consensus and the leaderless consensus. In the leader-follower consensus problem, there exists a leader that specifies the state for the whole group while in a leaderless consensus problem, there is not a priori reference state. The main goal of this thesis is the design of operational space controllers that solve the leader-follower and the leaderless consensus problems in networks composed of multiple heterogeneous robots. Furthermore, this document proposes novel operational space control schemes for bilateral teleoperation systems. In both scenarios, different conditions are studied, such as the absence of robot velocity measurements, constant and variable time-delays in the robot's interconnection, and uncertainty in the robot's physical parameters. Most of the previous consensus control algorithms, only work with the position or orientation but not with both. On the contrary, this dissertation deals with the entire pose of the robots that contains both the position and the orientation. Moreover, in order to render a singularity-free description of the orientation, the unit-quaternions are employed. The dissertation provides a rigorous stability analysis of the control algorithms and presents simulations and experiments that validate the effectiveness of the proposed controllers.Un enfoque interesante en el control cooperativo es el diseño de estrategias de control distribuido que requieran sólo información local para que un sistema multi-agente logre comportamientos específicos. Un comportamiento básico del control cooperativo es el consenso. Dado un sistema multi-agente, como una red de múltiples robots, se dice que los agentes llegan a un consenso si el estado de cada agente converge a un estado común. Algunos ejemplos de tareas cooperativas en las que los algoritmos de consenso son utilizados son los siguientes: el control de la formación, flocking, rendezvous y sincronización. Estas tareas cooperativas tienen varias aplicaciones posibles, como: sistemas de transporte (carreteras inteligentes , control de tráfico aéreo); sistemas militares (vuelo en formación, vigilancia, reconocimiento, ataque cooperativo) y redes de sensores móviles (interferómetros en el espacio, el muestreo del ambiente). La solución a los problemas de consenso implica el diseño de algoritmos de control de tal manera que los agentes pueden llegar a un acuerdo sobre sus estados. Hay dos problemas principales que se estudian en el consenso, el consenso líder-seguidor y el consenso sin líder. En el problema de consenso líder-seguidor, existe un líder que especifica el estado de todo el grupo, mientras que en un problema de consenso sin líder, no hay ningún estado de referencia definido a priori. El objetivo principal de esta tesis es el diseño de controladores en el espacio operacional que resuelvan los problemas de consenso líder-seguidor y sin líder en redes compuestas de múltiples robots heterogéneos. Además, este documento propone novedosos esquemas de control en el espacio operacional para sistemas de teleoperación bilateral. En ambos escenarios, se estudian diferentes condiciones, tales como la ausencia de medidas de velocidad de los robots, retardos constantes y variables en la interconexión de los robots y la incertidumbre en los parámetros físicos de los robots. La mayoría de los anteriores algoritmos de control que resuelven el consenso, sólo trabajan con la posición o la orientación, pero no con ambos. Por el contrario, esta tesis doctoral se ocupa de toda la pose de los robots que contiene tanto la posición y la orientación. Por otra parte, a fin de usar una representación de la orientación libre de singularidades, se emplean los cuaterniones unitarios. Esta tesis doctoral proporciona un análisis riguroso de la estabilidad de los algoritmos de control y presenta simulaciones y experimentos que validan la eficacia de los controladores propuesto

Technology for the Future: In-Space Technology Experiments Program, part 2

The purpose of the Office of Aeronautics and Space Technology (OAST) In-Space Technology Experiments Program In-STEP 1988 Workshop was to identify and prioritize technologies that are critical for future national space programs and require validation in the space environment, and review current NASA (In-Reach) and industry/ university (Out-Reach) experiments. A prioritized list of the critical technology needs was developed for the following eight disciplines: structures; environmental effects; power systems and thermal management; fluid management and propulsion systems; automation and robotics; sensors and information systems; in-space systems; and humans in space. This is part two of two parts and contains the critical technology presentations for the eight theme elements and a summary listing of critical space technology needs for each theme

NASA space station automation: AI-based technology review

Research and Development projects in automation for the Space Station are discussed. Artificial Intelligence (AI) based automation technologies are planned to enhance crew safety through reduced need for EVA, increase crew productivity through the reduction of routine operations, increase space station autonomy, and augment space station capability through the use of teleoperation and robotics. AI technology will also be developed for the servicing of satellites at the Space Station, system monitoring and diagnosis, space manufacturing, and the assembly of large space structures

Limited Information Shared Control and its Applications to Large Vehicle Manipulators

Diese Dissertation beschäftigt sich mit der kooperativen Regelung einer mobilen Arbeitsmaschine, welche aus einem Nutzfahrzeug und einem oder mehreren hydraulischen Manipulatoren besteht. Solche Maschinen werden für Aufgaben in der Straßenunterhaltungsaufgaben eingesetzt. Die Arbeitsumgebung des Manipulators ist unstrukturiert, was die Bestimmung einer Referenztrajektorie erschwert oder unmöglich macht. Deshalb wird in dieser Arbeit ein Ansatz vorgeschlagen, welcher nur das Fahrzeug automatisiert, während der menschliche Bediener ein Teil des Systems bleibt und den Manipulator steuert. Eine solche Teilautomatisierung des Gesamtsystems führt zu einer speziellen Klasse von Mensch-Maschine-Interaktionen, welche in der Literatur noch nicht untersucht wurde: Eine kooperative Regelung zwischen zwei Teilsystemen, bei der die Automatisierung keine Informationen von dem vom Menschen gesteuerten Teilsystem hat. Deswegen wird in dieser Arbeit ein systematischer Ansatz der kooperativen Regelung mit begrenzter Information vorgestellt, der den menschlichen Bediener unterstützen kann, ohne die Referenzen oder die Systemzustände des Manipulators zu messen. Außerdem wird ein systematisches Entwurfskonzept für die kooperative Regelung mit begrenzter Information vorgestellt. Für diese Entwurfsmethode werden zwei neue Unterklassen der sogenannten Potenzialspiele eingeführt, die eine systematische Berechnung der Parameter der entwickelten kooperativen Regelung ohne manuelle Abstimmung ermöglichen. Schließlich wird das entwickelte Konzept der kooperativen Regelung am Beispiel einer großen mobilen Arbeitsmaschine angewandt, um seine Vorteile zu ermitteln und zu bewerten. Nach der Analyse in Simulationen wird die praktische Anwendbarkeit der Methode in drei Experimenten mit menschlichen Probanden an einem Simulator untersucht. Die Ergebnisse zeigen die Überlegenheit des entwickelten kooperativen Regelungskonzepts gegenüber der manuellen Steuerung und der nicht-kooperativen Steuerung hinsichtlich sowohl der objektiven Performanz als auch der subjektiven Bewertung der Probanden. Somit zeigt diese Dissertation, dass die kooperative Regelung mobiler Arbeitsmaschinen mit den entwickelten theoretischen Konzepten sowohl hilfreich als auch praktisch anwendbar ist

Task space consensus in networks of heterogeneous and uncertain robotic systems with variable time-delays

This work deals with the leader-follower and the leaderless consensus problems in networks of multiple robot manipulators. The robots are non-identical, kinematically different (heterogeneous), and their physical parameters are uncertain. The main contribution of this work is a novel controller that solves the two consensus problems, in the task space, with the following features: it estimates the kinematic and the dynamic physical parameters; it is robust to interconnecting variable-time delays; it employs the singularity-free unit-quaternions to represent the orientation; and, using energy-like functions, the controller synthesis follows a constructive procedure. Simulations using a network with four heterogeneous manipulators illustrate the performance of the proposed controller.Peer ReviewedPostprint (author's final draft

Human–Robot Role Arbitration via Differential Game Theory

The industry needs controllers that allow smooth and natural physical Human-Robot Interaction (pHRI) to make production scenarios more flexible and user-friendly. Within this context, particularly interesting is Role Arbitration, which is the mechanism that assigns the role of the leader to either the human or the robot. This paper investigates Game-Theory (GT) to model pHRI, and specifically, Cooperative Game Theory (CGT) and Non-Cooperative Game Theory (NCGT) are considered. This work proposes a possible solution to the Role Arbitration problem and defines a Role Arbitration framework based on differential game theory to allow pHRI. The proposed method can allow trajectory deformation according to human will, avoiding reaching dangerous situations such as collisions with environmental features, robot joints and workspace limits, and possibly safety constraints. Three sets of experiments are proposed to evaluate different situations and compared with two other standard methods for pHRI, the Impedance Control, and the Manual Guidance. Experiments show that with our Role Arbitration method, different situations can be handled safely and smoothly with a low human effort. In particular, the performances of the IMP and MG vary according to the task. In some cases, MG performs well, and IMP does not. In some others, IMP performs excellently, and MG does not. The proposed Role Arbitration controller performs well in all the cases, showing its superiority and generality. The proposed method generally requires less force and ensures better accuracy in performing all tasks than standard controllers. Note to Practitioners—This work presents a method that allows role arbitration for physical Human-Robot Interaction, motivated by the need to adjust the role of leader/follower in a shared task according to the specific phase of the task or the knowledge of one of the two agents. This method suits applications such as object co-transportation, which requires final precise positioning but allows some trajectory deformation on the fly. It can also handle situations where the carried obstacle occludes human sight, and the robot helps the human to avoid possible environmental obstacles and position the objects at the target pose precisely. Currently, this method does not consider external contact, which is likely to arise in many situations. Future studies will investigate the modeling and detection of external contacts to include them in the interaction models this work addresses