Multi-robot cooperative platform : a task-oriented teleoperation paradigm

This thesis proposes the study and development of a teleoperation system based on multi-robot cooperation under the task oriented teleoperation paradigm: Multi-Robot Cooperative Paradigm, MRCP. In standard teleoperation, the operator uses the master devices to control the remote slave robot arms. These arms reproduce the desired movements and perform the task. With the developed work, the operator can virtually manipulate an object. MRCP automatically generates the arms orders to perform the task. The operator does not have to solve situations arising from possible restrictions that the slave arms may have. The research carried out is therefore aimed at improving the accuracy teleoperation tasks in complex environments, particularly in the field of robot assisted minimally invasive surgery. This field requires patient safety and the workspace entails many restrictions to teleoperation. MRCP can be defined as a platform composed of several robots that cooperate automatically to perform a teleoperated task, creating a robotic system with increased capacity (workspace volume, accessibility, dexterity ...). The cooperation is based on transferring the task between robots when necessary to enable a smooth task execution. The MRCP control evaluates the suitability of each robot to continue with the ongoing task and the optimal time to execute a task transfer between the current selected robot and the best candidate to continue with the task. From the operator¿s point of view, MRCP provides an interface that enables the teleoperation though the task-oriented paradigm: operator orders are translated into task actions instead of robot orders. This thesis is structured as follows: The first part is dedicated to review the current solutions in the teleoperation of complex tasks and compare them with those proposed in this research. The second part of the thesis presents and reviews in depth the different evaluation criteria to determine the suitability of each robot to continue with the execution of a task, considering the configuration of the robots and emphasizing the criterion of dexterity and manipulability. The study reviews the different required control algorithms to enable the task oriented telemanipulation. This proposed teleoperation paradigm is transparent to the operator. Then, the Thesis presents and analyses several experimental results using MRCP in the field of minimally invasive surgery. These experiments study the effectiveness of MRCP in various tasks requiring the cooperation of two hands. A type task is used: a suture using minimally invasive surgery technique. The analysis is done in terms of execution time, economy of movement, quality and patient safety (potential damage produced by undesired interaction between the tools and the vital tissues of the patient). The final part of the thesis proposes the implementation of different virtual aids and restrictions (guided teleoperation based on haptic visual and audio feedback, protection of restricted workspace regions, etc.) using the task oriented teleoperation paradigm. A framework is defined for implementing and applying a basic set of virtual aids and constraints within the framework of a virtual simulator for laparoscopic abdominal surgery. The set of experiments have allowed to validate the developed work. The study revealed the influence of virtual aids in the learning process of laparoscopic techniques. It has also demonstrated the improvement of learning curves, which paves the way for its implementation as a methodology for training new surgeons.Aquesta tesi doctoral proposa l'estudi i desenvolupament d'un sistema de teleoperació basat en la cooperació multi-robot sota el paradigma de la teleoperació orientada a tasca: Multi-Robot Cooperative Paradigm, MRCP. En la teleoperació clàssica, l'operador utilitza els telecomandaments perquè els braços robots reprodueixin els seus moviments i es realitzi la tasca desitjada. Amb el treball realitzat, l'operador pot manipular virtualment un objecte i és mitjançant el MRCP que s'adjudica a cada braç les ordres necessàries per realitzar la tasca, sense que l'operador hagi de resoldre les situacions derivades de possibles restriccions que puguin tenir els braços executors. La recerca desenvolupada està doncs orientada a millorar la teleoperació en tasques de precisió en entorns complexos i, en particular, en el camp de la cirurgia mínimament invasiva assistida per robots. Aquest camp imposa condicions de seguretat del pacient i l'espai de treball comporta moltes restriccions a la teleoperació. MRCP es pot definir com a una plataforma formada per diversos robots que cooperen de forma automàtica per dur a terme una tasca teleoperada, generant un sistema robòtic amb capacitats augmentades (volums de treball, accessibilitat, destresa,...). La cooperació es basa en transferir la tasca entre robots a partir de determinar quin és aquell que és més adequat per continuar amb la seva execució i el moment òptim per realitzar la transferència de la tasca entre el robot actiu i el millor candidat a continuar-la. Des del punt de vista de l'operari, MRCP ofereix una interfície de teleoperació que permet la realització de la teleoperació mitjançant el paradigma d'ordres orientades a la tasca: les ordres es tradueixen en accions sobre la tasca en comptes d'estar dirigides als robots. Aquesta tesi està estructurada de la següent manera: Primerament es fa una revisió de l'estat actual de les diverses solucions desenvolupades actualment en el camp de la teleoperació de tasques complexes, comparant-les amb les proposades en aquest treball de recerca. En el segon bloc de la tesi es presenten i s'analitzen a fons els diversos criteris per determinar la capacitat de cada robot per continuar l'execució d'una tasca, segons la configuració del conjunt de robots i fent especial èmfasi en el criteri de destresa i manipulabilitat. Seguint aquest estudi, es presenten els diferents processos de control emprats per tal d'assolir la telemanipulació orientada a tasca de forma transparent a l'operari. Seguidament es presenten diversos resultats experimentals aplicant MRCP al camp de la cirurgia mínimament invasiva. En aquests experiments s'estudia l'eficàcia de MRCP en diverses tasques que requereixen de la cooperació de dues mans. S'ha escollit una tasca tipus: sutura amb tècnica de cirurgia mínimament invasiva. L'anàlisi es fa en termes de temps d'execució, economia de moviment, qualitat i seguretat del pacient (potencials danys causats per la interacció no desitjada entre les eines i els teixits vitals del pacient). Finalment s'ha estudiat l'ús de diferents ajudes i restriccions virtuals (guiat de la teleoperació via retorn hàptic, visual o auditiu, protecció de regions de l'espai de treball, etc) dins el paradigma de teleoperació orientada a tasca. S'ha definint un marc d'aplicació base i implementant un conjunt de restriccions virtuals dins el marc d'un simulador de cirurgia laparoscòpia abdominal. El conjunt d'experiments realitzats han permès validar el treball realitzat. Aquest estudi ha permès determinar la influencia de les ajudes virtuals en el procés d'aprenentatge de les tècniques laparoscòpiques. S'ha evidenciat una millora en les corbes d'aprenentatge i obre el camí a la seva implantació com a metodologia d'entrenament de nous cirurgians.Postprint (published version

Manipulability in trajectory tracking for constrained redundant manipulators via sequential quadratic programming

Trajectory tracking methods for constrained redundant manipulators are presented in this thesis, where the end-effector of a redundant serial manipulator has to track a desired trajectory while some points on its kinematic chain satisfy one or more constraints. In addition, two manipulability indexes are taken into account in order to optimize the trajectory. The first index is defined in terms of the geometric Jacobian of the manipulator in the constrained configuration. The second index is based on the constrained Jacobian, which maps velocities from joint space to task space, taking into account the holonomic constraints. Three methods for solving the trajectory tracking problem are discussed. The first two, kinematic control (KC) and quadratic programming (QP), are widely discussed in literature. The third, sequential quadratic programming (SQP), is a new approach, unlike KC or QP, has as advantages (despite some shortcomings) not explicitly depend on pseudoinverse Jacobian, derivative from the desired trajectory and linearization of indexes or constraints. A discussion of these three methods is presented in terms of tracking error, constraint violation, singularity distance, among others through experiments performed on a Baxter collaborative robot.Métodos de rastreamento de trajetória para manipuladores redundantes restritos são apresentados nesta tese, onde o efetuador de um manipulador serial redundante tem que rastrear uma trajetória desejada enquanto alguns pontos em sua cadeia cinemática satisfazem uma ou mais restrições. Além disso, dois índices de manipulabilidade são levados em consideração a fim de otimizar a trajetória para evitar singularidades. O primeiro índice é definido em função do jacobiano geométrico do manipulador na configuração restrita. O segundo índice é baseado no Jacobiano restrito, o qual mapeia velocidades no espaço das juntas para a espaço da tarefa, levando em conta as restrições holonômicas. Três métodos para resolver o problema de rastreamento de trajetória são discutidos. Os dois primeiros, controle cinemático e programação quadrática (QP), são amplamente discutidos na literatura. O terceiro, programação quadrática sequencial (SQP), é uma nova abordagem, diferentemente do controle cinemático ou QP, tem como vantagens (apesar de algumas deficiências) não depender explicitamente da pseudo-inversa de jacobianos, derivadas da trajetória desejada e linearização de índices ou restrições. Uma discussão desses três métodos é apresentada em termos de erro de rastreamento, violação da restrição, distância de singularidades, entre outros através de experimentos realizados em um robô colaborativo Baxter

Design, analysis and kinematic control of highly redundant serial robotic arms

The use of robotic manipulators in industry has grown in the last decades to improve and speed up industrial processes. Industrial manipulators started to be investigated for machining tasks since they can cover larger workspaces, increasing the range of achievable operations and improving flexibility. The company Nimbl’Bot developed a new mechanism, or module, to build stiffer flexible serial modular robots for machining applications. This manipulator is a kinematic redundant robot with 21 degrees of freedom. This thesis thoroughly analysis the Nimbl’Bot robot features and is divided into three main topics. The first topic regards using a task priority kinematic redundancy resolution algorithm for the Nimbl’Bot robot tracking trajectory while optimizing its kinetostatic performances. The second topic is the kinematic redundant robot design optimization with respect to a desired application and its kinetostatic performance. For the third topic, a new workspace determination algorithm is proposed for kinematic redundant manipulators. Several simulation tests are proposed and tested on some Nimbl’Bot robot designs for each subjects

Redundancy Optimization Strategy for Hands-On Robotic Surgery

During hands-on cooperative surgery, the use of a redundant robot allows to address encumbrance issues in the Operating Room (OR), which can occur due to the presence of large medical instrumentation, such as the surgical microscope. This work presents a new Null Space Optimization (NSO) strategy to constraint the position of the manipulator’s elbow within predefined range of motions, according to the spatial requirements of the specific procedure, also taking into account the physical joint limits of the robotic assistant. The proposed strategy was applied to the 7 degrees of freedom (dof) lightweight robot LWR4+. The performance of the NSO was compared to two state-of-the-art null space optimization strategies, i.e. damped posture and fixed optimal posture, over a pool of three non-expert users in both strict (20deg) and negligible (100deg) angular encumbrance limitations. The NSO strategy was proved versatile in providing wide elbow mobility together with safe distance from relevant continuity null space boundaries, guaranteeing smooth guidance trajectories. Future works would be performed in order to evaluate the potential feasibility of NSO in a real surgical scenario