Multi-robot cooperative platform : a task-oriented teleoperation paradigm

This thesis proposes the study and development of a teleoperation system based on multi-robot cooperation under the task oriented teleoperation paradigm: Multi-Robot Cooperative Paradigm, MRCP. In standard teleoperation, the operator uses the master devices to control the remote slave robot arms. These arms reproduce the desired movements and perform the task. With the developed work, the operator can virtually manipulate an object. MRCP automatically generates the arms orders to perform the task. The operator does not have to solve situations arising from possible restrictions that the slave arms may have. The research carried out is therefore aimed at improving the accuracy teleoperation tasks in complex environments, particularly in the field of robot assisted minimally invasive surgery. This field requires patient safety and the workspace entails many restrictions to teleoperation. MRCP can be defined as a platform composed of several robots that cooperate automatically to perform a teleoperated task, creating a robotic system with increased capacity (workspace volume, accessibility, dexterity ...). The cooperation is based on transferring the task between robots when necessary to enable a smooth task execution. The MRCP control evaluates the suitability of each robot to continue with the ongoing task and the optimal time to execute a task transfer between the current selected robot and the best candidate to continue with the task. From the operator¿s point of view, MRCP provides an interface that enables the teleoperation though the task-oriented paradigm: operator orders are translated into task actions instead of robot orders. This thesis is structured as follows: The first part is dedicated to review the current solutions in the teleoperation of complex tasks and compare them with those proposed in this research. The second part of the thesis presents and reviews in depth the different evaluation criteria to determine the suitability of each robot to continue with the execution of a task, considering the configuration of the robots and emphasizing the criterion of dexterity and manipulability. The study reviews the different required control algorithms to enable the task oriented telemanipulation. This proposed teleoperation paradigm is transparent to the operator. Then, the Thesis presents and analyses several experimental results using MRCP in the field of minimally invasive surgery. These experiments study the effectiveness of MRCP in various tasks requiring the cooperation of two hands. A type task is used: a suture using minimally invasive surgery technique. The analysis is done in terms of execution time, economy of movement, quality and patient safety (potential damage produced by undesired interaction between the tools and the vital tissues of the patient). The final part of the thesis proposes the implementation of different virtual aids and restrictions (guided teleoperation based on haptic visual and audio feedback, protection of restricted workspace regions, etc.) using the task oriented teleoperation paradigm. A framework is defined for implementing and applying a basic set of virtual aids and constraints within the framework of a virtual simulator for laparoscopic abdominal surgery. The set of experiments have allowed to validate the developed work. The study revealed the influence of virtual aids in the learning process of laparoscopic techniques. It has also demonstrated the improvement of learning curves, which paves the way for its implementation as a methodology for training new surgeons.Aquesta tesi doctoral proposa l'estudi i desenvolupament d'un sistema de teleoperació basat en la cooperació multi-robot sota el paradigma de la teleoperació orientada a tasca: Multi-Robot Cooperative Paradigm, MRCP. En la teleoperació clàssica, l'operador utilitza els telecomandaments perquè els braços robots reprodueixin els seus moviments i es realitzi la tasca desitjada. Amb el treball realitzat, l'operador pot manipular virtualment un objecte i és mitjançant el MRCP que s'adjudica a cada braç les ordres necessàries per realitzar la tasca, sense que l'operador hagi de resoldre les situacions derivades de possibles restriccions que puguin tenir els braços executors. La recerca desenvolupada està doncs orientada a millorar la teleoperació en tasques de precisió en entorns complexos i, en particular, en el camp de la cirurgia mínimament invasiva assistida per robots. Aquest camp imposa condicions de seguretat del pacient i l'espai de treball comporta moltes restriccions a la teleoperació. MRCP es pot definir com a una plataforma formada per diversos robots que cooperen de forma automàtica per dur a terme una tasca teleoperada, generant un sistema robòtic amb capacitats augmentades (volums de treball, accessibilitat, destresa,...). La cooperació es basa en transferir la tasca entre robots a partir de determinar quin és aquell que és més adequat per continuar amb la seva execució i el moment òptim per realitzar la transferència de la tasca entre el robot actiu i el millor candidat a continuar-la. Des del punt de vista de l'operari, MRCP ofereix una interfície de teleoperació que permet la realització de la teleoperació mitjançant el paradigma d'ordres orientades a la tasca: les ordres es tradueixen en accions sobre la tasca en comptes d'estar dirigides als robots. Aquesta tesi està estructurada de la següent manera: Primerament es fa una revisió de l'estat actual de les diverses solucions desenvolupades actualment en el camp de la teleoperació de tasques complexes, comparant-les amb les proposades en aquest treball de recerca. En el segon bloc de la tesi es presenten i s'analitzen a fons els diversos criteris per determinar la capacitat de cada robot per continuar l'execució d'una tasca, segons la configuració del conjunt de robots i fent especial èmfasi en el criteri de destresa i manipulabilitat. Seguint aquest estudi, es presenten els diferents processos de control emprats per tal d'assolir la telemanipulació orientada a tasca de forma transparent a l'operari. Seguidament es presenten diversos resultats experimentals aplicant MRCP al camp de la cirurgia mínimament invasiva. En aquests experiments s'estudia l'eficàcia de MRCP en diverses tasques que requereixen de la cooperació de dues mans. S'ha escollit una tasca tipus: sutura amb tècnica de cirurgia mínimament invasiva. L'anàlisi es fa en termes de temps d'execució, economia de moviment, qualitat i seguretat del pacient (potencials danys causats per la interacció no desitjada entre les eines i els teixits vitals del pacient). Finalment s'ha estudiat l'ús de diferents ajudes i restriccions virtuals (guiat de la teleoperació via retorn hàptic, visual o auditiu, protecció de regions de l'espai de treball, etc) dins el paradigma de teleoperació orientada a tasca. S'ha definint un marc d'aplicació base i implementant un conjunt de restriccions virtuals dins el marc d'un simulador de cirurgia laparoscòpia abdominal. El conjunt d'experiments realitzats han permès validar el treball realitzat. Aquest estudi ha permès determinar la influencia de les ajudes virtuals en el procés d'aprenentatge de les tècniques laparoscòpiques. S'ha evidenciat una millora en les corbes d'aprenentatge i obre el camí a la seva implantació com a metodologia d'entrenament de nous cirurgians.Postprint (published version

Solving robotic kinematic problems : singularities and inverse kinematics

Kinematics is a branch of classical mechanics that describes the motion of points, bodies, and systems of bodies without considering the forces that cause such motion. For serial robot manipulators, kinematics consists of describing the open chain geometry as well as the position, velocity and/or acceleration of each one of its components. Rigid serial robot manipulators are designed as a sequence of rigid bodies, called links, connected by motor-actuated pairs, called joints, that provide relative motion between consecutive links. Two kinematic problems of special relevance for serial robots are: - Singularities: are the configurations where the robot loses at least one degree of freedom (DOF). This is equivalent to: (a) The robot cannot translate or rotate its end-effector in at least one direction. (b) Unbounded joint velocities are required to generate finite linear and angular velocities. Either if it is real-time teleoperation or off-line path planning, singularities must be addressed to make the robot exhibit a good performance for a given task. The objective is not only to identify the singularities and their associated singular directions but to design strategies to avoid or handle them. - Inverse kinematic problem: Given a particular position and orientation of the end-effector, also known as the end-effector pose, the inverse kinematics consists of finding the configurations that provide such desired pose. The importance of the inverse kinematics relies on its role in the programming and control of serial robots. Besides, since for each given pose the inverse kinematics has up to sixteen different solutions, the objective is to find a closed-form method for solving this problem, since closed-form methods allow to obtain all the solutions in a compact form. The main goal of the Ph.D. dissertation is to contribute to the solution of both problems. In particular, with respect to the singularity problem, a novel scheme for the identification of the singularities and their associated singular directions is introduced. Moreover, geometric algebra is used to simplify such identification and to provide a distance function in the configuration space of the robot that allows the definition of algorithms for avoiding them. With respect to the inverse kinematics, redundant robots are reduced to non-redundant ones by selecting a set of joints, denoted redundant joints, and by parameterizing their joint variables. This selection is made through a workspace analysis which also provides an upper bound for the number of different closed-form solutions. Once these joints have been identified, several closed-form methods developed for non-redundant manipulators can be applied to obtain the analytical expressions of all the solutions. One of these methods is a novel strategy developed using again the conformal model of the spatial geometric algebra. To sum up, the Ph.D dissertation provides a rigorous analysis of the two above-mentioned kinematic problems as well as novel strategies for solving them. To illustrate the different results introduced in the Ph.D. memory, examples are given at the end of each of its chapters.La cinemática es una rama de la mecánica clásica que describe el movimiento de puntos, cuerpos y sistemas de cuerpos sin considerar las fuerzas que causan dicho movimiento. Para un robot manipulador serie, la cinemática consiste en la descripción de su geometría, su posición, velocidad y/o aceleración. Los robots manipuladores serie están diseñados como una secuencia de elementos estructurales rígidos, llamados eslabones, conectados entres si por articulaciones actuadas, que permiten el movimiento relativo entre pares de eslabones consecutivos. Dos problemas cinemáticos de especial relevancia para robots serie son: - Singularidades: son aquellas configuraciones donde el robot pierde al menos un grado de libertad (GDL). Esto equivale a: (a) El robot no puede trasladar ni rotar su elemento terminal en al menos una dirección. (b) Se requieren velocidades articulares no acotadas para generar velocidades lineales y angulares finitas. Ya sea en un sistema teleoperado en tiempo real o planificando una trayectoria, las singularidades deben manejarse para que el robot muestre un rendimiento óptimo mientras realiza una tarea. El objetivo no es solo identificar las singularidades y sus direcciones singulares asociadas, sino diseñar estrategias para evitarlas o manejarlas. - Problema de la cinemática inversa: dada una posición y orientación del elemento terminal (también conocida como la pose del elemento terminal), la cinemática inversa consiste en obtener las configuraciones asociadas a dicha pose. La importancia de la cinemática inversa se basa en el papel que juega en la programación y el control de robots serie. Además, dado que para cada pose la cinemática inversa tiene hasta dieciséis soluciones diferentes, el objetivo es encontrar un método cerrado para resolver este problema, ya que los métodos cerrados permiten obtener todas las soluciones en una forma compacta. El objetivo principal de la tesis doctoral es contribuir a la solución de ambos problemas. En particular, con respecto al problema de las singularidades, se presenta un nuevo método para su identificación basado en el álgebra geométrica. Además, el álgebra geométrica permite definir una distancia en el espacio de configuraciones del robot que permite la definición de distintos algoritmos para evitar las configuraciones singulares. Con respecto a la cinemática inversa, los robots redundantes se reducen a robots no-redundantes mediante la selección de un conjunto de articulaciones, las articulaciones redundantes, para después parametrizar sus variables articulares. Esta selección se realiza a través de un análisis de espacio de trabajo que también proporciona un límite superior para el número de diferentes soluciones en forma cerrada. Una vez las articulaciones redundantes han sido identificadas, varios métodos en forma cerrada desarrollados para robots no-redundantes pueden aplicarse a fin de obtener las expresiones analíticas de todas las soluciones. Uno de dichos métodos es una nueva estrategia desarrollada usando el modelo conforme del álgebra geométrica tridimensional. En resumen, la tesis doctoral proporciona un análisis riguroso de los dos problemas cinemáticos mencionados anteriormente, así como nuevas estrategias para resolverlos. Para ilustrar los diferentes resultados presentados en la tesis, la memoria contiene varios ejemplos al final de cada uno de sus capítulos

Compliant control of Uni/ Multi- robotic arms with dynamical systems

Accomplishment of many interactive tasks hinges on the compliance of humans. Humans demonstrate an impressive capability of complying their behavior and more particularly their motions with the environment in everyday life. In humans, compliance emerges from different facets. For example, many daily activities involve reaching for grabbing tasks, where compliance appears in a form of coordination. Humans comply their handsâ motions with each other and with that of the object not only to establish a stable contact and to control the impact force but also to overcome sensorimotor imprecisions. Even though compliance has been studied from different aspects in humans, it is primarily related to impedance control in robotics. In this thesis, we leverage the properties of autonomous dynamical systems (DS) for immediate re-planning and introduce active complaint motion generators for controlling robots in three different scenarios, where compliance does not necessarily mean impedance and hence it is not directly related to control in the force/velocity domain. In the first part of the thesis, we propose an active compliant strategy for catching objects in flight, which is less sensitive to the timely control of the interception. The soft catching strategy consists in having the robot following the object for a short period of time. This leaves more time for the fingers to close on the object at the interception and offers more robustness than a âhardâ catching method in which the hand waits for the object at the chosen interception point. We show theoretically that the resulting DS will intercept the object at the intercept point, at the right time with the desired velocity direction. Stability and convergence of the approach are assessed through Lyapunov stability theory. In the second part, we propose a unified compliant control architecture for coordinately reaching for grabbing a moving object by a multi-arm robotic system. Due to the complexity of the task and of the system, each arm complies not only with the objectâs motion but also with the motion of other arms, in both task and joint spaces. At the task-space level, we propose a unified dynamical system that endows the multi-arm system with both synchronous and asynchronous behaviors and with the capability of smoothly transitioning between the two modes. At the joint space level, the compliance between the arms is achieved by introducing a centralized inverse kinematics (IK) solver under self-collision avoidance constraints; formulated as a quadratic programming problem (QP) and solved in real-time. In the last part, we propose a compliant dynamical system for stably transitioning from free motions to contacts. In this part, by modulating the robot's velocity in three regions, we show theoretically and empirically that the robot can (I) stably touch the contact surface (II) at a desired location, and (III) leave the surface or stop on the surface at a desired point

MUSME 2011 4 th International Symposium on Multibody Systems and Mechatronics

El libro de actas recoge las aportaciones de los autores a través de los correspondientes artículos a la Dinámica de Sistemas Multicuerpo y la Mecatrónica (Musme). Estas disciplinas se han convertido en una importante herramienta para diseñar máquinas, analizar prototipos virtuales y realizar análisis CAD sobre complejos sistemas mecánicos articulados multicuerpo. La dinámica de sistemas multicuerpo comprende un gran número de aspectos que incluyen la mecánica, dinámica estructural, matemáticas aplicadas, métodos de control, ciencia de los ordenadores y mecatrónica. Los artículos recogidos en el libro de actas están relacionados con alguno de los siguientes tópicos del congreso: Análisis y síntesis de mecanismos ; Diseño de algoritmos para sistemas mecatrónicos ; Procedimientos de simulación y resultados ; Prototipos y rendimiento ; Robots y micromáquinas ; Validaciones experimentales ; Teoría de simulación mecatrónica ; Sistemas mecatrónicos ; Control de sistemas mecatrónicosUniversitat Politècnica de València (2011). MUSME 2011 4 th International Symposium on Multibody Systems and Mechatronics. Editorial Universitat Politècnica de València. http://hdl.handle.net/10251/13224Archivo delegad

Industrial Robot Collision Handling in Harsh Environments

The focus in this thesis is on robot collision handling systems, mainly collision detection and collision avoidance for industrial robots operating in harsh environments (e.g. potentially explosive atmospheres found in the oil and gas sector). Collision detection should prevent the robot from colliding and therefore avoid a potential accident. Collision avoidance builds on the concept of collision detection and aims at enabling the robot to find a collision free path circumventing the obstacle and leading to the goal position. The work has been done in collaboration with ABB Process Automation Division with focus on applications in oil and gas. One of the challenges in this work has been to contribute to safer use of industrial robots in potentially explosive environments. One of the main ideas is that a robot should be able to work together with a human as a robotic co-worker on for instance an oil rig. The robot should then perform heavy lifting and precision tasks, while the operator controls the steps of the operation through typically a hand-held interface. In such situations, when the human works alongside with the robot in potentially explosive environments, it is important that the robot has a way of handling collisions. The work in this thesis presents solutions for collision detection in paper A, B and C, thereafter solutions for collision avoidance are presented in paper D and E. Paper A approaches the problem of collision avoidance comparing an expert system and a hidden markov model (HMM) approach. An industrial robot equipped with a laser scanner is used to gather environment data on arbitrary set of points in the work cell. The two methods are used to detect obstacles within the work cell and shows a different set of strengths. The expert system shows an advantage in algorithm performance and the HMM method shows its strength in its ease of learning models of the environment. Paper B builds upon Paper A by incorporating a CAD model of the environment. The CAD model allows for a very fast setup of the expert system where no manual map creation is needed. The HMM can be trained based on the CAD model, which addresses the previous dependency on real sensor data for training purposes. Paper C compares two different world-model representation techniques, namely octrees and point clouds using both a graphics processing unit (GPU) and a central processing unit (CPU). The GPU showed its strength for uncompressed point clouds and high resolution point cloud models. However, if the resolution gets low enough, the CPU starts to outperform the GPU. This shows that parallel problems containing large data sets are suitable for GPU processing, but smaller parallel problems are still handled better by the CPU. In paper D, real-time collision avoidance is studied for a lightweight industrial robot using a development platform controller. A Microsoft Kinect sensor is used for capturing 3D depth data of the environment. The environment data is used together with an artificial potential fields method for generating virtual forces used for obstacle avoidance. The forces are projected onto the end-effector, preventing collision with the environment while moving towards the goal. Forces are also projected on to the elbow of the 7-Degree of freedom robot, which allows for nullspace movement. The algorithms for manipulating the sensor data and calculating virtual forces were developed for the GPU, this resulted in fast algorithms and is the enabling factor for real-time collision avoidance. Finally, paper E builds on the work in paper D by providing a framework for using the algorithms on a standard industrial controller and robot with minimal modifications. Further, algorithms were specifically developed for the robot controller to handle reactive movement. In addition, a full collision avoidance system for an end-user application which is very simple to implement is presented. The work described in this thesis presents solutions for collision detection and collision avoidance for safer use of robots. The work is also a step towards making businesses more competitive by enabling easy integration of collision handling for industrial robots