Geometry-aware Manipulability Learning, Tracking and Transfer

Body posture influences human and robots performance in manipulation tasks, as appropriate poses facilitate motion or force exertion along different axes. In robotics, manipulability ellipsoids arise as a powerful descriptor to analyze, control and design the robot dexterity as a function of the articulatory joint configuration. This descriptor can be designed according to different task requirements, such as tracking a desired position or apply a specific force. In this context, this paper presents a novel \emph{manipulability transfer} framework, a method that allows robots to learn and reproduce manipulability ellipsoids from expert demonstrations. The proposed learning scheme is built on a tensor-based formulation of a Gaussian mixture model that takes into account that manipulability ellipsoids lie on the manifold of symmetric positive definite matrices. Learning is coupled with a geometry-aware tracking controller allowing robots to follow a desired profile of manipulability ellipsoids. Extensive evaluations in simulation with redundant manipulators, a robotic hand and humanoids agents, as well as an experiment with two real dual-arm systems validate the feasibility of the approach.Comment: Accepted for publication in the Intl. Journal of Robotics Research (IJRR). Website: https://sites.google.com/view/manipulability. Code: https://github.com/NoemieJaquier/Manipulability. 24 pages, 20 figures, 3 tables, 4 appendice

Multi-robot cooperative platform : a task-oriented teleoperation paradigm

This thesis proposes the study and development of a teleoperation system based on multi-robot cooperation under the task oriented teleoperation paradigm: Multi-Robot Cooperative Paradigm, MRCP. In standard teleoperation, the operator uses the master devices to control the remote slave robot arms. These arms reproduce the desired movements and perform the task. With the developed work, the operator can virtually manipulate an object. MRCP automatically generates the arms orders to perform the task. The operator does not have to solve situations arising from possible restrictions that the slave arms may have. The research carried out is therefore aimed at improving the accuracy teleoperation tasks in complex environments, particularly in the field of robot assisted minimally invasive surgery. This field requires patient safety and the workspace entails many restrictions to teleoperation. MRCP can be defined as a platform composed of several robots that cooperate automatically to perform a teleoperated task, creating a robotic system with increased capacity (workspace volume, accessibility, dexterity ...). The cooperation is based on transferring the task between robots when necessary to enable a smooth task execution. The MRCP control evaluates the suitability of each robot to continue with the ongoing task and the optimal time to execute a task transfer between the current selected robot and the best candidate to continue with the task. From the operator¿s point of view, MRCP provides an interface that enables the teleoperation though the task-oriented paradigm: operator orders are translated into task actions instead of robot orders. This thesis is structured as follows: The first part is dedicated to review the current solutions in the teleoperation of complex tasks and compare them with those proposed in this research. The second part of the thesis presents and reviews in depth the different evaluation criteria to determine the suitability of each robot to continue with the execution of a task, considering the configuration of the robots and emphasizing the criterion of dexterity and manipulability. The study reviews the different required control algorithms to enable the task oriented telemanipulation. This proposed teleoperation paradigm is transparent to the operator. Then, the Thesis presents and analyses several experimental results using MRCP in the field of minimally invasive surgery. These experiments study the effectiveness of MRCP in various tasks requiring the cooperation of two hands. A type task is used: a suture using minimally invasive surgery technique. The analysis is done in terms of execution time, economy of movement, quality and patient safety (potential damage produced by undesired interaction between the tools and the vital tissues of the patient). The final part of the thesis proposes the implementation of different virtual aids and restrictions (guided teleoperation based on haptic visual and audio feedback, protection of restricted workspace regions, etc.) using the task oriented teleoperation paradigm. A framework is defined for implementing and applying a basic set of virtual aids and constraints within the framework of a virtual simulator for laparoscopic abdominal surgery. The set of experiments have allowed to validate the developed work. The study revealed the influence of virtual aids in the learning process of laparoscopic techniques. It has also demonstrated the improvement of learning curves, which paves the way for its implementation as a methodology for training new surgeons.Aquesta tesi doctoral proposa l'estudi i desenvolupament d'un sistema de teleoperació basat en la cooperació multi-robot sota el paradigma de la teleoperació orientada a tasca: Multi-Robot Cooperative Paradigm, MRCP. En la teleoperació clàssica, l'operador utilitza els telecomandaments perquè els braços robots reprodueixin els seus moviments i es realitzi la tasca desitjada. Amb el treball realitzat, l'operador pot manipular virtualment un objecte i és mitjançant el MRCP que s'adjudica a cada braç les ordres necessàries per realitzar la tasca, sense que l'operador hagi de resoldre les situacions derivades de possibles restriccions que puguin tenir els braços executors. La recerca desenvolupada està doncs orientada a millorar la teleoperació en tasques de precisió en entorns complexos i, en particular, en el camp de la cirurgia mínimament invasiva assistida per robots. Aquest camp imposa condicions de seguretat del pacient i l'espai de treball comporta moltes restriccions a la teleoperació. MRCP es pot definir com a una plataforma formada per diversos robots que cooperen de forma automàtica per dur a terme una tasca teleoperada, generant un sistema robòtic amb capacitats augmentades (volums de treball, accessibilitat, destresa,...). La cooperació es basa en transferir la tasca entre robots a partir de determinar quin és aquell que és més adequat per continuar amb la seva execució i el moment òptim per realitzar la transferència de la tasca entre el robot actiu i el millor candidat a continuar-la. Des del punt de vista de l'operari, MRCP ofereix una interfície de teleoperació que permet la realització de la teleoperació mitjançant el paradigma d'ordres orientades a la tasca: les ordres es tradueixen en accions sobre la tasca en comptes d'estar dirigides als robots. Aquesta tesi està estructurada de la següent manera: Primerament es fa una revisió de l'estat actual de les diverses solucions desenvolupades actualment en el camp de la teleoperació de tasques complexes, comparant-les amb les proposades en aquest treball de recerca. En el segon bloc de la tesi es presenten i s'analitzen a fons els diversos criteris per determinar la capacitat de cada robot per continuar l'execució d'una tasca, segons la configuració del conjunt de robots i fent especial èmfasi en el criteri de destresa i manipulabilitat. Seguint aquest estudi, es presenten els diferents processos de control emprats per tal d'assolir la telemanipulació orientada a tasca de forma transparent a l'operari. Seguidament es presenten diversos resultats experimentals aplicant MRCP al camp de la cirurgia mínimament invasiva. En aquests experiments s'estudia l'eficàcia de MRCP en diverses tasques que requereixen de la cooperació de dues mans. S'ha escollit una tasca tipus: sutura amb tècnica de cirurgia mínimament invasiva. L'anàlisi es fa en termes de temps d'execució, economia de moviment, qualitat i seguretat del pacient (potencials danys causats per la interacció no desitjada entre les eines i els teixits vitals del pacient). Finalment s'ha estudiat l'ús de diferents ajudes i restriccions virtuals (guiat de la teleoperació via retorn hàptic, visual o auditiu, protecció de regions de l'espai de treball, etc) dins el paradigma de teleoperació orientada a tasca. S'ha definint un marc d'aplicació base i implementant un conjunt de restriccions virtuals dins el marc d'un simulador de cirurgia laparoscòpia abdominal. El conjunt d'experiments realitzats han permès validar el treball realitzat. Aquest estudi ha permès determinar la influencia de les ajudes virtuals en el procés d'aprenentatge de les tècniques laparoscòpiques. S'ha evidenciat una millora en les corbes d'aprenentatge i obre el camí a la seva implantació com a metodologia d'entrenament de nous cirurgians.Postprint (published version

Automatic motion of manipulator using sampling based motion planning algorithms - application in service robotics

The thesis presents new approaches for autonomous motion execution of a robotic arm. The calculation of the motion is called motion planning and requires the computation of robot arm's path. The text covers the calculation of the path and several algorithms have been therefore implemented and tested in several real scenarios. The work focuses on sampling based planners, which means that the path is created by connecting explicitly random generated points in the free space. The algorithms can be divided into three categories: those that are working in configuration space(C-Space)(C- Space is the set of all possible joint angles of a robotic arm) , the mixed approaches using both Cartesian and C-Space and those that are using only the Cartesian space. Although Cartesian space seems more appropriate, due to dimensionality, this work illustrates that the C-Space planners can achieve comparable or better results. Initially an enhanced approach for efficient collision detection in C-Space, used by the planners, is presented. Afterwards the N dimensional cuboid region, notated as Rq, is defined. The Rq configures the C-Space so that the sampling is done close to a selected, called center, cell. The approach is enhanced by the decomposition of the Cartesian space into cells. A cell is selected appropriately if: (a) is closer to the target position and (b) lies inside the constraints. Inverse kinematics(IK) are applied to calculate a centre configuration used later by the Rq. The CellBiRRT is proposed and combines all the features. Continuously mixed approaches that do not require goal configuration or an analytic solution of IK are presented. Rq regions as well as Cells are also integrated in these approaches. A Cartesian sampling based planner using quaternions for linear interpolation is also proposed and tested. The common feature of the so far algorithms is the feasibility which is normally against the optimality. Therefore an additional part of this work deals with the optimality of the path. An enhanced approach of CellBiRRT, called CellBiRRT*, is developed and promises to compute shorter paths in a reasonable time. An on-line method using both CellBiRRT and CellBiRRT* is proposed where the path of the robot arm is improved and recalculated even if sudden changes in the environment are detected. Benchmarking with the state of the art algorithms show the good performance of the proposed approaches. The good performance makes the algorithms suitable for real time applications. In this work several applications are described: Manipulative skills, an approach for an semi-autonomous control of the robot arm and a motion planning library. The motion planning library provides the necessary interface for easy use and further development of the motion planning algorithms. It can be used as the part connecting the manipulative skill designing and the motion of a robotic arm

Grasping for the Task:Human Principles for Robot Hands

The significant advances made in the design and construction of anthropomorphic robot hands, endow them with prehensile abilities reaching that of humans. However, using these powerful hands with the same level of expertise that humans display is a big challenge for robots. Traditional approaches use finger-tip (precision) or enveloping (power) methods to generate the best force closure grasps. However, this ignores the variety of prehensile postures available to the hand and also the larger context of arm action. This thesis explores a paradigm for grasp formation based on generating oppositional pressure within the hand, which has been proposed as a functional basis for grasping in humans (MacKenzie and Iberall, 1994). A set of opposition primitives encapsulates the hand's ability to generate oppositional forces. The oppositional intention encoded in a primitive serves as a guide to match the hand to the object, quantify its functional ability and relate this to the arm. In this thesis we leverage the properties of opposition primitives to both interpret grasps formed by humans and to construct grasps for a robot considering the larger context of arm action. In the first part of the thesis we examine the hypothesis that hand representation schemes based on opposition are correlated with hand function. We propose hand-parameters describing oppositional intention and compare these with commonly used methods such as joint angles, joint synergies and shape features. We expect that opposition-based parameterizations, which take an interaction-based perspective of a grasp, are able to discriminate between grasps that are similar in shape but different in functional intent. We test this hypothesis using qualitative assessment of precision and power capabilities found in existing grasp taxonomies. The next part of the thesis presents a general method to recognize oppositional intention manifested in human grasp demonstrations. A data glove instrumented with tactile sensors is used to provide the raw information regarding hand configuration and interaction force. For a grasp combining several cooperating oppositional intentions, hand surfaces can be simultaneously involved in multiple oppositional roles. We characterize the low-level interactions between different surfaces of the hand based on captured interaction force and reconstructed hand surface geometry. This is subsequently used to separate out and prioritize multiple and possibly overlapping oppositional intentions present in the demonstrated grasp. We evaluate our method on several human subjects across a wide range of hand functions. The last part of the thesis applies the properties encoded in opposition primitives to optimize task performance of the arm, for tasks where the arm assumes the dominant role. For these tasks, choosing the strongest power grasp available (from a force-closure sense) may constrain the arm to a sub-optimal configuration. Weaker grasp components impose fewer constraints on the hand, and can therefore explore a wider region of the object relative pose space. We take advantage of this to find the good arm configurations from a task perspective. The final hand-arm configuration is obtained by trading of overall robustness in the grasp with ability of the arm to perform the task. We validate our approach, using the tasks of cutting, hammering, screw-driving and opening a bottle-cap, for both human and robotic hand-arm systems

Kontextsensitive Körperregulierung für redundante Roboter

In the past few decades the classical 6 degrees of freedom manipulators' dominance has been challenged by the rise of 7 degrees of freedom redundant robots. Similarly, with increased availability of humanoid robots in academic research, roboticists suddenly have access to highly dexterous platforms with multiple kinematic chains capable of undertaking multiple tasks simultaneously. The execution of lower-priority tasks, however, are often done in task/scenario specific fashion. Consequently, these systems are not scalable and slight changes in the application often implies re-engineering the entire control system and deployment which impedes the development process over time. This thesis introduces an alternative systematic method of addressing the secondary tasks and redundancy resolution called, context aware body regulation. Contexts consist of one or multiple tasks, however, unlike the conventional definitions, the tasks within a context are not rigidly defined and maintain some level of abstraction. For instance, following a particular trajectory constitutes a concrete task while performing a Cartesian motion with the end-effector represents an abstraction of the same task and is more appropriate for context formulation. Furthermore, contexts are often made up of multiple abstract tasks that collectively describe a reoccurring situation. Body regulation is an umbrella term for a collection of schemes for addressing the robots' redundancy when a particular context occurs. Context aware body regulation offers several advantages over traditional methods. Most notably among them are reusability, scalability and composability of contexts and body regulation schemes. These three fundamental concerns are realized theoretically by in-depth study and through mathematical analysis of contexts and regulation strategies; and are practically implemented by a component based software architecture that complements the theoretical aspects. The findings of the thesis are applicable to any redundant manipulator and humanoids, and allow them to be used in real world applications. Proposed methodology presents an alternative approach for the control of robots and offers a new perspective for future deployment of robotic solutions.Im Verlauf der letzten Jahrzehnte wich der Einfluss klassischer Roboterarme mit 6 Freiheitsgraden zunehmend denen neuer und vielfältigerer Manipulatoren mit 7 Gelenken. Ebenso stehen der Forschung mit den neuartigen Humanoiden inzwischen auch hoch-redundante Roboterplattformen mit mehreren kinematischen Ketten zur Verfügung. Diese überaus flexiblen und komplexen Roboter-Kinematiken ermöglichen generell das gleichzeitige Verfolgen mehrerer priorisierter Bewegungsaufgaben. Die Steuerung der weniger wichtigen Aufgaben erfolgt jedoch oft in anwendungsspezifischer Art und Weise, welche die Skalierung der Regelung zu generellen Kontexten verhindert. Selbst kleine Änderungen in der Anwendung bewirken oft schon, dass große Teile der Robotersteuerung überarbeitet werden müssen, was wiederum den gesamten Entwicklungsprozess behindert. Diese Dissertation stellt eine alternative, systematische Methode vor um die Redundanz neuer komplexer Robotersysteme zu bewältigen und vielfältige, priorisierte Bewegungsaufgaben parallel zu steuern: Die so genannte kontextsensitive Körperregulierung. Darin bestehen Kontexte aus einer oder mehreren Bewegungsaufgaben. Anders als in konventionellen Anwendungen sind die Aufgaben nicht fest definiert und beinhalten eine gewisse Abstraktion. Beispielsweise stellt das Folgen einer bestimmten Trajektorie eine sehr konkrete Bewegungsaufgabe dar, während die Ausführung einer Kartesischen Bewegung mit dem Endeffektor eine Abstraktion darstellt, die für die Kontextformulierung besser geeignet ist. Kontexte setzen sich oft aus mehreren solcher abstrakten Aufgaben zusammen und beschreiben kollektiv eine sich wiederholende Situation. Durch die Verwendung der kontextsensitiven Körperregulierung ergeben sich vielfältige Vorteile gegenüber traditionellen Methoden: Wiederverwendbarkeit, Skalierbarkeit, sowie Komponierbarkeit von Konzepten. Diese drei fundamentalen Eigenschaften werden in der vorliegenden Arbeit theoretisch mittels gründlicher mathematischer Analyse aufgezeigt und praktisch mittels einer auf Komponenten basierenden Softwarearchitektur realisiert. Die Ergebnisse dieser Dissertation lassen sich auf beliebige redundante Manipulatoren oder humanoide Roboter anwenden und befähigen diese damit zur realen Anwendung außerhalb des Labors. Die hier vorgestellte Methode zur Regelung von Robotern stellt damit eine neue Perspektive für die zukünftige Entwicklung von robotischen Lösungen dar

Parallel Manipulators

In recent years, parallel kinematics mechanisms have attracted a lot of attention from the academic and industrial communities due to potential applications not only as robot manipulators but also as machine tools. Generally, the criteria used to compare the performance of traditional serial robots and parallel robots are the workspace, the ratio between the payload and the robot mass, accuracy, and dynamic behaviour. In addition to the reduced coupling effect between joints, parallel robots bring the benefits of much higher payload-robot mass ratios, superior accuracy and greater stiffness; qualities which lead to better dynamic performance. The main drawback with parallel robots is the relatively small workspace. A great deal of research on parallel robots has been carried out worldwide, and a large number of parallel mechanism systems have been built for various applications, such as remote handling, machine tools, medical robots, simulators, micro-robots, and humanoid robots. This book opens a window to exceptional research and development work on parallel mechanisms contributed by authors from around the world. Through this window the reader can get a good view of current parallel robot research and applications

Estimation of objects’ inertial parameters, and their usage in robot grasping and manipulation

The subject of this thesis is the estimation of an object's inertial parameters by a robotic arm, and the exploitation of those parameters in the design of efficient manipulation criteria. The inertial parameters of objects describe the resistance of the object to an applied force, and dictate its motion. Research has shown that humans intuitively exploit them for their everyday manipulations. As humans are very capable of performing efficient manipulations, it is natural that robots should use the inertial parameters as well. Additionally, as the inertial parameters are not straightforward to calculate, there is the need for development of methods that can estimate them online. This thesis focuses on two directions, developing novel methods so that robots can accurately estimate the inertial parameters of an object, as well as developing manipulation criteria that can make robot task completion more efficient. The relevant literature is gathered, categorised and analytically described, and the innovation gaps are identified. The thesis offers novel research solutions on the problem of estimation of the inertial parameters with minimal robot interaction. The paradigm is shifted from the existing literature, and a data-driven estimation algorithm is introduced, that achieves accurate results with both simulated and real data. Additionally, the presented research is offering novel manipulation criteria that are affected by the object's inertial parameters. The results suggest that knowledge of the inertial parameters can make the robot tasks more power-efficient and safe to their surroundings. The core methodology is shown to be versatile to the robotic platform. Though most experiments are performed on a terrestrial robot, a numerical example is also shown for a space robot. The results of the thesis suggest that the developed methods can be used in various environments, with the most suitable being extreme environments where accuracy, efficiency and autonomy is required