The Shape of Damping: Optimizing Damping Coefficients to Improve Transparency on Bilateral Telemanipulation

This thesis presents a novel optimization-based passivity control algorithm for hapticenabled bilateral teleoperation systems involving multiple degrees of freedom. In particular, in the context of energy-bounding control, the contribution focuses on the implementation of a passivity layer for an existing time-domain scheme, ensuring optimal transparency of the interaction along subsets of the environment space which are preponderant for the given task, while preserving the energy bounds required for passivity. The involved optimization problem is convex and amenable to real-time implementation. The effectiveness of the proposed design is validated via an experiment performed on a virtual teleoperated environment. The interplay between transparency and stability is a critical aspect in haptic-enabled bilateral teleoperation control. While it is important to present the user with the true impedance of the environment, destabilizing factors such as time delays, stiff environments, and a relaxed grasp on the master device may compromise the stability and safety of the system. Passivity has been exploited as one of the the main tools for providing sufficient conditions for stable teleoperation in several controller design approaches, such as the scattering algorithm, timedomain passivity control, energy bounding algorithm, and passive set position modulation. In this work it is presented an innovative energy-based approach, which builds upon existing time-domain passivity controllers, improving and extending their effectiveness and functionality. The set of damping coefficients are prioritized in each degree of freedom, the resulting transparency presents a realistic force feedback in comparison to the other directions. Thus, the prioritization takes effect using a quadratic programming algorithm to find the optimal values for the damping. Finally, the energy tanks approach on passivity control is a solution used to ensure stability in a system for robotics bilateral manipulation. The bilateral telemanipulation must maintain the principle of passivity in all moments to preserve the system\u2019s stability. This work presents a brief introduction to haptic devices as a master component on the telemanipulation chain; the end effector in the slave side is a representation of an interactive object within an environment having a force sensor as feedback signal. The whole interface is designed into a cross-platform framework named ROS, where the user interacts with the system. Experimental results are presented

Haptic Guidance for Extended Range Telepresence

A novel navigation assistance for extended range telepresence is presented. The haptic information from the target environment is augmented with guidance commands to assist the user in reaching desired goals in the arbitrarily large target environment from the spatially restricted user environment. Furthermore, a semi-mobile haptic interface was developed, one whose lightweight design and setup configuration atop the user provide for an absolutely safe operation and high force display quality

Monitoring companion for industrial robotic processes

For system integrators, optimizing complex industrial robotic applications (e.g. robotised welding) is a difficult and time-consuming task. This procedure is rendered tedious and often very hard to achieve when the operator cannot access the robotic system once in operation, perhaps because the installation is far away or because of the operational environment. In these circumstances, as an alternative to physically visiting the installation site, the system integrator may rely on additional nearby sensors to remotely acquire the necessary process information. While it is hard to completely replace this trial and error approach, it is possible to provide a way to gather process information more effectively that can be used in several robotic installations.This thesis investigates the use of a "monitoring robot" in addition to the task robot(s) that belong to the industrial process to be optimized. The monitoring robot can be equipped with several different sensors and can be moved into close proximity of any installed task robot so that it can be used to collect information from that process during and/or after the operation without interfering. The thesis reviews related work in the industry and in the field of teleoperation to identify the most important challenges in remote monitoring and teleoperation. From the background investigation it is clear that two very important issues are: i) the nature of the teleoperator’s interface and; ii) the efficiency of the shared control between the human operator and the monitoring system. In order to investigate these two issues efficiently it was necessary to create experimental scenarios that operate independently from any application scenario, so an abstract problem domain is created. This way the monitoring system's control and interface can be evaluated in a context that presents challenges that are typical of a remote monitoring task but are not application domain specific. Therefore the validity of the proposed approach can be assessed from a generic and, therefore, more powerful and widely applicable perspective. The monitoring framework developed in this thesis is described, both in the shared control design choices based on virtual fixtures (VF) and the implementation in a 3D visualization environment. The monitoring system developed is evaluated with a usability study with user participants. The usability study aims at assessing the system's performance along with its acceptance and ease of use in a static monitoring task, accompanied by user\hyp{}filled TLX questionnaires. Since future work will apply this system in real robotic welding scenarios, this thesis finally reports some preliminary work in such an application

Computer-Aided Bilateral Teleoperation of Manipulators

Haptic bilateral teleoperation is often a challenging and mentally demanding job for the operators of robot control systems. It is especially difficult in cases such as the remote maintenance of the ITER divertor region. The difficulty of the ITER divertor maintenance hails from a multitude of reasons: the residual radiation level of the ITER reactor during a shutdown is too high for any human access, the maintenance tunnels of the divertor are confined, the operators have to operate heavy loads in delicate tasks, and only a limited number of radiation tolerant cameras are available for providing video feedback. In addition, most of the maintenance work cannot be automated because of the dynamic nature and complexity of the tasks. Haptic shared control systems can be used for reducing the amount of mental and physical workload perceived by the operators of remote maintenance systems. To reduce the workload, a haptic shared control system assists the operators by generating virtual forces based on the virtual models of the teleoperation environment and sensor data from the slave manipulator. The generated assistance forces are laid over the force feedback signals from the teleoperation environment. The assisting forces can e.g. guide the operators along optimal paths and prevent collisions in the teleoperation environment. In addition to the reduction of the operator workload, teleoperation tasks also become faster and safer with haptic shared control. This thesis investigates the implementation techniques and theory of haptic bilateral teleoperation and shared control systems. Based on the theoretical analysis, an experimental haptic shared control system, called the Computer Assisted Teleoperation (CAT) was developed. The intention of CAT is to assist the remote handling (RH) system operators of the Divertor Test Platform 2 (DTP2) in ITER remote maintenance research. The effectiveness of CAT is evaluated in a teleoperation experiment performed with a 6 DOF Water Hydraulic MANipulator (WHMAN) developed for the ITER divertor maintenance. The results of the experiment gives directive indication that the CAT system improves the execution times of a bilateral teleoperation task and simultaneously reduces the workload perceived by the operators of the system

A learning-based shared control architecture for interactive task execution

Shared control is a key technology for various robotic applications in which a robotic system and a human operator are meant to collaborate efficiently. In order to achieve efficient task execution in shared control, it is essential to predict the desired behavior for a given situation or context to simplify the control task for the human operator. To do this prediction, we use Learning from Demonstration (LfD), which is a popular approach for transferring human skills to robots. We encode the demonstrated behavior as trajectory distributions and generalize the learned distributions to new situations. The goal of this paper is to present a shared control framework that uses learned expert distributions to gain more autonomy. Our approach controls the balance between the controller’s autonomy and the human preference based on the distributions of the demonstrated trajectories. Moreover, the learned distributions are autonomously refined from collaborative task executions, resulting in a master-slave system with increasing autonomy that requires less user input with an increasing number of task executions. We experimentally validated that our shared control approach enables efficient task executions. Moreover, the conducted experiments demonstrated that the developed system improves its performances through interactive task executions with our shared control

Model Driven Robotic Assistance for Human-Robot Collaboration

While robots routinely perform complex assembly tasks in highly structured factory environments, it is challenging to apply completely autonomous robotic systems in less structured manipulation tasks, such as surgery and machine assembly/repair, due to the limitations of machine intelligence, sensor data interpretation and environment modeling. A practical, yet effective approach to accomplish these tasks is through human-robot collaboration, in which the human operator and the robot form a partnership and complement each other in performing a complex task. We recognize that humans excel at determining task goals and recognizing constraints, if given sufficient feedback about the interaction between the tool (e.g., end-effector of the robot) and the environment. Robots are precise, unaffected by fatigue and able to work in environments not suitable for humans. We hypothesize that by providing the operator with adequate information about the task, through visual and force (haptic) feedback, the operator can: (1) define the task model, in terms of task goals and virtual fixture constraints through an interactive, or immersive augmented reality interface, and (2) have the robot actively assist the operator to enhance the execution time, quality and precision of the tasks. We validate our approaches through the implementations of both cooperative (i.e., hands-on) control and telerobotic systems, for image-guided robotic neurosurgery and telerobotic manipulation tasks for satellite servicing under significant time delay

Adaptive assistance-based on decision-making models for telerobotics systems

Esta tesis propone una nueva estrategia de asistencia háptica en la interacción humano-robot. Dado que el humano es el elemento fundamental del sistema, es necesario proponer estrategias que se adapten a su comportamiento, además de garantizar un mejoramiento del desempeño en la tarea. El inconveniente surge cuando se requiere asistir al operador en mejorar el desempeño de la tarea y permitir al usuario total control de la tarea cuando sea necesario, desviándose del plan original con el objetivo de abordar situaciones imprevistas. Desde una perspectiva enfocada en el control, se debe resolver el compromiso existente entre proveer un alto nivel de asistencia para mejorar el desempeño de la tarea y un bajo nivel de asistencia para permitir al operador desviarse del plan pre-programado (original). Se propone entonces incorporar en la asistencia háptica un mecanismo de toma de decisiones usado por los humanos en tareas básicas de decisión entre dos alternativas. Este mecanismo de decisión se incorporar como el método de selección de parámetros en un controlador adaptativo de estructura fija (i.e. un controlador de impedencia/admitancia de parámetros variables). Los resultados experimentales demuestran que el modelo de toma de decisión, i.e. el modelo drift-diffusion modificado, permite asignar el nivel de autonomía de una forma que resulta intuitiva para el usuario y mejora el desempeño en la tarea. Además la estrategia de asistencia basada en modelos de toma de decisión proporciona un mecanismo de sintonizaci ón para resolver diferentes requerimientos de la tarea, lo cual es importante en entornos no estructurados. Dado el número de parámetros configurables presentes en la asistencia, la etapa experimental expone la función de cada uno de estos parámetros. Se realizó un experimento con usuarios en un entorno de teleoperación donde se evalúa estadísticamente el comportamiento de la asistencia en entornos parcialmente estructurados y se compara con la asistencia proporcionada por un experto humano, la cual puede ser considerada como la asistencia adaptativa nominal.Abstract. This thesis proposes a novel haptic assistance method for human-robot interaction. Since the human is the main element of the system, it is necessary to propose strategies that adapt the robot’s dynamics to the human behavior, while guaranteeing an improvement in task performance. The main issue arises when the assistance must chose between assisting the operator to improve task performance or allowing the user to have full authority over the task when necessary, allowing him/her to deviate from the original plan in order to handle unforeseen situations. From a control systems’ perspective, the assistance has to solve the trade-off between high assistance levels to improve task performance and low assistance level to allow the user to deviate from the preprogrammed (original) plan. The main results of this work incorporate into the haptic assistance a human-like decision-making mechanism used in two-alternative force choice tasks. Our experimental results show that the drift-diffusion, which is a decision-making model proposed in the cognitive area, allocates control authority in a way that is intuitive for the user. The the proposed assistance provides a tunable (decision-making) mechanism that is capable of fulfilling different task requirements, which is an important when dealing with unstructured environments. Given the number of configurable parameters in the assistance mechanism, the experimental procedure exposes the effects of changing them. A user study in a telerobotic scenario was performed to evaluate the behavior of the assistance in a partially structured environment; the proposed assistance is compared to the assistance provided by a human expert, which may be considered as the nominal adaptive assistance.Doctorad

A Method for Digital Representation of Human Movements

In this work we present a method to produce a model of human motion based on an expansion in functions series. The model is thought to reproduce the learned movements generalizing them to different conditions. We will show, with an example, how the proposed method is capable to produce the model from a reduced set of examples preserving the relevant features of the demonstrations while guaranteeing constraints at boundaries

Intuitive, iterative and assisted virtual guides programming for human-robot comanipulation

Pendant très longtemps, l'automatisation a été assujettie à l'usage de robots industriels traditionnels placés dans des cages et programmés pour répéter des tâches plus ou moins complexes au maximum de leur vitesse et de leur précision. Cette automatisation, dite rigide, possède deux inconvénients majeurs : elle est chronophage dû aux contraintes contextuelles applicatives et proscrit la présence humaine. Il existe désormais une nouvelle génération de robots avec des systèmes moins encombrants, peu coûteux et plus flexibles. De par leur structure et leurs modes de fonctionnement ils sont intrinsèquement sûrs ce qui leurs permettent de travailler main dans la main avec les humains. Dans ces nouveaux espaces de travail collaboratifs, l'homme peut être inclus dans la boucle comme un agent décisionnel actif. En tant qu'instructeur ou collaborateur il peut influencer le processus décisionnel du robot : on parle de robots collaboratifs (ou cobots). Dans ce nouveau contexte, nous faisons usage de guides virtuels. Ils permettent aux cobots de soulager les efforts physiques et la charge cognitive des opérateurs. Cependant, la définition d'un guide virtuel nécessite souvent une expertise et une modélisation précise de la tâche. Cela restreint leur utilité aux scénarios à contraintes fixes. Pour palier ce problème et améliorer la flexibilité de la programmation du guide virtuel, cette thèse présente une nouvelle approche par démonstration : nous faisons usage de l'apprentissage kinesthésique de façon itérative et construisons le guide virtuel avec une spline 6D. Grâce à cette approche, l'opérateur peut modifier itérativement les guides tout en gardant leur assistance. Cela permet de rendre le processus plus intuitif et naturel ainsi que de réduire la pénibilité. La modification locale d'un guide virtuel en trajectoire est possible par interaction physique avec le robot. L'utilisateur peut déplacer un point clé cartésien ou modifier une portion entière du guide avec une nouvelle démonstration partielle. Nous avons également étendu notre approche aux guides virtuels 6D, où les splines en déplacement sont définies via une interpolation Akima (pour la translation) et une 'interpolation quadratique des quaternions (pour l'orientation). L'opérateur peut initialement définir un guide virtuel en trajectoire, puis utiliser l'assistance en translation pour ne se concentrer que sur la démonstration de l'orientation. Nous avons appliqué notre approche dans deux scénarios industriels utilisant un cobot. Nous avons ainsi démontré l'intérêt de notre méthode qui améliore le confort de l'opérateur lors de la comanipulation.For a very long time, automation was driven by the use of traditional industrial robots placed in cages, programmed to repeat more or less complex tasks at their highest speed and with maximum accuracy. This robot-oriented solution is heavily dependent on hard automation which requires pre-specified fixtures and time consuming programming, hindering robots from becoming flexible and versatile tools. These robots have evolved towards a new generation of small, inexpensive, inherently safe and flexible systems that work hand in hand with humans. In these new collaborative workspaces the human can be included in the loop as an active agent. As a teacher and as a co-worker he can influence the decision-making process of the robot. In this context, virtual guides are an important tool used to assist the human worker by reducing physical effort and cognitive overload during tasks accomplishment. However, the construction of virtual guides often requires expert knowledge and modeling of the task. These limitations restrict the usefulness of virtual guides to scenarios with unchanging constraints. To overcome these challenges and enhance the flexibility of virtual guides programming, this thesis presents a novel approach that allows the worker to create virtual guides by demonstration through an iterative method based on kinesthetic teaching and displacement splines. Thanks to this approach, the worker is able to iteratively modify the guides while being assisted by them, making the process more intuitive and natural while reducing its painfulness. Our approach allows local refinement of virtual guiding trajectories through physical interaction with the robots. We can modify a specific cartesian keypoint of the guide or re- demonstrate a portion. We also extended our approach to 6D virtual guides, where displacement splines are defined via Akima interpolation (for translation) and quadratic interpolation of quaternions (for orientation). The worker can initially define a virtual guiding trajectory and then use the assistance in translation to only concentrate on defining the orientation along the path. We demonstrated that these innovations provide a novel and intuitive solution to increase the human's comfort during human-robot comanipulation in two industrial scenarios with a collaborative robot (cobot)

Force control and nonlinear master-slave force profile to manage an admittance type multi-fingered haptic user interface

Natural movements and force feedback are important elements in using teleoperated equipment if complex and speedy manipulation tasks are to be accomplished in remote and/or hazardous environments, such as hot cells, glove boxes, decommissioning, explosives disarmament, and space to name a few. In order to achieve this end the research presented in this paper has developed an admittance type exoskeleton like multi-fingered haptic hand user interface that secures the user’s palm and provides 3-dimensional force feedback to the user’s fingertips. Atypical to conventional haptic hand user interfaces that limit themselves to integrating the human hand’s characteristics just into the system’s mechanical design this system also perpetuates that inspiration into the designed user interface’s controller. This is achieved by manifesting the property differences of manipulation and grasping activities as they pertain to the human hand into a nonlinear master-slave force relationship. The results presented in this paper show that the admittance-type system has sufficient bandwidth that it appears nearly transparent to the user when the user is in free motion and when the system is subjected to a manipulation task, increased performance is achieved using the nonlinear force relationship compared to the traditional linear scaling techniques implemented in the vast majority of systems