Shared-Control Teleoperation Paradigms on a Soft Growing Robot Manipulator

Semi-autonomous telerobotic systems allow both humans and robots to exploit their strengths, while enabling personalized execution of a task. However, for new soft robots with degrees of freedom dissimilar to those of human operators, it is unknown how the control of a task should be divided between the human and robot. This work presents a set of interaction paradigms between a human and a soft growing robot manipulator, and demonstrates them in both real and simulated scenarios. The robot can grow and retract by eversion and inversion of its tubular body, a property we exploit to implement interaction paradigms. We implemented and tested six different paradigms of human-robot interaction, beginning with full teleoperation and gradually adding automation to various aspects of the task execution. All paradigms were demonstrated by two expert and two naive operators. Results show that humans and the soft robot manipulator can split control along degrees of freedom while acting simultaneously. In the simple pick-and-place task studied in this work, performance improves as the control is gradually given to the robot, because the robot can correct certain human errors. However, human engagement and enjoyment may be maximized when the task is at least partially shared. Finally, when the human operator is assisted by haptic feedback based on soft robot position errors, we observed that the improvement in performance is highly dependent on the expertise of the human operator.Comment: 15 pages, 14 figure

A Shared-Control Teleoperation Architecture for Nonprehensile Object Transportation

This article proposes a shared-control teleoperation architecture for robot manipulators transporting an object on a tray. Differently from many existing studies about remotely operated robots with firm grasping capabilities, we consider the case in which, in principle, the object can break its contact with the robot end-effector. The proposed shared-control approach automatically regulates the remote robot motion commanded by the user and the end-effector orientation to prevent the object from sliding over the tray. Furthermore, the human operator is provided with haptic cues informing about the discrepancy between the commanded and executed robot motion, which assist the operator throughout the task execution. We carried out trajectory tracking experiments employing an autonomous 7-degree-of-freedom (DoF) manipulator and compared the results obtained using the proposed approach with two different control schemes (i.e., constant tray orientation and no motion adjustment). We also carried out a human-subjects study involving 18 participants in which a 3-DoF haptic device was used to teleoperate the robot linear motion and display haptic cues to the operator. In all experiments, the results clearly show that our control approach outperforms the other solutions in terms of sliding prevention, robustness, commands tracking, and user’s preference

The Shape of Damping: Optimizing Damping Coefficients to Improve Transparency on Bilateral Telemanipulation

This thesis presents a novel optimization-based passivity control algorithm for hapticenabled bilateral teleoperation systems involving multiple degrees of freedom. In particular, in the context of energy-bounding control, the contribution focuses on the implementation of a passivity layer for an existing time-domain scheme, ensuring optimal transparency of the interaction along subsets of the environment space which are preponderant for the given task, while preserving the energy bounds required for passivity. The involved optimization problem is convex and amenable to real-time implementation. The effectiveness of the proposed design is validated via an experiment performed on a virtual teleoperated environment. The interplay between transparency and stability is a critical aspect in haptic-enabled bilateral teleoperation control. While it is important to present the user with the true impedance of the environment, destabilizing factors such as time delays, stiff environments, and a relaxed grasp on the master device may compromise the stability and safety of the system. Passivity has been exploited as one of the the main tools for providing sufficient conditions for stable teleoperation in several controller design approaches, such as the scattering algorithm, timedomain passivity control, energy bounding algorithm, and passive set position modulation. In this work it is presented an innovative energy-based approach, which builds upon existing time-domain passivity controllers, improving and extending their effectiveness and functionality. The set of damping coefficients are prioritized in each degree of freedom, the resulting transparency presents a realistic force feedback in comparison to the other directions. Thus, the prioritization takes effect using a quadratic programming algorithm to find the optimal values for the damping. Finally, the energy tanks approach on passivity control is a solution used to ensure stability in a system for robotics bilateral manipulation. The bilateral telemanipulation must maintain the principle of passivity in all moments to preserve the system\u2019s stability. This work presents a brief introduction to haptic devices as a master component on the telemanipulation chain; the end effector in the slave side is a representation of an interactive object within an environment having a force sensor as feedback signal. The whole interface is designed into a cross-platform framework named ROS, where the user interacts with the system. Experimental results are presented

A Compliant Partitioned Shared Control Strategy for an Orbital Robot

In this letter, a novel partitioned shared controller is proposed, which exploits a fully-actuated orbital robot to perform a primary end-effector task involving environmental interactions. This task is remotely performed using a bilateral teleoperation controller, while a secondary task is automatically controlled in situ for operational safety in a partitioned manner. In particular, the proposed method is derived as a modified 4-Channel teleoperation architecture. The orbital robot’s momentum and shape (joints) dynamics are exploited to benefit the controller design. Asymptotic stability and finite-gain L2-stability are proved in the absence and presence of external interactions, respectively. Furthermore, the proposed method is validated experimentally on a hardware-in-the-loop facility

A review on manipulation skill acquisition through teleoperation-based learning from demonstration

Manipulation skill learning and generalization have gained increasing attention due to the wide applications of robot manipulators and the spurt of robot learning techniques. Especially, the learning from demonstration method has been exploited widely and successfully in the robotic community, and it is regarded as a promising direction to realize the manipulation skill learning and generalization. In addition to the learning techniques, the immersive teleoperation enables the human to operate a remote robot with an intuitive interface and achieve the telepresence. Thus, it is a promising way to transfer manipulation skills from humans to robots by combining the learning methods and the teleoperation, and adapting the learned skills to different tasks in new situations. This review, therefore, aims to provide an overview of immersive teleoperation for skill learning and generalization to deal with complex manipulation tasks. To this end, the key technologies, e.g. manipulation skill learning, multimodal interfacing for teleoperation and telerobotic control, are introduced. Then, an overview is given in terms of the most important applications of immersive teleoperation platform for robot skill learning. Finally, this survey discusses the remaining open challenges and promising research topics

A Stable and Transparent Framework for Adaptive Shared Control of Robots

In mixed-initiative haptic shared control of robots, humans and automatic control system work in parallel. The commands to the robot are a weighted sum of forces from these two agents. This thesis develops control methods to improve the force feedback performance for mixed-initiative shared teleoperation and to adapt the control authority between human and automatic control system in a stable manner even in the presence of communication delays. All methods are validated on real robotic hardware

A Comprehensive Survey of the Tactile Internet: State of the art and Research Directions

The Internet has made several giant leaps over the years, from a fixed to a mobile Internet, then to the Internet of Things, and now to a Tactile Internet. The Tactile Internet goes far beyond data, audio and video delivery over fixed and mobile networks, and even beyond allowing communication and collaboration among things. It is expected to enable haptic communication and allow skill set delivery over networks. Some examples of potential applications are tele-surgery, vehicle fleets, augmented reality and industrial process automation. Several papers already cover many of the Tactile Internet-related concepts and technologies, such as haptic codecs, applications, and supporting technologies. However, none of them offers a comprehensive survey of the Tactile Internet, including its architectures and algorithms. Furthermore, none of them provides a systematic and critical review of the existing solutions. To address these lacunae, we provide a comprehensive survey of the architectures and algorithms proposed to date for the Tactile Internet. In addition, we critically review them using a well-defined set of requirements and discuss some of the lessons learned as well as the most promising research directions

Bimanual robot control for surface treatment tasks

This work develops a method to perform surface treatment tasks using a bimanual robotic system, i.e. two robot arms cooperatively performing the task. In particular, one robot arm holds the workpiece while the other robot arm has the treatment tool attached to its end-effector. Moreover, the human user teleoperates all the six coordinates of the former robot arm and two coordinates of the latter robot arm, i.e. the teleoperator can move the treatment tool on the plane given by the workpiece surface. Furthermore, a force sensor attached to the treatment tool is used to automatically attain the desired pressure between the tool and the workpiece and to automatically keep the tool orientation orthogonal to the workpiece surface. In addition, to assist the human user during the teleoperation, several constraints are defined for both robot arms in order to avoid exceeding the allowed workspace, e.g. to avoid collisions with other objects in the environment. The theory used in this work to develop the bimanual robot control relies on sliding mode control and task prioritisation. Finally, the feasibility and effectiveness of the method are shown through experimental results using two robot arms

Bimanual robot control for surface treatment tasks

This is an Author's Accepted Manuscript of an article published in Alberto García, J. Ernesto Solanes, Luis Gracia, Pau Muñoz-Benavent, Vicent Girbés-Juan & Josep Tornero (2022) Bimanual robot control for surface treatment tasks, International Journal of Systems Science, 53:1, 74-107, DOI: 10.1080/00207721.2021.1938279 [copyright Taylor & Francis], available online at: http://www.tandfonline.com/10.1080/00207721.2021.1938279[EN] This work develops a method to perform surface treatment tasks using a bimanual robotic system, i.e. two robot arms cooperatively performing the task. In particular, one robot arm holds the work-piece while the other robot arm has the treatment tool attached to its end-effector. Moreover, the human user teleoperates all the six coordinates of the former robot arm and two coordinates of the latter robot arm, i.e. the teleoperator can move the treatment tool on the plane given by the work- piece surface. Furthermore, a force sensor attached to the treatment tool is used to automatically attain the desired pressure between the tool and the workpiece and to automatically keep the tool orientation orthogonal to the workpiece surface. In addition, to assist the human user during the teleoperation, several constraints are defined for both robot arms in order to avoid exceeding the allowed workspace, e.g. to avoid collisions with other objects in the environment. The theory used in this work to develop the bimanual robot control relies on sliding mode control and task prioritisation. Finally, the feasibility and effectiveness of the method are shown through experimental results using two robot arms.This work was supported by Generalitat Valenciana [grant numbers ACIF/2019/007 and GV/2021/181] and Spanish Ministry of Science and Innovation [grant number PID2020117421RB-C21].García-Fernández, A.; Solanes, JE.; Gracia Calandin, LI.; Muñoz-Benavent, P.; Girbés-Juan, V.; Tornero, J. (2022). Bimanual robot control for surface treatment tasks. International Journal of Systems Science. 53(1):74-107. https://doi.org/10.1080/00207721.2021.19382797410753

Sistema de aquisição de dados por interface háptica

Mestrado em Engenharia MecânicaNeste trabalho e apresentada uma interface háptica com realimentação de força para a teleoperação de um robô humanoide é que aborda um novo conceito destinado à aprendizagem por demonstração em robôs, denominado de ensino telecinestésico. A interface desenvolvida pretende promover o ensino cinestésico num ambiente de tele-robótica enriquecido pela virtualização háptica do ambiente e restrições do robô. Os dados recolhidos através desta poderão então ser usados em aprendizagem por demonstração, uma abordagem poderosa que permite aprender padrões de movimento sem a necessidade de modelos dinâmicos complexos, mas que geralmente é apresentada com demonstrações que não são fornecidas teleoperando os robôs. Várias experiências são referidas onde o ensino cinestésico em aprendizagem robótica foi utilizado com um sucesso considerável, bem como novas metodologias e aplicações com aparelhos hápticos. Este trabalho foi realizado com base na plataforma proprietária de 27 graus-de-liberdade do Projeto Humanoide da Universidade de Aveiro (PHUA), definindo novas methodologias de comando em tele-operação, uma nova abordagem de software e ainda algumas alterações ao hardware. Um simulador de corpo inteiro do robô em MATLAB SimMechanics é apresentado que é capaz de determinar os requisitos dinâmicos de binário de cada junta para uma dada postura ou movimento, exemplificando com um movimento efectuado para subir um degrau. Ir a mostrar algumas das potencialidades mas também algumas das limitações restritivas do software. Para testar esta nova abordagem tele-cinestésica são dados exemplos onde o utilizador pode desenvolver demonstrações interagindo fisicamente com o robô humanoide através de um joystick háptico PHANToM. Esta metodologia ir a mostrar que permite uma interação natural para o ensino e perceção tele-robóticos, onde o utilizador fornece instruções e correções funcionais estando ciente da dinâmica do sistema e das suas capacidades e limitações físicas. Ser a mostrado que a abordagem consegue atingir um bom desempenho mesmo com operadores inexperientes ou não familiarizados com o sistema. Durante a interação háptica, a informação sensorial e as ordens que guiam a uma tarefa específica podem ser gravados e posteriormente utilizados para efeitos de aprendizagem.In this work an haptic interface using force feedback for the teleoperation of a humanoid robot is presented, that approaches a new concept for robot learning by demonstration known as tele-kinesthethic teaching. This interface aims at promoting kinesthethic teaching in telerobotic environments enriched by the haptic virtualization of the robot's environment and restrictions. The data collected through this interface can later be in robot learning by demonstration, a powerful approach for learning motion patterns without complex dynamical models, but which is usually presented using demonstrations that are not provided by teleoperating the robots. Several experiments are referred where kinesthetic teaching for robot learning was used with considerable success, as well as other new methodologies and applications with haptic devices. This work was conducted on the proprietary 27 DOF University of Aveiro Humanoid Project (PHUA) robot, de ning new wiring and software solutions, as well as a new teleoperation command methodology. A MATLAB Sim- Mechanics full body robot simulator is presented that is able to determine dynamic joint torque requirements for a given robot movement or posture, exempli ed with a step climbing application. It will show some of the potentialities but also some restricting limitations of the software. To test this new tele-kinesthetic approach, examples are shown where the user can provide demonstrations by physically interacting with the humanoid robot through a PHANToM haptic joystick. This methodology will show that it enables a natural interface for telerobotic teaching and sensing, in which the user provides functional guidance and corrections while being aware of the dynamics of the system and its physical capabilities and / or constraints. It will also be shown that the approach can have a good performance even with inexperienced or unfamiliarized operators. During haptic interaction, the sensory information and the commands guiding the execution of a speci c task can be recorded and that data log from the human-robot interaction can be later used for learning purposes