Human to robot hand motion mapping methods: review and classification

In this article, the variety of approaches proposed in literature to address the problem of mapping human to robot hand motions are summarized and discussed. We particularly attempt to organize under macro-categories the great quantity of presented methods, that are often difficult to be seen from a general point of view due to different fields of application, specific use of algorithms, terminology and declared goals of the mappings. Firstly, a brief historical overview is reported, in order to provide a look on the emergence of the human to robot hand mapping problem as a both conceptual and analytical challenge that is still open nowadays. Thereafter, the survey mainly focuses on a classification of modern mapping methods under six categories: direct joint, direct Cartesian, taskoriented, dimensionality reduction based, pose recognition based and hybrid mappings. For each of these categories, the general view that associates the related reported studies is provided, and representative references are highlighted. Finally, a concluding discussion along with the authors’ point of view regarding future desirable trends are reported.This work was supported in part by the European Commission’s Horizon 2020 Framework Programme with the project REMODEL under Grant 870133 and in part by the Spanish Government under Grant PID2020-114819GB-I00.Peer ReviewedPostprint (published version

Teleoperación [de robots]: técnicas, aplicaciones, entorno sensorial y teleoperación inteligente

En este trabajo centraremos la atención en los sistemas robóticos teleoperados, especialmente analizaremos los sistemas teleoperados desde internet, veremos una clasificación de las metodologías de teleoperación, los diferentes sistemas de control y daremos una visión del estado del arte en este ámbito de conocimiento

Survey: Robot Programming by Demonstration

Robot PbD started about 30 years ago, growing importantly during the past decade. The rationale for moving from purely preprogrammed robots to very flexible user-based interfaces for training the robot to perform a task is three-fold. First and foremost, PbD, also referred to as {\em imitation learning} is a powerful mechanism for reducing the complexity of search spaces for learning. When observing either good or bad examples, one can reduce the search for a possible solution, by either starting the search from the observed good solution (local optima), or conversely, by eliminating from the search space what is known as a bad solution. Imitation learning is, thus, a powerful tool for enhancing and accelerating learning in both animals and artifacts. Second, imitation learning offers an implicit means of training a machine, such that explicit and tedious programming of a task by a human user can be minimized or eliminated (Figure \ref{fig:what-how}). Imitation learning is thus a ``natural'' means of interacting with a machine that would be accessible to lay people. And third, studying and modeling the coupling of perception and action, which is at the core of imitation learning, helps us to understand the mechanisms by which the self-organization of perception and action could arise during development. The reciprocal interaction of perception and action could explain how competence in motor control can be grounded in rich structure of perceptual variables, and vice versa, how the processes of perception can develop as means to create successful actions. PbD promises were thus multiple. On the one hand, one hoped that it would make the learning faster, in contrast to tedious reinforcement learning methods or trials-and-error learning. On the other hand, one expected that the methods, being user-friendly, would enhance the application of robots in human daily environments. Recent progresses in the field, which we review in this chapter, show that the field has make a leap forward the past decade toward these goals and that these promises may be fulfilled very soon

Intuitive Human-Machine Interfaces for Non-Anthropomorphic Robotic Hands

As robots become more prevalent in our everyday lives, both in our workplaces and in our homes, it becomes increasingly likely that people who are not experts in robotics will be asked to interface with robotic devices. It is therefore important to develop robotic controls that are intuitive and easy for novices to use. Robotic hands, in particular, are very useful, but their high dimensionality makes creating intuitive human-machine interfaces for them complex. In this dissertation, we study the control of non-anthropomorphic robotic hands by non-roboticists in two contexts: collaborative manipulation and assistive robotics. In the field of collaborative manipulation, the human and the robot work side by side as independent agents. Teleoperation allows the human to assist the robot when autonomous grasping is not able to deal sufficiently well with corner cases or cannot operate fast enough. Using the teleoperator’s hand as an input device can provide an intuitive control method, but finding a mapping between a human hand and a non-anthropomorphic robot hand can be difficult, due to the hands’ dissimilar kinematics. In this dissertation, we seek to create a mapping between the human hand and a fully actuated, non-anthropomorphic robot hand that is intuitive enough to enable effective real-time teleoperation, even for novice users. We propose a low-dimensional and continuous teleoperation subspace which can be used as an intermediary for mapping between different hand pose spaces. We first propose the general concept of the subspace, its properties and the variables needed to map from the human hand to a robot hand. We then propose three ways to populate the teleoperation subspace mapping. Two of our mappings use a dataglove to harvest information about the user's hand. We define the mapping between joint space and teleoperation subspace with an empirical definition, which requires a person to define hand motions in an intuitive, hand-specific way, and with an algorithmic definition, which is kinematically independent, and uses objects to define the subspace. Our third mapping for the teleoperation subspace uses forearm electromyography (EMG) as a control input. Assistive orthotics is another area of robotics where human-machine interfaces are critical, since, in this field, the robot is attached to the hand of the human user. In this case, the goal is for the robot to assist the human with movements they would not otherwise be able to achieve. Orthotics can improve the quality of life of people who do not have full use of their hands. Human-machine interfaces for assistive hand orthotics that use EMG signals from the affected forearm as input are intuitive and repeated use can strengthen the muscles of the user's affected arm. In this dissertation, we seek to create an EMG based control for an orthotic device used by people who have had a stroke. We would like our control to enable functional motions when used in conjunction with a orthosis and to be robust to changes in the input signal. We propose a control for a wearable hand orthosis which uses an easy to don, commodity forearm EMG band. We develop an supervised algorithm to detect a user’s intent to open and close their hand, and pair this algorithm with a training protocol which makes our intent detection robust to changes in the input signal. We show that this algorithm, when used in conjunction with an orthosis over several weeks, can improve distal function in users. Additionally, we propose two semi-supervised intent detection algorithms designed to keep our control robust to changes in the input data while reducing the length and frequency of our training protocol

Stabilizer architecture for humanoid robots collaborating with humans

Hoy en día, los avances en las tecnologías de información y comunicación permiten el uso de robots como compañeros en las actividades con los seres humanos. Mientras que la mayoría de las investigaciones existentes se dedica a la interacción entre humanos y robots, el marco de esta investigación está centrado en el uso de robots como agentes de colaboración. En particular, este estudio está dedicado a los robots humanoides que puedan ayudar a la gente en varias tareas en entornos de trabajo. Los robots humanoides son sin duda los m as adecuados para este tipo de situaciones: pueden usar las mismas herramientas que los seres humanos y son lo m as probablemente aceptados por ellos. Después de explicar las ventajas de las tareas de colaboración entre los humanos y los robots y las diferencias con respecto a los sistemas de interacción y de teleoperación, este trabajo se centra en el nivel de las tecnologías que es necesario para lograr ese objetivo. El problema más complicado en el control de humanoides es el balance de la estructura. Este estudio se centra en técnicas novedosas para la estimación de la actitud del robot, que se utilizarán para el control. El control del robot se basa en un modelo muy conocido y simplificado: el péndulo invertido. Este modelo permite tener un control en tiempo real sobre la estructura, mientras que esté sometida a fuerzas externas / disturbios. Trayectorias suaves para el control de humanoides se han propuesto y probado en plataformas reales: éstos permiten reducir los impactos del robot con su entorno. Finalmente, el estudio extiende estos resultados a una contribución para la arquitectura de colaboración humano-humanoide. Dos tipos de colaboraciones humano humanoide se analizan: la colaboración física, donde robots y humanos comparten el mismo espacio y tienen un contacto físico (o por medio de un objeto), y una colaboración a distancia, en la que el ser humano está relativamente lejos del robot y los dos agentes colaboran por medio de una interfaz. El paradigma básico de esta colaboración robótica es: lo que es difícil (o peligroso) para el ser humano se hace por medio del robot y lo que es difícil para el robot lo puede mejor hacer el humano. Es importante destacar que el contexto de los experimentos no se basa en una unica plataforma humanoide; por el contrario, tres plataformas han sido objeto de los experimentos: se han empleado los robots HOAP-3, HRP-2 y TEO. ----------------------------------------------------------------------------------------------------------------------------------------------------------Nowadays, the advances in information and communication technologies permit the use of robots as companions in activities with humans. While most of the existing research is dedicated to the interaction between humans and robots, the framework of this research is the use of robots as collaborative agents. In particular, this study is dedicated to humanoid robots which should assist people in several tasks in working environments. Humanoid robots are certainly the most adequate for such situations: they can use the same tools as humans and are most likely accepted by them. After explaining the advantages of collaborative tasks among humans and robots and the differences with respect to interaction and teleoperation systems, this work focuses on the level of technologies which is necessary in order to achieve such a goal. The most complicated problem in humanoid control is the structure balance. This study focuses in novel techniques in the attitude estimation of the robot, to be used for the control. The control of the robot is based on a very well-known and simplified model: the double inverted pendulum. This model permits having a real-time control on the structure while submitted to external forces/disturbances. The control actions are strongly dependent on the three stability regions, which are determined by the position of the ZMP in the support polygon. Smooth trajectories for the humanoid control have been proposed and tested on real platforms: these permit reducing the impacts of the robot with its environment. Finally, the study extends these results to a contribution for human-humanoid collaboration architecture. Two types of human-humanoid collaborations are analyzed: a physical collaboration, where robot and human share the same space and have a physical contact (or by means of an object), and a remote collaboration, in which the human is relatively far away from the robot and the two agents collaborate using an interface. The basic paradigm for this robotic collaboration is: what is difficult (or dangerous) for the human is done by the robot and what is difficult for the robot is better done by the human. Importantly, the testing context is not based on a single humanoid platform; on the contrary, three platforms have been object of the experiments: the Hoap-3, HRP-2 and HRP2 robot have been employed