Neural Learning of Vector Fields for Encoding Stable Dynamical Systems

Lemme A, Reinhart F, Neumann K, Steil JJ. Neural Learning of Vector Fields for Encoding Stable Dynamical Systems. Neurocomputing. 2014;141:3-14

Multicontact Motion Retargeting Using Whole-Body Optimization of Full Kinematics and Sequential Force Equilibrium

This article presents a multicontact motion adaptation framework that enables teleoperation of high degree-of-freedom robots, such as quadrupeds and humanoids, for loco-manipulation tasks in multicontact settings. Our proposed algorithms optimize whole-body configurations and formulate the retargeting of multicontact motions as sequential quadratic programming, which is robust and stable near the edges of feasibility constraints. Our framework allows real-time operation of the robot and reduces cognitive load for the operator because infeasible commands are automatically adapted into physically stable and viable motions on the robot. The results in simulations with full dynamics demonstrated the effectiveness of teleoperating different legged robots interactively and generating rich multicontact movements. We evaluated the computational efficiency of the proposed algorithms, and further validated and analyzed multicontact loco-manipulation tasks on humanoid and quadruped robots by reaching, active pushing, and various traversal on uneven terrains

Development of lower-limb rehabilitation exercises using 3-PRS Parallel Robot and Dynamic Movement Primitives

[EN] The design of rehabilitation exercises applied to sprained ankles requires extreme caution, regarding the trajectories and the speed of the movements that will affect the patient. This paper presents a technique that allows a 3-PRS parallel robot to control such exercises, consisting of dorsi/plantar flexion and inversion/eversion ankle movements. The work includes a position control scheme for the parallel robot in order to follow a reference trajectory for each limb with the possibility of stopping the exercise in mid-execution without control loss. This stop may be motivated by the forces that the robot applies to the patient, acting like an alarm mechanism. The procedure introduced here is based on Dynamic Movement Primitives (DMPs).This work has been partially funded by FEDER-CICYT project with reference DPI2017-84201-R financed by Ministerio de Economía, Industria e Innovación (Spain).Escarabajal Sánchez, RJ.; Abu Dakka, FJM.; Pulloquinga Zapata, J.; Mata Amela, V.; Vallés Miquel, M.; Valera Fernández, Á. (2020). Development of lower-limb rehabilitation exercises using 3-PRS Parallel Robot and Dynamic Movement Primitives. Multidisciplinary Journal for Education, Social and Technological Sciences. 7(2):30-44. https://doi.org/10.4995/muse.2020.13907OJS304472Abu-Dakka, F. J., Valera, A., Escalera, J. A., Vallés, M., Mata, V., & Abderrahim, M. (2015). Trajectory adaptation and learning for ankle rehabilitation using a 3-PRS parallel robot. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9245, 483-494. https://doi.org/10.1007/978-3-319-22876-1_41Atkeson, C. G., Moore, A. W., & Schaal, S. (1997). Locally Weighted Learning. Artificial Intelligence Review, 11(1-5), 11-73. https://doi.org/10.1007/978-94-017-2053-3_2Brockett, C. L., & Chapman, G. J. (2016). Biomechanics of the ankle. Orthopaedics and Trauma, 30(3), 232-238. https://doi.org/10.1016/j.mporth.2016.04.015Dai, J. S., Zhao, T., & Nester, C. (2004). Sprained Ankle Physiotherapy Based Mechanism Synthesis and Stiffness Analysis of a Robotic Rehabilitation Device. Autonomous Robots, 16(2), 207-218. https://doi.org/10.1023/B:AURO.0000016866.80026.d7Díaz-Rodríguez, M., Mata, V., Valera, Á., & Page, Á. (2010). A methodology for dynamic parameters identification of 3-DOF parallel robots in terms of relevant parameters. Mechanism and Machine Theory, 45(9), 1337-1356. https://doi.org/10.1016/j.mechmachtheory.2010.04.007Díaz, I., Gil, J. J., & Sánchez, E. (2011). Lower-Limb Robotic Rehabilitation: Literature Review and Challenges. Journal of Robotics, 2011(i), 1-11. https://doi.org/10.1155/2011/759764Fanger, Y., Umlauft, J., & Hirche, S. (2016). Gaussian Processes for Dynamic Movement Primitives with application in knowledge-based cooperation. IEEE International Conference on Intelligent Robots and Systems, 2016-Novem, 3913-3919. https://doi.org/10.1109/IROS.2016.7759576Gosselin, C., & Angeles, J. (1990). Singularity Analysis of Closed-Loop Kinematic Chains. IEEE Transactions on Robotics and Automation, 6(3), 281-290. https://doi.org/10.1109/70.56660Hesse, S., & Uhlenbrock, D. (2000). A mechanized gait trainer for restoration of gait. Journal of Rehabilitation Research and Development, 37(6), 701-708.Ijspeert, A. J., Nakanishi, J., Hoffmann, H., Pastor, P., & Schaal, S. (2013). Dynamical movement primitives: Learning attractor models formotor behaviors. Neural Computation, 25(2), 328-373. https://doi.org/10.1162/NECO_a_00393Ijspeert, A. J., Nakanishi, J., & Schaal, S. (2002). Movement imitation with nonlinear dynamical systems in humanoid robots. Proceedings - IEEE International Conference on Robotics and Automation, 2, 1398-1403. https://doi.org/10.1109/ROBOT.2002.1014739Liu, G., Gao, J., Yue, H., Zhang, X., & Lu, G. (2006). Design and kinematics simulation of parallel robots for ankle rehabilitation. 2006 IEEE International Conference on Mechatronics and Automation, ICMA 2006, 2006, 1109-1113. https://doi.org/10.1109/ICMA.2006.257780Nakanishi, J., Morimoto, J., Endo, G., Cheng, G., Schaal, S., & Kawato, M. (2004). Learning from demonstration and adaptation of biped locomotion. Robotics and Autonomous Systems, 47(2-3), 79-91. https://doi.org/10.1016/j.robot.2004.03.003Nemec, B., & Ude, A. (2012). Action sequencing using dynamic movement primitives. Robotica, 30(5), 837-846. https://doi.org/10.1017/S0263574711001056Patel, Y. D., & George, P. M. (2012). Parallel Manipulators Applications-A Survey. Modern Mechanical Engineering, 02(03), 57-64. https://doi.org/10.4236/mme.2012.23008Paul, R. P. (1981). Robot Manipulators: Mathematics, Programming, and Control : the Computer Control of Robot Manipulators (p. 279).Reinkensmeyer, D. J., Aoyagi, D., Emken, J. L., Galvez, J. A., Ichinose, W., Kerdanyan, G., Maneekobkunwong, S., Minakata, K., Nessler, J. A., Weber, R., Roy, R. R., De Leon, R., Bobrow, J. E., Harkema, S. J., & Reggie Edgerton, V. (2006). Tools for understanding and optimizing robotic gait training. Journal of Rehabilitation Research and Development, 43(5), 657-670. https://doi.org/10.1682/JRRD.2005.04.0073Safran, M. R., Benedetti, R. S., Bartolozzi, A. R., & Mandelbaum, B. R. (1999). Lateral ankle sprains: A comprehensive review part 1: Etiology, pathoanatomy, histopathogenesis, and diagnosis. In Medicine and Science in Sports and Exercise (Vol. 31, Issue 7 SUPPL., pp. S429-S437).https://doi.org/10.1097/00005768-199907001-00004Saglia, J. A., Tsagarakis, N. G., Dai, J. S., & Caldwell, D. G. (2013). Control strategies for patient-assisted training using the ankle rehabilitation robot (ARBOT). IEEE/ASME Transactions on Mechatronics, 18(6), 1799-1808. https://doi.org/10.1109/TMECH.2012.2214228Schaal, S. (2006). Dynamic Movement Primitives -A Framework for Motor Control in Humans and Humanoid Robotics. In Adaptive Motion of Animals and Machines (pp. 261-280). https://doi.org/10.1007/4-431-31381-8_23Sui, P., Yao, L., Lin, Z., Yan, H., & Dai, J. S. (2009). Analysis and synthesis of ankle motion and rehabilitation robots. 2009 IEEE International Conference on Robotics and Biomimetics, ROBIO 2009, 3, 2533-2538. https://doi.org/10.1109/ROBIO.2009.5420487Tsoi, Y. H., Xie, S. Q., & Graham, A. E. (2009). Design, modeling and control of an ankle rehabilitation robot. Studies in Computational Intelligence, 177, 377-399. https://doi.org/10.1007/978-3-540-89933-4_18Vallés, M., Díaz-Rodrguez, M., Valera, Á., Mata, V., & Page, Á. (2012). Mechatronic development and dynamic control of a 3-dof parallel manipulator. Mechanics Based Design of Structures and Machines, 40(4), 434-452. https://doi.org/10.1080/15397734.2012.687292Xie, S. (2016). Advanced robotics for medical rehabilitation: current state of the art and recent advances. In Springer tracts in advanced robotics (Issue 108). https://doi.org/10.1007/978-3-319-19896-5Yoon, J., Ryu, J., & Lim, K. B. (2006). Reconfigurable ankle rehabilitation robot for various exercises. Journal of Robotic Systems, 22(SUPPL.), 15-33. https://doi.org/10.1002/rob.2015

Emergent coordination between humans and robots

Emergent coordination or movement synchronization is an often observed phenomenon in human behavior. Humans synchronize their gait when walking next to each other, they synchronize their postural sway when standing closely, and they also synchronize their movement behavior in many other situations of daily life. Why humans are doing this is an important question of ongoing research in many disciplines: apparently movement synchronization plays a role in children’s development and learning; it is related to our social and emotional behavior in interaction with others; it is an underlying principle in the organization of communication by means of language and gesture; and finally, models explaining movement synchronization between two individuals can also be extended to group behavior. Overall, one can say that movement synchronization is an important principle of human interaction behavior. Besides interacting with other humans, in recent years humans do more and more interact with technology. This was first expressed in the interaction with machines in industrial settings, was taken further to human-computer interaction and is now facing a new challenge: the interaction with active and autonomous machines, the interaction with robots. If the vision of today’s robot developers comes true, in the near future robots will be fully integrated not only in our workplace, but also in our private lives. They are supposed to support humans in activities of daily living and even care for them. These circumstances however require the development of interactional principles which the robot can apply to the direct interaction with humans. In this dissertation the problem of robots entering the human society will be outlined and the need for the exploration of human interaction principles that are transferable to human-robot interaction will be emphasized. Furthermore, an overview on human movement synchronization as a very important phenomenon in human interaction will be given, ranging from neural correlates to social behavior. The argument of this dissertation is that human movement synchronization is a simple but striking human interaction principle that can be applied in human-robot interaction to support human activity of daily living, demonstrated on the example of pick-and-place tasks. This argument is based on five publications. In the first publication, human movement synchronization is explored in goal-directed tasks which bare similar requirements as pick-and-place tasks in activities of daily living. In order to explore if a merely repetitive action of the robot is sufficient to encourage human movement synchronization, the second publication reports a human-robot interaction study in which a human interacts with a non-adaptive robot. Here however, movement synchronization between human and robot does not emerge, which underlines the need for adaptive mechanisms. Therefore, in the third publication, human adaptive behavior in goal-directed movement synchronization is explored. In order to make the findings from the previous studies applicable to human-robot interaction, in the fourth publication the development of an interaction model based on dynamical systems theory is outlined which is ready for implementation on a robotic platform. Following this, a brief overview on a first human-robot interaction study based on the developed interaction model is provided. The last publication describes an extension of the previous approach which also includes the human tendency to make use of events to adapt their movements to. Here, also a first human-robot interaction study is reported which confirms the applicability of the model. The dissertation concludes with a discussion on the presented findings in the light of human-robot interaction and psychological aspects of joint action research as well as the problem of mutual adaptation.Spontan auftretende Koordination oder Bewegungssynchronisierung ist ein häufig zu beobachtendes Phänomen im Verhalten von Menschen. Menschen synchronisieren ihre Schritte beim nebeneinander hergehen, sie synchronisieren die Schwingbewegung zum Ausgleich der Körperbalance wenn sie nahe beieinander stehen und sie synchronisieren ihr Bewegungsverhalten generell in vielen weiteren Handlungen des täglichen Lebens. Die Frage nach dem warum ist eine Frage mit der sich die Forschung in der Psychologie, Neuro- und Bewegungswissenschaft aber auch in der Sozialwissenschaft nach wie vor beschäftigt: offenbar spielt die Bewegungssynchronisierung eine Rolle in der kindlichen Entwicklung und beim Erlernen von Fähigkeiten und Verhaltensmustern; sie steht in direktem Bezug zu unserem sozialen Verhalten und unserer emotionalen Wahrnehmung in der Interaktion mit Anderen; sie ist ein grundlegendes Prinzip in der Organisation von Kommunikation durch Sprache oder Gesten; außerdem können Modelle, die Bewegungssynchronisierung zwischen zwei Individuen erklären, auch auf das Verhalten innerhalb von Gruppen ausgedehnt werden. Insgesamt kann man also sagen, dass Bewegungssynchronisierung ein wichtiges Prinzip im menschlichen Interaktionsverhalten darstellt. Neben der Interaktion mit anderen Menschen interagieren wir in den letzten Jahren auch zunehmend mit der uns umgebenden Technik. Hier fand zunächst die Interaktion mit Maschinen im industriellen Umfeld Beachtung, später die Mensch-Computer-Interaktion. Seit kurzem sind wir jedoch mit einer neuen Herausforderung konfrontiert: der Interaktion mit aktiven und autonomen Maschinen, Maschinen die sich bewegen und aktiv mit Menschen interagieren, mit Robotern. Sollte die Vision der heutigen Roboterentwickler Wirklichkeit werde, so werden Roboter in der nahen Zukunft nicht nur voll in unser Arbeitsumfeld integriert sein, sondern auch in unser privates Leben. Roboter sollen den Menschen in ihren täglichen Aktivitäten unterstützen und sich sogar um sie kümmern. Diese Umstände erfordern die Entwicklung von neuen Interaktionsprinzipien, welche Roboter in der direkten Koordination mit dem Menschen anwenden können. In dieser Dissertation wird zunächst das Problem umrissen, welches sich daraus ergibt, dass Roboter zunehmend Einzug in die menschliche Gesellschaft finden. Außerdem wird die Notwendigkeit der Untersuchung menschlicher Interaktionsprinzipien, die auf die Mensch-Roboter-Interaktion transferierbar sind, hervorgehoben. Die Argumentation der Dissertation ist, dass die menschliche Bewegungssynchronisierung ein einfaches aber bemerkenswertes menschliches Interaktionsprinzip ist, welches in der Mensch-Roboter-Interaktion angewendet werden kann um menschliche Aktivitäten des täglichen Lebens, z.B. Aufnahme-und-Ablege-Aufgaben (pick-and-place tasks), zu unterstützen. Diese Argumentation wird auf fünf Publikationen gestützt. In der ersten Publikation wird die menschliche Bewegungssynchronisierung in einer zielgerichteten Aufgabe untersucht, welche die gleichen Anforderungen erfüllt wie die Aufnahme- und Ablageaufgaben des täglichen Lebens. Um zu untersuchen ob eine rein repetitive Bewegung des Roboters ausreichend ist um den Menschen zur Etablierung von Bewegungssynchronisierung zu ermutigen, wird in der zweiten Publikation eine Mensch-Roboter-Interaktionsstudie vorgestellt in welcher ein Mensch mit einem nicht-adaptiven Roboter interagiert. In dieser Studie wird jedoch keine Bewegungssynchronisierung zwischen Mensch und Roboter etabliert, was die Notwendigkeit von adaptiven Mechanismen unterstreicht. Daher wird in der dritten Publikation menschliches Adaptationsverhalten in der Bewegungssynchronisierung in zielgerichteten Aufgaben untersucht. Um die so gefundenen Mechanismen für die Mensch-Roboter Interaktion nutzbar zu machen, wird in der vierten Publikation die Entwicklung eines Interaktionsmodells basierend auf Dynamischer Systemtheorie behandelt. Dieses Modell kann direkt in eine Roboterplattform implementiert werden. Anschließend wird kurz auf eine erste Studie zur Mensch- Roboter Interaktion basierend auf dem entwickelten Modell eingegangen. Die letzte Publikation beschreibt eine Weiterentwicklung des bisherigen Vorgehens welche der Tendenz im menschlichen Verhalten Rechnung trägt, die Bewegungen an Ereignissen auszurichten. Hier wird außerdem eine erste Mensch-Roboter- Interaktionsstudie vorgestellt, die die Anwendbarkeit des Modells bestätigt. Die Dissertation wird mit einer Diskussion der präsentierten Ergebnisse im Kontext der Mensch-Roboter-Interaktion und psychologischer Aspekte der Interaktionsforschung sowie der Problematik von beiderseitiger Adaptivität abgeschlossen

Learning to reach and reaching to learn: a unified approach to path planning and reactive control through reinforcement learning

The next generation of intelligent robots will need to be able to plan reaches. Not just ballistic point to point reaches, but reaches around things such as the edge of a table, a nearby human, or any other known object in the robot’s workspace. Planning reaches may seem easy to us humans, because we do it so intuitively, but it has proven to be a challenging problem, which continues to limit the versatility of what robots can do today. In this document, I propose a novel intrinsically motivated RL system that draws on both Path/Motion Planning and Reactive Control. Through Reinforcement Learning, it tightly integrates these two previously disparate approaches to robotics. The RL system is evaluated on a task, which is as yet unsolved by roboticists in practice. That is to put the palm of the iCub humanoid robot on arbitrary target objects in its workspace, start- ing from arbitrary initial configurations. Such motions can be generated by planning, or searching the configuration space, but this typically results in some kind of trajectory, which must then be tracked by a separate controller, and such an approach offers a brit- tle runtime solution because it is inflexible. Purely reactive systems are robust to many problems that render a planned trajectory infeasible, but lacking the capacity to search, they tend to get stuck behind constraints, and therefore do not replace motion planners. The planner/controller proposed here is novel in that it deliberately plans reaches without the need to track trajectories. Instead, reaches are composed of sequences of reactive motion primitives, implemented by my Modular Behavioral Environment (MoBeE), which provides (fictitious) force control with reactive collision avoidance by way of a realtime kinematic/geometric model of the robot and its workspace. Thus, to the best of my knowledge, mine is the first reach planning approach to simultaneously offer the best of both the Path/Motion Planning and Reactive Control approaches. By controlling the real, physical robot directly, and feeling the influence of the con- straints imposed by MoBeE, the proposed system learns a stochastic model of the iCub’s configuration space. Then, the model is exploited as a multiple query path planner to find sensible pre-reach poses, from which to initiate reaching actions. Experiments show that the system can autonomously find practical reaches to target objects in workspace and offers excellent robustness to changes in the workspace configuration as well as noise in the robot’s sensory-motor apparatus

Passive exercise adaptation for ankle rehabilitation based on learning control framework

This article belongs to the Special Issue Human-Robot Interaction.Ankle injuries are among the most common injuries in sport and daily life. However, for their recovery, it is important for patients to perform rehabilitation exercises. These exercises are usually done with a therapist's guidance to help strengthen the patient's ankle joint and restore its range of motion. However, in order to share the load with therapists so that they can offer assistance to more patients, and to provide an efficient and safe way for patients to perform ankle rehabilitation exercises, we propose a framework that integrates learning techniques with a 3-PRS parallel robot, acting together as an ankle rehabilitation device. In this paper, we propose to use passive rehabilitation exercises for dorsiflexion/plantar flexion and inversion/eversion ankle movements. The therapist is needed in the first stage to design the exercise with the patient by teaching the robot intuitively through learning from demonstration. We then propose a learning control scheme based on dynamic movement primitives and iterative learning control, which takes the designed exercise trajectory as a demonstration (an input) together with the recorded forces in order to reproduce the exercise with the patient for a number of repetitions defined by the therapist. During the execution, our approach monitors the sensed forces and adapts the trajectory by adding the necessary offsets to the original trajectory to reduce its range without modifying the original trajectory and subsequently reducing the measured forces. After a predefined number of repetitions, the algorithm restores the range gradually, until the patient is able to perform the originally designed exercise. We validate the proposed framework with both real experiments and simulation using a Simulink model of the rehabilitation parallel robot that has been developed in our lab

Passive Exercise Adaptation for Ankle Rehabilitation Based on Learning Control Framework

[EN] Ankle injuries are among the most common injuries in sport and daily life. However, for their recovery, it is important for patients to perform rehabilitation exercises. These exercises are usually done with a therapist's guidance to help strengthen the patient's ankle joint and restore its range of motion. However, in order to share the load with therapists so that they can offer assistance to more patients, and to provide an efficient and safe way for patients to perform ankle rehabilitation exercises, we propose a framework that integrates learning techniques with a 3-PRS parallel robot, acting together as an ankle rehabilitation device. In this paper, we propose to use passive rehabilitation exercises for dorsiflexion/plantar flexion and inversion/eversion ankle movements. The therapist is needed in the first stage to design the exercise with the patient by teaching the robot intuitively through learning from demonstration. We then propose a learning control scheme based on dynamic movement primitives and iterative learning control, which takes the designed exercise trajectory as a demonstration (an input) together with the recorded forces in order to reproduce the exercise with the patient for a number of repetitions defined by the therapist. During the execution, our approach monitors the sensed forces and adapts the trajectory by adding the necessary offsets to the original trajectory to reduce its range without modifying the original trajectory and subsequently reducing the measured forces. After a predefined number of repetitions, the algorithm restores the range gradually, until the patient is able to perform the originally designed exercise. We validate the proposed framework with both real experiments and simulation using a Simulink model of the rehabilitation parallel robot that has been developed in our lab.This work has been partially funded by the FEDER-CICYT project with reference DPI2017-84201-R (Integracion de modelos biomecanicos en el desarrollo y operacion de robots rehabilitadores reconfigurables) financed by Ministerio de Economia, Industria e Innovacion (Spain).Abu-Dakka, FJ.; Valera Fernández, Á.; Escalera, JA.; Abderrahim, M.; Page Del Pozo, AF.; Mata Amela, V. (2020). Passive Exercise Adaptation for Ankle Rehabilitation Based on Learning Control Framework. Sensors. 20(21):1-23. https://doi.org/10.3390/s20216215S123202