Multi-modal Skill Memories for Online Learning of Interactive Robot Movement Generation

Queißer J. Multi-modal Skill Memories for Online Learning of Interactive Robot Movement Generation. Bielefeld: Universität Bielefeld; 2018.Modern robotic applications pose complex requirements with respect to the adaptation of actions regarding the variability in a given task. Reinforcement learning can optimize for changing conditions, but relearning from scratch is hardly feasible due to the high number of required rollouts. This work proposes a parameterized skill that generalizes to new actions for changing task parameters. The actions are encoded by a meta-learner that provides parameters for task-specific dynamic motion primitives. Experimental evaluation shows that the utilization of parameterized skills for initialization of the optimization process leads to a more effective incremental task learning. A proposed hybrid optimization method combines a fast coarse optimization on a manifold of policy parameters with a fine-grained parameter search in the unrestricted space of actions. It is shown that the developed algorithm reduces the number of required rollouts for adaptation to new task conditions. Further, this work presents a transfer learning approach for adaptation of learned skills to new situations. Application in illustrative toy scenarios, for a 10-DOF planar arm, a humanoid robot point reaching task and parameterized drumming on a pneumatic robot validate the approach. But parameterized skills that are applied on complex robotic systems pose further challenges: the dynamics of the robot and the interaction with the environment introduce model inaccuracies. In particular, high-level skill acquisition on highly compliant robotic systems such as pneumatically driven or soft actuators is hardly feasible. Since learning of the complete dynamics model is not feasible due to the high complexity, this thesis examines two alternative approaches: First, an improvement of the low-level control based on an equilibrium model of the robot. Utilization of an equilibrium model reduces the learning complexity and this thesis evaluates its applicability for control of pneumatic and industrial light-weight robots. Second, an extension of parameterized skills to generalize for forward signals of action primitives that result in an enhanced control quality of complex robotic systems. This thesis argues for a shift in the complexity of learning the full dynamics of the robot to a lower dimensional task-related learning problem. Due to the generalization in relation to the task variability, online learning for complex robots as well as complex scenarios becomes feasible. An experimental evaluation investigates the generalization capabilities of the proposed online learning system for robot motion generation. Evaluation is performed through simulation of a compliant 2-DOF arm and scalability to a complex robotic system is demonstrated for a pneumatically driven humanoid robot with 8-DOF

A Novel Framework for a Systematic Integration of Pneumatic-Muscle-Actuator-Driven Joints into Robotic Systems Via a Torque Control Interface

In this paper, two different torque control approaches for PMA-driven (PMA = Pneumatic muscle actuator) revolute joints are presented and tested. In previous work controllers for PMA-driven robots are typically customized for the use on a specific robotic system. In contrast, the proposed controllers define a general control interface for every robot that is actuated by PMA-driven joints. It will be shown that controlling the torque of a PMA-driven joint enables the use of standard robotic motion control frameworks, because the torque represents the natural input of the robotic equation of motion. Therefore, both proposed torque control approaches are interconnecting PMAs and their challenging characteristics on the one hand and “conventional” motion control strategies for robots on the other hand. After a detailed discussion of two different torque control approaches, we show that a torque controller handles all characteristics and dynamics of a PMA-driven joint internally, which implies that only its bandwidth and its static torque characteristic must be taken into account for the design of the outer motion control loop. This feature simplifies the integration of PMA-driven joints in robotic systems enormously, as will be demonstrated by a design of a cascade-structured, flatness-based motion controller for an exemplary robot with one degree of freedom.BMWi, ZF4007503, Development of a PMA-driven Exoskeleton for the Upper Bod

Model Based Control of Soft Robots: A Survey of the State of the Art and Open Challenges

Continuum soft robots are mechanical systems entirely made of continuously deformable elements. This design solution aims to bring robots closer to invertebrate animals and soft appendices of vertebrate animals (e.g., an elephant's trunk, a monkey's tail). This work aims to introduce the control theorist perspective to this novel development in robotics. We aim to remove the barriers to entry into this field by presenting existing results and future challenges using a unified language and within a coherent framework. Indeed, the main difficulty in entering this field is the wide variability of terminology and scientific backgrounds, making it quite hard to acquire a comprehensive view on the topic. Another limiting factor is that it is not obvious where to draw a clear line between the limitations imposed by the technology not being mature yet and the challenges intrinsic to this class of robots. In this work, we argue that the intrinsic effects are the continuum or multi-body dynamics, the presence of a non-negligible elastic potential field, and the variability in sensing and actuation strategies.Comment: 69 pages, 13 figure

Robot Assisted Shoulder Rehabilitation: Biomechanical Modelling, Design and Performance Evaluation

The upper limb rehabilitation robots have made it possible to improve the motor recovery in stroke survivors while reducing the burden on physical therapists. Compared to manual arm training, robot-supported training can be more intensive, of longer duration, repetitive and task-oriented. To be aligned with the most biomechanically complex joint of human body, the shoulder, specific considerations have to be made in the design of robotic shoulder exoskeletons. It is important to assist all shoulder degrees-of-freedom (DOFs) when implementing robotic exoskeletons for rehabilitation purposes to increase the range of motion (ROM) and avoid any joint axes misalignments between the robot and human’s shoulder that cause undesirable interaction forces and discomfort to the user. The main objective of this work is to design a safe and a robotic exoskeleton for shoulder rehabilitation with physiologically correct movements, lightweight modules, self-alignment characteristics and large workspace. To achieve this goal a comprehensive review of the existing shoulder rehabilitation exoskeletons is conducted first to outline their main advantages and disadvantages, drawbacks and limitations. The research has then focused on biomechanics of the human shoulder which is studied in detail using robotic analysis techniques, i.e. the human shoulder is modelled as a mechanism. The coupled constrained structure of the robotic exoskeleton connected to a human shoulder is considered as a hybrid human-robot mechanism to solve the problem of joint axes misalignments. Finally, a real-scale prototype of the robotic shoulder rehabilitation exoskeleton was built to test its operation and its ability for shoulder rehabilitation

Experiments in thrusterless robot locomotion control for space applications

While performing complex assembly tasks or moving about in space, a space robot should minimize the amount of propellant consumed. A study is presented of space robot locomotion and orientation without the use of thrusters. The goal was to design a robot control paradigm that will perform thrusterless locomotion between two points on a structure, and to implement this paradigm on an experimental robot. A two arm free flying robot was constructed which floats on a cushion of air to simulate in 2-D the drag free, zero-g environment of space. The robot can impart momentum to itself by pushing off from an external structure in a coordinated two arm maneuver, and can then reorient itself by activating a momentum wheel. The controller design consists of two parts: a high level strategic controller and a low level dynamic controller. The control paradigm was verified experimentally by commanding the robot to push off from a structure with both arms, rotate 180 degs while translating freely, and then to catch itself on another structure. This method, based on the computed torque, provides a linear feedback law in momentum and its derivatives for a system of rigid bodies

Adaptive control for wearable robots in human-centered rehabilitation tasks

Robotic rehabilitation therapies have been improving by providing the needed assistance to the patient, in a human-centered environment, and also helping the therapist to choose the necessary procedure. This thesis presents an adaptive "Assistance-as-needed" strategy which adheres to the specific needs of the patient and with the inputs from the therapist, whenever needed. The exertion of assistive and responsive behavior of the lower limb wearable robot is dedicated for the rehabilitation of incomplete spinal cord injury (SCI) patients. The main objective is to propose and evaluate an adaptive control model on a wearable robot, assisting the user and adhering to their needs, with no or less combination of external devices. The adaptation must be more interactive to understand the user needs and their volitional orders. Similarly, by using the existing muscular strength, in incomplete SCI patients, as a motivation to pursue the movement and assist them, only when needed. The adaptive behavior of the wearable robot is proposed by monitoring the interaction and movement of the user. This adaptation is achieved by modulating the stiffness of the exoskeleton in function of joint parameters, such as positions and interaction torques. These joint parameters are measured from the user independently and then used to update the new stiffness value. The adaptive algorithm performs with no need of external sensors, making it simple in terms of usage. In terms of rehabilitation, it is also desirable to be compatible with combination of assistive devices such as muscle stimulation, neural activity (BMI) and body balance (Wii), to deliver a user friendly and effective therapy. Combination of two control approaches has been employed, to improve the efficiency of the adaptive control model and was evaluated using a wearable lower limb exoskeleton device, H1. The control approaches, Hierarchical and Task based approach have been used to assist the patient as needed and simultaneously motivate the patient to pursue the therapy. Hierarchical approach facilitates combination of multiple devices to deliver an effective therapy by categorizing the control architecture in two layers, Low level and High level control. Task-based approaches engage in each task individually and allow the possibility to combine them at any point of time. It is also necessary to provide an interaction based approach to ensure the complete involvement of the user and for an effective therapy. By means of this dissertation, a task based adaptive control is proposed, in function of human-orthosis interaction, which is applied on a hierarchical control scheme. This control scheme is employed in a wearable robot, with the intention to be applied or accommodated to different pathologies, with its adaptive capabilities. The adaptive control model for gait assistance provides a comprehensive solution through a single implementation: Adaptation inside a gait cycle, continuous support through gait training and in real time. The performance of this control model has been evaluated with healthy subjects, as a preliminary study, and with paraplegic patients. Results of the healthy subjects showed a significant change in the pattern of the interaction torques, elucidating a change in the effort and adaptation to the user movement. In case of patients, the adaptation showed a significant improvement in the joint performance (flexion/extension range) and change in interaction torques. The change in interaction torques (positive to negative) reflects the active participation of the patient, which also explained the adaptive performance. The patients also reported that the movement of the exoskeleton is flexible and the walking patterns were similar to their own distinct patterns. The presented work is performed as part of the project HYPER, funded by Ministerio de Ciencia y Innovación, Spain. (CSD2009 - 00067 CONSOLIDER INGENIOLas terapias de rehabilitación robóticas han sido mejoradas gracias a la inclusión de la asistencia bajo demanda, adaptada a las variaciones de las necesidades del paciente, así como a la inclusión de la ayuda al terapeuta en la elección del procedimiento necesario. Esta tesis presenta una estrategia adaptativa de asistencia bajo demanda, la cual se ajusta a las necesidades específicas del paciente junto a las aportaciones del terapeuta siempre que sea necesario. El esfuerzo del comportamiento asistencial y receptivo del robot personal portátil para extremidades inferiores está dedicado a la rehabilitación de pacientes con lesión de la médula espinal (LME) incompleta. El objetivo principal es proponer y evaluar un modelo de control adaptativo en un robot portátil, ayudando al usuario y cumpliendo con sus necesidades, en ausencia o con reducción de dispositivos externos. La adaptación debe ser más interactiva para entender las necesidades del usuario y sus intenciones u órdenes volitivas. De modo similar, usando la fuerza muscular existente (en pacientes con LME incompleta) como motivación para lograr el movimiento y asistirles solo cuando sea necesario. El comportamiento adaptativo del robot portátil se propone mediante la monitorización de la interacción y movimiento del usuario. Esta adaptación conjunta se consigue modulando la rigidez en función de los parámetros de la articulación, tales como posiciones y pares de torsión. Dichos parámetros se miden del usuario de forma independiente y posteriormente se usan para actualizar el nuevo valor de la rigidez. El desempeño del algoritmo adaptativo no requiere de sensores externos, lo que favorece la simplicidad de su uso. Para una adecuada rehabilitación, efectiva y accesible para el usuario, es necesaria la compatibilidad con diversos mecanismos de asistencia tales como estimulación muscular, actividad neuronal y equilibrio corporal. Para mejorar la eficiencia del modelo de control adaptativo se ha empleado una combinación de dos enfoques de control, y para su evaluación se ha utilizado un exoesqueleto robótico H1. Los enfoques de control Jerárquico y de Tarea se han utilizado para ayudar al usuario según sea necesario, y al mismo tiempo motivarle para continuar el tratamiento. Enfoque jerárquico facilita la combinación de múltiples dispositivos para ofrecer un tratamiento eficaz mediante la categorización de la arquitectura de control en dos niveles : el control de bajo nivel y de alto nivel. Los enfoques basados en tareas involucran a la persona en cada tarea individual, y ofrecen la posibilidad de combinarlas en cualquier momento. También es necesario proporcionar un enfoque basado en la interacción con el usuario, para asegurar su participación y lograr así una terapia eficaz. Mediante esta tesis, proponemos un control adaptativo basado en tareas y en función de la interacción persona-ortesis, que se aplica en un esquema de control jerárquico. Este esquema de control se emplea en un robot portátil, con la intención de ser aplicado o acomodado a diferentes patologías, con sus capacidades de adaptación. El modelo de control adaptativo propuesto proporciona una solución integral a través de una única aplicación: adaptación dentro de la marcha y apoyo continúo a través de ejercicios de movilidad en tiempo real. El rendimiento del modelo se ha evaluado en sujetos sanos según un estudio preliminar, y posteriormente también en pacientes parapléjicos. Los resultados en sujetos sanos mostraron un cambio significativo en el patrón de los pares de interacción, elucidando un cambio en la energía y la adaptación al movimiento del usuario. En el caso de los pacientes, la adaptación mostró una mejora significativa en la actuación conjunta (rango de flexión / extensión) y el cambio en pares de interacción. El cambio activo en pares de interacción (positivo a negativo) refleja la participación activa del paciente, lo que también explica el comportamiento adaptativo

Dynamics and Controls of Fluidic Pressure-Fed Mechanism (FPFM) of Nanopositioning System

Flexure or compliant mechanisms are employed in many precisions engineered devices due to their compactness, linearity, resolution, etc. Yet, critical issues remain in motion errors, thermal instability, limited bandwidth, and vibration of dynamic systems. Those issues cannot be negligible to maintain high precision and accuracy for precision engineering applications. In this thesis, a novel fluidic pressure-fed mechanism (FPFM) is proposed and investigated. The proposed method is designing internal fluidic channels inside the spring structure of the flexure mechanism using the additive manufacturing (AM) process to overcome addressed challenges. By applying pneumatic/hydraulic pressure and filling media into fluidic channels, dynamic characteristics of each spring structure of the flexure mechanism can be altered or adjusted to correct motion errors, increase operating speed, and suppress vibration. Additionally, FPFM can enhance thermal stability by flowing fluids without affecting the motion quality of the dynamic system. Lastly, the motion of the nanopositioning system driven by FPFM can provide sub-nanometer resolution motion, and this enables the nanopositioning system to have two linear motion in a monolithic structure. The main objective of this thesis is to propose and validate the feasibility of FPFM that can ultimately be used for a monolithic FPFM dual-mode stage for providing high positioning performance without motion errors while reducing vibration and increasing thermal stability and bandwidth

33rd Aerospace Mechanisms Symposium

The proceedings of the 33rd Aerospace Mechanisms Symposium are reported. JPL hosted the conference, which was held at the Pasadena Conference and Exhibition Center, Pasadena, California, on May 19-21, 1999. Lockheed Martin Missiles and Space cosponsored the symposium. Technology areas covered include bearings and tribology; pointing, solar array and deployment mechanisms; orbiter/space station; and other mechanisms for spacecraft