Queißer J. Multi-modal Skill Memories for Online Learning of Interactive Robot Movement Generation. Bielefeld: Universität Bielefeld; 2018.Modern robotic applications pose complex requirements with respect to the adaptation of
actions regarding the variability in a given task. Reinforcement learning can optimize for
changing conditions, but relearning from scratch is hardly feasible due to the high number of
required rollouts. This work proposes a parameterized skill that generalizes to new actions
for changing task parameters. The actions are encoded by a meta-learner that provides
parameters for task-specific dynamic motion primitives. Experimental evaluation shows that
the utilization of parameterized skills for initialization of the optimization process leads to a
more effective incremental task learning. A proposed hybrid optimization method combines
a fast coarse optimization on a manifold of policy parameters with a fine-grained parameter
search in the unrestricted space of actions. It is shown that the developed algorithm reduces
the number of required rollouts for adaptation to new task conditions. Further, this work
presents a transfer learning approach for adaptation of learned skills to new situations.
Application in illustrative toy scenarios, for a 10-DOF planar arm, a humanoid robot point
reaching task and parameterized drumming on a pneumatic robot validate the approach.
But parameterized skills that are applied on complex robotic systems pose further
challenges: the dynamics of the robot and the interaction with the environment introduce
model inaccuracies. In particular, high-level skill acquisition on highly compliant robotic
systems such as pneumatically driven or soft actuators is hardly feasible. Since learning of
the complete dynamics model is not feasible due to the high complexity, this thesis examines
two alternative approaches: First, an improvement of the low-level control based on an
equilibrium model of the robot. Utilization of an equilibrium model reduces the learning
complexity and this thesis evaluates its applicability for control of pneumatic and industrial
light-weight robots. Second, an extension of parameterized skills to generalize for forward
signals of action primitives that result in an enhanced control quality of complex robotic
systems. This thesis argues for a shift in the complexity of learning the full dynamics of the
robot to a lower dimensional task-related learning problem. Due to the generalization in
relation to the task variability, online learning for complex robots as well as complex scenarios
becomes feasible. An experimental evaluation investigates the generalization capabilities of
the proposed online learning system for robot motion generation. Evaluation is performed
through simulation of a compliant 2-DOF arm and scalability to a complex robotic system
is demonstrated for a pneumatically driven humanoid robot with 8-DOF
