On the Entanglement between Evolvability and Fitness: an Experimental Study on Voxel-based Soft Robots

The concept of evolvability, that is the capacity to produce heritable and adaptive phenotypic variation, is crucial in the current understanding of evolution. However, while its meaning is intuitive, there is no consensus on how to quantitatively measure it. As a consequence, it is hard to evaluate the interplay between evolvability and fitness and its dependency on key factors like the evolutionary algorithm (EA) or the representation of the individuals. Here, we propose to use MAP-Elites, a well-established Quality Diversity EA, as a support structure for measuring evolvability and for highlighting its interplay with fitness. We map the solutions generated during the evolutionary process to a MAP-Elites-like grid and then visualize their fitness and evolvability as maps. This procedures does not affect the EA execution and can hence be applied to any EA: it only requires to have two descriptors for the solutions that can be used to meaningfully characterize them. We apply this general methodology to the case of Voxel-based Soft Robots, a kind of modular robots with a body composed of uniform elements whose volume is individually varied by the robot brain. Namely, we optimize the robots for the task of locomotion using evolutionary computation. We consider four representations, two for the brain only and two for both body and brain of the VSR, and two EAs (MAP-Elites and a simple evolutionary strategy) and examine the evolvability and fitness maps. The experiments suggest that our methodology permits to discover interesting patterns in the maps: fitness maps appear to depend more on the representation of the solution, whereas evolvability maps appear to depend more on the EA. As an aside, we find that MAP-Elites is particularly effective in the simultaneous evolution of the body and the brain of Voxel-based Soft Robots

Constrained DMPs for Feasible Skill Learning on Humanoid Robots

In the context of humanoid skill learning, movement primitives have gained much attention because of their compact representation and convenient combination with a myriad of optimization approaches. Among them, a well-known scheme is to use Dynamic Movement Primitives (DMPs) with reinforcement learning (RL) algorithms. While various remarkable results have been reported, skill learning with physical constraints has not been sufficiently investigated. For example, when RL is employed to optimize the robot joint trajectories, the exploration noise could drive the resulting trajectory out of the joint limits. In this paper, we focus on robot skill learning characterized by joint limit avoidance, by introducing the novel Constrained Dynamic Movement Primitives (CDMPs). By controlling a set of transformed states (called exogenous states) instead of the original DMPs states, CDMPs are capable of maintaining the joint trajectories within the safety limits. We validate CDMPs on the humanoid robot iCub, showing the applicability of our approach

Learning to Avoid Obstacles With Minimal Intervention Control

Programming by demonstration has received much attention as it offers a general framework which allows robots to efficiently acquire novel motor skills from a human teacher. While traditional imitation learning that only focuses on either Cartesian or joint space might become inappropriate in situations where both spaces are equally important (e.g., writing or striking task), hybrid imitation learning of skills in both Cartesian and joint spaces simultaneously has been studied recently. However, an important issue which often arises in dynamical or unstructured environments is overlooked, namely how can a robot avoid obstacles? In this paper, we aim to address the problem of avoiding obstacles in the context of hybrid imitation learning. Specifically, we propose to tackle three subproblems: (i) designing a proper potential field so as to bypass obstacles, (ii) guaranteeing joint limits are respected when adjusting trajectories in the process of avoiding obstacles, and (iii) determining proper control commands for robots such that potential human-robot interaction is safe. By solving the aforementioned subproblems, the robot is capable of generalizing observed skills to new situations featuring obstacles in a feasible and safe manner. The effectiveness of the proposed method is validated through a toy example as well as a real transportation experiment on the iCub humanoid robot

Learning to Sequence Multiple Tasks with Competing Constraints

Imitation learning offers a general framework where robots can efficiently acquire novel motor skills from demonstrations of a human teacher. While many promising achievements have been shown, the majority of them are only focused on single-stroke movements, without taking into account the problem of multi-tasks sequencing. Conceivably, sequencing different atomic tasks can further augment the robot's capabilities as well as avoid repetitive demonstrations. In this paper, we propose to address the issue of multi-tasks sequencing with emphasis on handling the so-called competing constraints, which emerge due to the existence of the concurrent constraints from Cartesian and joint trajectories. Specifically, we explore the null space of the robot from an information-theoretic perspective in order to maintain imitation fidelity during transition between consecutive tasks. The effectiveness of the proposed method is validated through simulated and real experiments on the iCub humanoid robot

On the emergence of whole-body strategies from humanoid robot push-recovery learning

none7Balancing and push-recovery are essential capabilities enabling humanoid robots to solve complex locomotion tasks. In this context, classical control systems tend to be based on simplified physical models and hard-coded strategies. Although successful in specific scenarios, this approach requires demanding tuning of parameters and switching logic between specifically-designed controllers for handling more general perturbations. We apply model-free Deep Reinforcement Learning for training a general and robust humanoid push-recovery policy in a simulation environment. Our method targets high-dimensional whole-body humanoid control and is validated on the iCub humanoid. Reward components incorporating expert knowledge on humanoid control enable fast learning of several robust behaviors by the same policy, spanning the entire body. We validate our method with extensive quantitative analyses in simulation, including out-of-sample tasks which demonstrate policy robustness and generalization, both key requirements towards real-world robot deployment.openFerigo D.; Camoriano R.; Viceconte P.M.; Calandriello D.; Traversaro S.; Rosasco L.; Pucci D.Ferigo, D.; Camoriano, R.; Viceconte, P. M.; Calandriello, D.; Traversaro, S.; Rosasco, L.; Pucci, D

Constrained DMPs for Feasible Skill Learning on Humanoid Robots

In the context of humanoid skill learning, movement primitives have gained much attention because of their compact representation and convenient combination with a myriad of optimization approaches. Among them, a well-known scheme is to use Dynamic Movement Primitives (DMPs) with reinforcement learning (RL) algorithms. While various remarkable results have been reported, skill learning with physical constraints has not been sufficiently investigated. For example, when RL is employed to optimize the robot joint trajectories, the exploration noise could drive the resulting trajectory out of the joint limits. In this paper, we focus on robot skill learning characterized by joint limit avoidance, by introducing the novel Constrained Dynamic Movement Primitives (CDMPs). By controlling a set of transformed states (called exogenous states) instead of the original DMPs states, CDMPs are capable of maintaining the joint trajectories within the safety limits. We validate CDMPs on the humanoid robot iCub, showing the applicability of our approach

Learning to Sequence Multiple Tasks with Competing Constraints

Imitation learning offers a general framework where robots can efficiently acquire novel motor skills from demonstrations of a human teacher. While many promising achievements have been shown, the majority of them are only focused on single-stroke movements, without taking into account the problem of multi-tasks sequencing. Conceivably, sequencing different atomic tasks can further augment the robot's capabilities as well as avoid repetitive demonstrations. In this paper, we propose to address the issue of multi-tasks sequencing with emphasis on handling the so-called competing constraints, which emerge due to the existence of the concurrent constraints from Cartesian and joint trajectories. Specifically, we explore the null space of the robot from an information-theoretic perspective in order to maintain imitation fidelity during transition between consecutive tasks. The effectiveness of the proposed method is validated through simulated and real experiments on the iCub humanoid robot

Efficacy of pregabalin in a case of stiff-person syndrome: Clinical and neurophysiological evidence.

Symptomatic treatment of stiff-person syndrome (SPS) might be challenging and a significant improvement of stiffness and rigidity is generally reached with high doses of benzodiazepines or baclofen causing side effects. A 71-year old woman diagnosed with SPS complained of marked stiffness of trunk and lower limb muscles with sudden painful spasms. She was unable to walk and she could not lean on her right leg. Cortical silent period (CSP) duration evaluated from right abductor pollicis brevis (APB) with transcranial magnetic stimulation was shortened. Polygraphic electromyographic (EMG) evaluation from paraspinal and leg muscles disclosed continuous motor unit activity at rest with interference muscular pattern. Symptomatic treatment with diazepam was withdrawn because of excessive sedation. In order to relieve the intense lumbar pain, she was prescribed pregabalin; since the day after, rigidity and painful spasms dramatically improved and she could walk without assistance. The clinical benefit persisted at 3months follow-up and was paralleled by almost complete disappearance of EMG activity at rest and prolongation of CSP. The clinical and electrophysiological data in this SPS patient suggest the possible efficacy of pregabalin as symptomatic treatment without any significant side effects, which needs to be replicated in larger case series