Tracking Control for a Spherical Pendulum via Curriculum Reinforcement Learning

Reinforcement Learning (RL) allows learning non-trivial robot control laws purely from data. However, many successful applications of RL have relied on ad-hoc regularizations, such as hand-crafted curricula, to regularize the learning performance. In this paper, we pair a recent algorithm for automatically building curricula with RL on massively parallelized simulations to learn a tracking controller for a spherical pendulum on a robotic arm via RL. Through an improved optimization scheme that better respects the non-Euclidean task structure, we allow the method to reliably generate curricula of trajectories to be tracked, resulting in faster and more robust learning compared to an RL baseline that does not exploit this form of structured learning. The learned policy matches the performance of an optimal control baseline on the real system, demonstrating the potential of curriculum RL to jointly learn state estimation and control for non-linear tracking tasks

High Acceleration Reinforcement Learning for Real-World Juggling with Binary Rewards

Robots that can learn in the physical world will be important to enable robots to escape their stiff and pre-programmed movements. For dynamic high-acceleration tasks, such as juggling, learning in the real-world is particularly challenging as one must push the limits of the robot and its actuation without harming the system, amplifying the necessity of sample efficiency and safety for robot learning algorithms. In contrast to prior work which mainly focuses on the learning algorithm, we propose a learning system, that directly incorporates these requirements in the design of the policy representation, initialization, and optimization. We demonstrate that this system enables the high-speed Barrett WAM manipulator to learn juggling two balls from 56 minutes of experience with a binary reward signal and finally juggles continuously for up to 33 minutes or about 4500 repeated catches. The videos documenting the learning process and the evaluation can be found at https://sites.google.com/view/jugglingbo

SKID RAW: Skill Discovery From Raw Trajectories

Integrating robots in complex everyday environments requires a multitude of problems to be solved. One crucial feature among those is to equip robots with a mechanism for teaching them a new task in an easy and natural way. When teaching tasks that involve sequences of different skills, with varying order and number of these skills, it is desirable to only demonstrate full task executions instead of all individual skills. For this purpose, we propose a novel approach that simultaneously learns to segment trajectories into reoccurring patterns and the skills to reconstruct these patterns from unlabelled demonstrations without further supervision. Moreover, the approach learns a skill conditioning that can be used to understand possible sequences of skills, a practical mechanism to be used in, for example, human-robot-interactions for a more intelligent and adaptive robot behaviour. The Bayesian and variational inference based approach is evaluated on synthetic and real human demonstrations with varying complexities and dimensionality, showing the successful learning of segmentations and skill libraries from unlabelled data

Generierung von Laufmustern für einen elastisch angetriebenen Vierbeiner mit modal abgestimmten Beinen

Die Verwendung elastischer Elemente in Robotersystemen bietet die Möglichkeit, durch kurzzeitige Speicherung kinetischer Energie, hoch dynamische und energie-effiziente Bewegungsabläufe zu generieren. Dies trifft insbesondere auf zyklische Bewegungen, wie Hämmern, Gehen oder Rennen zu. In dieser Arbeit werden die dynamischen Eigenschaften eines am Deutschen Zentrum für Luft- und Raumfahrt (DLR) entwickelten vierbeinigen Laufroboters in Form einer modalen Analyse untersucht. Darauf aufbauend werden Regler zur Generierung diverser Laufmuster erarbeitet. Dazu werden zeitdiskret schaltende Zustandsmaschinen zum Erzeugen der Grundbewegung verwendet und parallel ein zeitkontinuierlicher Dämpfungsregler welcher in Kombination mit den physikalischen Federn des mechanischen Systems das Balancieren umsetzt. Die Validierung des Reglerkonzepts erfolgt in Simulation. Es stellt sich heraus, dass dieser Regelansatz für die Generierung hoch dynamischer Laufmuster geeignet ist, jedoch weniger für die Generierung langsamer Bewegungen

Dynamic Locomotion Gaits of a Compliantly Actuated Quadruped With SLIP-Like Articulated Legs Embodied in the Mechanical Design

The spring loaded inverted pendulum (SLIP) model has been extensively shown to be fundamental for legged locomotion. However, the way this low-order template model dynamics is anchored in high-dimensional articulated multibody systems describing compliantly actuated robots (and animals) is not obvious and has not been shown so far. In this letter, an articulated leg mechanism and a corresponding quadrupedal robot design are introduced, for which the natural oscillation dynamics is structurally equivalent to the SLIP. On the basis of this property, computationally simple and robust control methods are proposed, which implement the gaits of pronking, trotting, and dynamic walking in the real robotic system. Experiments with a compliantly actuated quadruped featuring only low-performance electrical drives validate the effectiveness of the proposed approach

STAT1 and STAT3 Exhibit a Crosstalk and Are Associated with Increased Inflammation in Hepatocellular Carcinoma

Liver cancers, which are mostly hepatocellular carcinoma (HCC) and cholangiocarcinoma (CCA), are very aggressive tumors with poor prognosis. Therapeutic options with curative intent are largely limited to surgery and available systemic therapies show limited benefit. Signal transducer and activator of transcription 1 (STAT1) and 3 (STAT3) are key transcription factors activated by pro-inflammatory cytokines such as interferon-γ (IFN-γ) and interleukin-6 (IL-6). In this study, we combined in vitro cell culture experiments and immunohistochemical analyses of human HCC (N = 124) and CCA (N = 138) specimens. We observed that in the absence of STAT3, IL-6 induced the activation of STAT1 and its target genes suggesting that IL-6 derived from the tumor microenvironment may activate both STAT1 and STAT3 target genes in HCC tumor cells. In addition, STAT1 and STAT3 were highly activated in a subset of HCC, which exhibited a high degree of infiltrating CD8- and FOXP3-positive immune cells and PD-L1 expression. Our results demonstrate that STAT1 and STAT3 are expressed and activated in HCC and tumor infiltrating immune cells. In addition, HCC cases with high STAT1 and STAT3 expression also exhibited a high degree of immune cell infiltration, suggesting increased immunological tolerance