Deep Lagrangian Networks for end-to-end learning of energy-based control for under-actuated systems

Applying Deep Learning to control has a lot of potential for enabling the intelligent design of robot control laws. Unfortunately common deep learning approaches to control, such as deep reinforcement learning, require an unrealistic amount of interaction with the real system, do not yield any performance guarantees, and do not make good use of extensive insights from model-based control. In particular, common black-box approaches -- that abandon all insight from control -- are not suitable for complex robot systems. We propose a deep control approach as a bridge between the solid theoretical foundations of energy-based control and the flexibility of deep learning. To accomplish this goal, we extend Deep Lagrangian Networks (DeLaN) to not only adhere to Lagrangian Mechanics but also ensure conservation of energy and passivity of the learned representation. This novel extension is embedded within generic model-based control laws to enable energy control of under-actuated systems. The resulting DeLaN for energy control (DeLaN 4EC) is the first model learning approach using generic function approximation that is capable of learning energy control. DeLaN 4EC exhibits excellent real-time control on the physical Furuta Pendulum and learns to swing-up the pendulum while the control law using system identification does not.Comment: Published at IROS 201

Assessing Transferability from Simulation to Reality for Reinforcement Learning

Learning robot control policies from physics simulations is of great interest to the robotics community as it may render the learning process faster, cheaper, and safer by alleviating the need for expensive real-world experiments. However, the direct transfer of learned behavior from simulation to reality is a major challenge. Optimizing a policy on a slightly faulty simulator can easily lead to the maximization of the `Simulation Optimization Bias` (SOB). In this case, the optimizer exploits modeling errors of the simulator such that the resulting behavior can potentially damage the robot. We tackle this challenge by applying domain randomization, i.e., randomizing the parameters of the physics simulations during learning. We propose an algorithm called Simulation-based Policy Optimization with Transferability Assessment (SPOTA) which uses an estimator of the SOB to formulate a stopping criterion for training. The introduced estimator quantifies the over-fitting to the set of domains experienced while training. Our experimental results on two different second order nonlinear systems show that the new simulation-based policy search algorithm is able to learn a control policy exclusively from a randomized simulator, which can be applied directly to real systems without any additional training

09341 Abstracts Collection -- Cognition, Control and Learning for Robot Manipulation in Human Environments

From 16.08. to 21.08.2009, the Dagstuhl Seminar 09341 ``Cognition, Control and Learning for Robot Manipulation in Human Environments \u27\u27 was held in Schloss Dagstuhl~--~Leibniz Center for Informatics. During the seminar, several participants presented their current research, and ongoing work and open problems were discussed. Abstracts of the presentations given during the seminar as well as abstracts of seminar results and ideas are put together in this paper. The first section describes the seminar topics and goals in general. Links to extended abstracts or full papers are provided, if available

Characterization of Vertebrate Cohesin Complexes and Their Regulation in Prophase

In eukaryotes, sister chromatids remain connected from the time of their synthesis until they are separated in anaphase. This cohesion depends on a complex of proteins called cohesins. In budding yeast, the anaphase-promoting complex (APC) pathway initiates anaphase by removing cohesins from chromosomes. In vertebrates, cohesins dissociate from chromosomes already in prophase. To study their mitotic regulation we have purified two 14S cohesin complexes from human cells. Both complexes contain SMC1, SMC3, SCC1, and either one of the yeast Scc3p orthologs SA1 and SA2. SA1 is also a subunit of 14S cohesin in Xenopus. These complexes interact with PDS5, a protein whose fungal orthologs have been implicated in chromosome cohesion, condensation, and recombination. The bulk of SA1- and SA2-containing complexes and PDS5 are chromatin-associated until they become soluble from prophase to telophase. Reconstitution of this process in mitotic Xenopus extracts shows that cohesin dissociation does neither depend on cyclin B proteolysis nor on the presence of the APC. Cohesins can also dissociate from chromatin in the absence of cyclin-dependent kinase 1 activity. These results suggest that vertebrate cohesins are regulated by a novel prophase pathway which is distinct from the APC pathway that controls cohesins in yeast

Single cohesin molecules generate force by two distinct mechanisms

Spatial organization of DNA is facilitated by cohesin protein complexes that move on DNA and extrude DNA loops. How cohesin works mechanistically as a molecular machine is poorly understood. Here, we measure mechanical forces generated by conformational changes in single cohesin molecules. We show that bending of SMC coiled coils is driven by random thermal fluctuations leading to a ~32 nm head-hinge displacement that resists forces up to 1 pN; ATPase head engagement occurs in a single step of ~10 nm and is driven by an ATP dependent head-head movement, resisting forces up to 15 pN. Our molecular dynamic simulations show that the energy of head engagement can be stored in a mechanically strained conformation of NIPBL and released during disengagement. These findings reveal how single cohesin molecules generate force by two distinct mechanisms. We present a model, which proposes how this ability may power different aspects of cohesin-DNA interaction

Regulation of Human Separase by Securin Binding and Autocleavage

AbstractBackground: Sister chromatid separation is initiated by separase, a protease that cleaves cohesin and thereby dissolves sister chromatid cohesion. Separase is activated by the degradation of its inhibitor securin and by the removal of inhibitory phosphates. In human cells, separase activation also coincides with the cleavage of separase, but it is not known if this reaction activates separase, which protease cleaves separase, and how separase cleavage is regulated.Results: Inhibition of separase expression in human cells by RNA interference causes the formation of polyploid cells with large lobed nuclei. In mitosis, many of these cells contain abnormal chromosome plates with unseparated sister chromatids. Inhibitor binding experiments in vitro reveal that securin prevents the access of substrate analogs to the active site of separase. Upon securin degradation, the active site of full-length separase becomes accessible, allowing rapid autocatalytic cleavage of separase at one of three sites. The resulting N- and C-terminal fragments remain associated and can be reinhibited by securin. A noncleavable separase mutant retains its ability to cleave cohesin in vitro.Conclusions: Our results suggest that separase is required for sister chromatid separation during mitosis in human cells. Our data further indicate that securin inhibits separase by blocking the access of substrates to the active site of separase. Securin proteolysis allows autocatalytic processing of separase into a cleaved form, but separase cleavage is not essential for separase activation