Model-Based Control Using Koopman Operators

This paper explores the application of Koopman operator theory to the control of robotic systems. The operator is introduced as a method to generate data-driven models that have utility for model-based control methods. We then motivate the use of the Koopman operator towards augmenting model-based control. Specifically, we illustrate how the operator can be used to obtain a linearizable data-driven model for an unknown dynamical process that is useful for model-based control synthesis. Simulated results show that with increasing complexity in the choice of the basis functions, a closed-loop controller is able to invert and stabilize a cart- and VTOL-pendulum systems. Furthermore, the specification of the basis function are shown to be of importance when generating a Koopman operator for specific robotic systems. Experimental results with the Sphero SPRK robot explore the utility of the Koopman operator in a reduced state representation setting where increased complexity in the basis function improve open- and closed-loop controller performance in various terrains, including sand.Comment: 8 page

Biomechanics

Biomechanics is a vast discipline within the field of Biomedical Engineering. It explores the underlying mechanics of how biological and physiological systems move. It encompasses important clinical applications to address questions related to medicine using engineering mechanics principles. Biomechanics includes interdisciplinary concepts from engineers, physicians, therapists, biologists, physicists, and mathematicians. Through their collaborative efforts, biomechanics research is ever changing and expanding, explaining new mechanisms and principles for dynamic human systems. Biomechanics is used to describe how the human body moves, walks, and breathes, in addition to how it responds to injury and rehabilitation. Advanced biomechanical modeling methods, such as inverse dynamics, finite element analysis, and musculoskeletal modeling are used to simulate and investigate human situations in regard to movement and injury. Biomechanical technologies are progressing to answer contemporary medical questions. The future of biomechanics is dependent on interdisciplinary research efforts and the education of tomorrow’s scientists

Prediction of stable walking for a toy that cannot stand

Previous experiments [M. J. Coleman and A. Ruina, Phys. Rev. Lett. 80, 3658 (1998)] showed that a gravity-powered toy with no control and which has no statically stable near-standing configurations can walk stably. We show here that a simple rigid-body statically-unstable mathematical model based loosely on the physical toy can predict stable limit-cycle walking motions. These calculations add to the repertoire of rigid-body mechanism behaviors as well as further implicating passive-dynamics as a possible contributor to stability of animal motions.Comment: Note: only corrections so far have been fixing typo's in these comments. 3 pages, 2 eps figures, uses epsf.tex, revtex.sty, amsfonts.sty, aps.sty, aps10.sty, prabib.sty; Accepted for publication in Phys. Rev. E. 4/9/2001 ; information about Andy Ruina's lab (including Coleman's, Garcia's and Ruina's other publications and associated video clips) can be found at: http://www.tam.cornell.edu/~ruina/hplab/index.html and more about Georg Bock's Simulation Group with whom Katja Mombaur is affiliated can be found at http://www.iwr.uni-heidelberg.de/~agboc

Benchmarking Deep Reinforcement Learning for Continuous Control

Recently, researchers have made significant progress combining the advances in deep learning for learning feature representations with reinforcement learning. Some notable examples include training agents to play Atari games based on raw pixel data and to acquire advanced manipulation skills using raw sensory inputs. However, it has been difficult to quantify progress in the domain of continuous control due to the lack of a commonly adopted benchmark. In this work, we present a benchmark suite of continuous control tasks, including classic tasks like cart-pole swing-up, tasks with very high state and action dimensionality such as 3D humanoid locomotion, tasks with partial observations, and tasks with hierarchical structure. We report novel findings based on the systematic evaluation of a range of implemented reinforcement learning algorithms. Both the benchmark and reference implementations are released at https://github.com/rllab/rllab in order to facilitate experimental reproducibility and to encourage adoption by other researchers.Comment: 14 pages, ICML 201

Musculoskeletal modelling of an ostrich (Struthio camelus) pelvic limb: influence of limb orientation on muscular capacity during locomotion

We developed a three-dimensional, biomechanical computer model of the 36 major pelvic limb muscle groups in an ostrich (Struthio camelus) to investigate muscle function in this, the largest of extant birds and model organism for many studies of locomotor mechanics, body size, anatomy and evolution. Combined with experimental data, we use this model to test two main hypotheses. We first query whether ostriches use limb orientations (joint angles) that optimize the moment-generating capacities of their muscles during walking or running. Next, we test whether ostriches use limb orientations at mid-stance that keep their extensor muscles near maximal, and flexor muscles near minimal, moment arms. Our two hypotheses relate to the control priorities that a large bipedal animal might evolve under biomechanical constraints to achieve more effective static weight support. We find that ostriches do not use limb orientations to optimize the moment-generating capacities or moment arms of their muscles. We infer that dynamic properties of muscles or tendons might be better candidates for locomotor optimization. Regardless, general principles explaining why species choose particular joint orientations during locomotion are lacking, raising the question of whether such general principles exist or if clades evolve different patterns (e.g., weighting of muscle force–length or force–velocity properties in selecting postures). This leaves theoretical studies of muscle moment arms estimated for extinct animals at an impasse until studies of extant taxa answer these questions. Finally, we compare our model’s results against those of two prior studies of ostrich limb muscle moment arms, finding general agreement for many muscles. Some flexor and extensor muscles exhibit self-stabilization patterns (posture-dependent switches between flexor/extensor action) that ostriches may use to coordinate their locomotion. However, some conspicuous areas of disagreement in our results illustrate some cautionary principles. Importantly, tendon-travel empirical measurements of muscle moment arms must be carefully designed to preserve 3D muscle geometry lest their accuracy suffer relative to that of anatomically realistic models. The dearth of accurate experimental measurements of 3D moment arms of muscles in birds leaves uncertainty regarding the relative accuracy of different modelling or experimental datasets such as in ostriches. Our model, however, provides a comprehensive set of 3D estimates of muscle actions in ostriches for the first time, emphasizing that avian limb mechanics are highly three-dimensional and complex, and how no muscles act purely in the sagittal plane. A comparative synthesis of experiments and models such as ours could provide powerful synthesis into how anatomy, mechanics and control interact during locomotion and how these interactions evolve. Such a framework could remove obstacles impeding the analysis of muscle function in extinct taxa