Hierarchical Decomposition of Nonlinear Dynamics and Control for System Identification and Policy Distillation

The control of nonlinear dynamical systems remains a major challenge for autonomous agents. Current trends in reinforcement learning (RL) focus on complex representations of dynamics and policies, which have yielded impressive results in solving a variety of hard control tasks. However, this new sophistication and extremely over-parameterized models have come with the cost of an overall reduction in our ability to interpret the resulting policies. In this paper, we take inspiration from the control community and apply the principles of hybrid switching systems in order to break down complex dynamics into simpler components. We exploit the rich representational power of probabilistic graphical models and derive an expectation-maximization (EM) algorithm for learning a sequence model to capture the temporal structure of the data and automatically decompose nonlinear dynamics into stochastic switching linear dynamical systems. Moreover, we show how this framework of switching models enables extracting hierarchies of Markovian and auto-regressive locally linear controllers from nonlinear experts in an imitation learning scenario.Comment: 2nd Annual Conference on Learning for Dynamics and Contro

Revisiting the Complexity of Stability of Continuous and Hybrid Systems

We develop a framework to give upper bounds on the "practical" computational complexity of stability problems for a wide range of nonlinear continuous and hybrid systems. To do so, we describe stability properties of dynamical systems using first-order formulas over the real numbers, and reduce stability problems to the delta-decision problems of these formulas. The framework allows us to obtain a precise characterization of the complexity of different notions of stability for nonlinear continuous and hybrid systems. We prove that bounded versions of the stability problems are generally decidable, and give upper bounds on their complexity. The unbounded versions are generally undecidable, for which we give upper bounds on their degrees of unsolvability

Sensitivity analysis of hybrid systems with state jumps with application to trajectory tracking

This paper addresses the sensitivity analysis for hybrid systems with discontinuous (jumping) state trajectories. We consider state-triggered jumps in the state evolution, potentially accompanied by mode switching in the control vector field as well. For a given trajectory with state jumps, we show how to construct an approximation of a nearby perturbed trajectory corresponding to a small variation of the initial condition and input. A major complication in the construction of such an approximation is that, in general, the jump times corresponding to a nearby perturbed trajectory are not equal to those of the nominal one. The main contribution of this work is the development of a notion of error to clarify in which sense the approximate trajectory is, at each instant of time, a firstorder approximation of the perturbed trajectory. This notion of error naturally finds application in the (local) tracking problem of a time-varying reference trajectory of a hybrid system. To illustrate the possible use of this new error definition in the context of trajectory tracking, we outline how the standard linear trajectory tracking control for nonlinear systems -based on linear quadratic regulator (LQR) theory to compute the optimal feedback gain- could be generalized for hybrid systems

ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Robotics

Physical simulators have been widely used in robot planning and control. Among them, differentiable simulators are particularly favored, as they can be incorporated into gradient-based optimization algorithms that are efficient in solving inverse problems such as optimal control and motion planning. Simulating deformable objects is, however, more challenging compared to rigid body dynamics. The underlying physical laws of deformable objects are more complex, and the resulting systems have orders of magnitude more degrees of freedom and therefore they are significantly more computationally expensive to simulate. Computing gradients with respect to physical design or controller parameters is typically even more computationally challenging. In this paper, we propose a real-time, differentiable hybrid Lagrangian-Eulerian physical simulator for deformable objects, ChainQueen, based on the Moving Least Squares Material Point Method (MLS-MPM). MLS-MPM can simulate deformable objects including contact and can be seamlessly incorporated into inference, control and co-design systems. We demonstrate that our simulator achieves high precision in both forward simulation and backward gradient computation. We have successfully employed it in a diverse set of control tasks for soft robots, including problems with nearly 3,000 decision variables.Comment: In submission to ICRA 2019. Supplemental Video: https://www.youtube.com/watch?v=4IWD4iGIsB4 Project Page: https://github.com/yuanming-hu/ChainQuee

Using a 3DOF Parallel Robot and a Spherical Bat to hit a Ping-Pong Ball

Playing the game of Ping-Pong is a challenge to human abilities since it requires developing skills, such as fast reaction capabilities, precision of movement and high speed mental responses. These processes include the utilization of seven DOF of the human arm, and translational movements through the legs, torso, and other extremities of the body, which are used for developing different game strategies or simply imposing movements that affect the ball such as spinning movements. Computationally, Ping-Pong requires a huge quantity of joints and visual information to be processed and analysed, something which really represents a challenge for a robot. In addition, in order for a robot to develop the task mechanically, it requires a large and dexterous workspace, and good dynamic capacities. Although there are commercial robots that are able to play Ping-Pong, the game is still an open task, where there are problems to be solved and simplified. All robotic Ping-Pong players cited in the bibliography used at least four DOF to hit the ball. In this paper, a spherical bat mounted on a 3-DOF parallel robot is proposed. The spherical bat is used to drive the trajectory of a Ping-Pong ball.Fil: Trasloheros, Alberto. Universidad Aeronáutica de Querétaro; MéxicoFil: Sebastián, José María. Universidad Politécnica de Madrid; España. Consejo Superior de Investigaciones Científicas; EspañaFil: Torrijos, Jesús. Consejo Superior de Investigaciones Científicas; España. Universidad Politécnica de Madrid; EspañaFil: Carelli Albarracin, Ricardo Oscar. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - San Juan. Instituto de Automática. Universidad Nacional de San Juan. Facultad de Ingeniería. Instituto de Automática; ArgentinaFil: Roberti, Flavio. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - San Juan. Instituto de Automática. Universidad Nacional de San Juan. Facultad de Ingeniería. Instituto de Automática; Argentin