Adaptive swing-up and balancing control of acrobot systems

Thesis (S.B.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2009.Cataloged from PDF version of thesis.Includes bibliographical references (p. 22).The field of underactuated robotics has become the core of agile mobile robotics research. Significant past effort has been put into understanding the swing-up control of the acrobot system. This thesis implements an online, adaptive swing-up and balancing controller with no previous knowledge of the system's mass or geometric parameters. A least squares method is used to identify the 5 parameters necessary to completely characterize acrobot dynamics. Swing up is accomplished using partial feedback linearization and a pump up strategy to add energy to the system. The controller then catches the swung up system in the basin of attraction of an LQR controller computed using the estimated parameter values generated from online system identification. These results are then simulated using a MATLAB simulation environment.by Luke B. Johnson.S.B

MATLAB-based Tools for Modelling and Control of Underactuated Mechanical Systems

Underactuated systems, defined as nonlinear mechanical systems with fewer control inputs than degrees of freedom, appear in a broad range of applications including robotics, aerospace, marine and locomotive systems. Studying the complex low-order nonlinear dynamics of appropriate benchmark underactuated systems often enables us to gain insight into the principles of modelling and control of advanced, higher-order underactuated systems. Such benchmarks include the Acrobot, Pendubot and the reaction (inertia) wheel pendulum. The aim of this paper is to introduce novel MATLAB-based tools which were developed to provide complex software support for modelling and control of these three benchmark systems. The presented tools include a Simulink block library, a set of demo simulation schemes and several innovative functions for mathematical and simulation model generation

Polynomial mechanics and optimal control

We describe a new algorithm for trajectory optimization of mechanical systems. Our method combines pseudo-spectral methods for function approximation with variational discretization schemes that exactly preserve conserved mechanical quantities such as momentum. We thus obtain a global discretization of the Lagrange-d'Alembert variational principle using pseudo-spectral methods. Our proposed scheme inherits the numerical convergence characteristics of spectral methods, yet preserves momentum-conservation and symplecticity after discretization. We compare this algorithm against two other established methods for two examples of underactuated mechanical systems; minimum-effort swing-up of a two-link and a three-link acrobot.Comment: Final version to EC

Normal forms for underactuated mechanical systems with symmetry

We introduce cascade normal forms for underactuated mechanical systems that are convenient for control design. These normal forms include three classes of cascade systems, namely, nonlinear systems in strict feedback form, feedforward form, and nontriangular quadratic form (to be defined). In each case, the transformation to cascade systems is provided in closed-form. We apply our results to the Acrobot, the rotating pendulum, and the cart-pole system

Empowerment for Continuous Agent-Environment Systems

This paper develops generalizations of empowerment to continuous states. Empowerment is a recently introduced information-theoretic quantity motivated by hypotheses about the efficiency of the sensorimotor loop in biological organisms, but also from considerations stemming from curiosity-driven learning. Empowemerment measures, for agent-environment systems with stochastic transitions, how much influence an agent has on its environment, but only that influence that can be sensed by the agent sensors. It is an information-theoretic generalization of joint controllability (influence on environment) and observability (measurement by sensors) of the environment by the agent, both controllability and observability being usually defined in control theory as the dimensionality of the control/observation spaces. Earlier work has shown that empowerment has various interesting and relevant properties, e.g., it allows us to identify salient states using only the dynamics, and it can act as intrinsic reward without requiring an external reward. However, in this previous work empowerment was limited to the case of small-scale and discrete domains and furthermore state transition probabilities were assumed to be known. The goal of this paper is to extend empowerment to the significantly more important and relevant case of continuous vector-valued state spaces and initially unknown state transition probabilities. The continuous state space is addressed by Monte-Carlo approximation; the unknown transitions are addressed by model learning and prediction for which we apply Gaussian processes regression with iterated forecasting. In a number of well-known continuous control tasks we examine the dynamics induced by empowerment and include an application to exploration and online model learning

Benchmarking Deep Reinforcement Learning for Continuous Control

Recently, researchers have made significant progress combining the advances in deep learning for learning feature representations with reinforcement learning. Some notable examples include training agents to play Atari games based on raw pixel data and to acquire advanced manipulation skills using raw sensory inputs. However, it has been difficult to quantify progress in the domain of continuous control due to the lack of a commonly adopted benchmark. In this work, we present a benchmark suite of continuous control tasks, including classic tasks like cart-pole swing-up, tasks with very high state and action dimensionality such as 3D humanoid locomotion, tasks with partial observations, and tasks with hierarchical structure. We report novel findings based on the systematic evaluation of a range of implemented reinforcement learning algorithms. Both the benchmark and reference implementations are released at https://github.com/rllab/rllab in order to facilitate experimental reproducibility and to encourage adoption by other researchers.Comment: 14 pages, ICML 201

Producing Periodic Motion for Underactuated Systems

This project considers a special class of aero-dynamics of two different underactuated systems. The two systems under consideration are the Pendubot (actuator at the hip) and the Acrobot (actuator at the knee). The main objectives are to show stability and periodicity of the two systems. The next step is to change the zero-dynamics such that resulting periodic orbits produce walking gait patterns. The stability analysis involves searching for a Lyapunov function which would prove stability and in turn, as can be shown, imply periodicty. The Lyapunov function would also be a powerful help in knowing how to change the zero-dynamics so that desired motions are obtained. However, a Lyapunov function could not be found, which implies that the succeeding steps are hard to make. We present in this work some conclusions that would follow from a Lyapunov function and highlight some properties of the systems. We also show how the Pendubot could be brought to upright position