Adaptive Control of 4-DoF Robot manipulator

In experimental robotics, researchers may face uncertainties in parameters of a robot manipulator that they are working with. This uncertainty may be caused by deviations in the manufacturing process of a manipulator, or changes applied to manipulator in the lab for sake of experiments. Another situation when dynamical and inertial parameters of a robot are uncertain arises, is the grasping of objects by a manipulator. In all these situations there is a need for adaptive control strategies that would identify changes in dynamical properties of manipulator and adjust for them. This article presents a work on designing of an adaptive control strategy for 4-DoF manipulator with uncertain dynamical properties, and outcomes of testing of this strategy applied to control of simulator of robot.Comment: 7 pages, 4(5) figure

Rigid vs compliant contact: An experimental study on biped walking

Contact modeling plays a central role in motion planning, simulation, and control of legged robots, as legged locomotion is realized through contact. The two prevailing approaches to model the contact consider rigid and compliant premise at interaction ports. Contrary to the dynamics model of legged systems with rigid contact (without impact) which is straightforward to develop, there is no consensus among researchers to employ a standard compliant contact model. Our main goal in this paper is to study the dynamics model structure of bipedal walking systems with a rigid contact and a \textit{novel} compliant contact model and to present experimental validation of both models. For the model with rigid contact, after developing the model of the articulated bodies in flight phase without any contact with the environment, we apply the holonomic constraints at contact points and develop a constrained dynamics model of the robot in both single and double support phases. For the model with compliant contact, we propose a novel nonlinear contact model and simulate the motion of the robot using this model. In order to show the performance of the developed models, we compare obtained results from these models to the empirical measurements from bipedal walking of the human-sized humanoid robot SURENA III, which has been designed and fabricated at CAST, University of Tehran. This analysis shows the merit of both models in estimating dynamic behavior of the robot walking on a semi-rigid surface. The model with rigid contact, which is less complex and independent of the physical properties of the contacting bodies, can be employed for model-based motion optimization, analysis as well as control, while the model with compliant contact and more complexity is suitable for more realistic simulation scenarios

A Tour of Reinforcement Learning: The View from Continuous Control

This manuscript surveys reinforcement learning from the perspective of optimization and control with a focus on continuous control applications. It surveys the general formulation, terminology, and typical experimental implementations of reinforcement learning and reviews competing solution paradigms. In order to compare the relative merits of various techniques, this survey presents a case study of the Linear Quadratic Regulator (LQR) with unknown dynamics, perhaps the simplest and best-studied problem in optimal control. The manuscript describes how merging techniques from learning theory and control can provide non-asymptotic characterizations of LQR performance and shows that these characterizations tend to match experimental behavior. In turn, when revisiting more complex applications, many of the observed phenomena in LQR persist. In particular, theory and experiment demonstrate the role and importance of models and the cost of generality in reinforcement learning algorithms. This survey concludes with a discussion of some of the challenges in designing learning systems that safely and reliably interact with complex and uncertain environments and how tools from reinforcement learning and control might be combined to approach these challenges.Comment: minor revision with a few clarifying passages and corrected typo

Learning Agile Robotic Locomotion Skills by Imitating Animals

Reproducing the diverse and agile locomotion skills of animals has been a longstanding challenge in robotics. While manually-designed controllers have been able to emulate many complex behaviors, building such controllers involves a time-consuming and difficult development process, often requiring substantial expertise of the nuances of each skill. Reinforcement learning provides an appealing alternative for automating the manual effort involved in the development of controllers. However, designing learning objectives that elicit the desired behaviors from an agent can also require a great deal of skill-specific expertise. In this work, we present an imitation learning system that enables legged robots to learn agile locomotion skills by imitating real-world animals. We show that by leveraging reference motion data, a single learning-based approach is able to automatically synthesize controllers for a diverse repertoire behaviors for legged robots. By incorporating sample efficient domain adaptation techniques into the training process, our system is able to learn adaptive policies in simulation that can then be quickly adapted for real-world deployment. To demonstrate the effectiveness of our system, we train an 18-DoF quadruped robot to perform a variety of agile behaviors ranging from different locomotion gaits to dynamic hops and turns

Dynamically Stable 3D Quadrupedal Walking with Multi-Domain Hybrid System Models and Virtual Constraint Controllers

Hybrid systems theory has become a powerful approach for designing feedback controllers that achieve dynamically stable bipedal locomotion, both formally and in practice. This paper presents an analytical framework 1) to address multi-domain hybrid models of quadruped robots with high degrees of freedom, and 2) to systematically design nonlinear controllers that asymptotically stabilize periodic orbits of these sophisticated models. A family of parameterized virtual constraint controllers is proposed for continuous-time domains of quadruped locomotion to regulate holonomic and nonholonomic outputs. The properties of the Poincare return map for the full-order and closed-loop hybrid system are studied to investigate the asymptotic stabilization problem of dynamic gaits. An iterative optimization algorithm involving linear and bilinear matrix inequalities is then employed to choose stabilizing virtual constraint parameters. The paper numerically evaluates the analytical results on a simulation model of an advanced 3D quadruped robot, called GR Vision 60, with 36 state variables and 12 control inputs. An optimal amble gait of the robot is designed utilizing the FROST toolkit. The power of the analytical framework is finally illustrated through designing a set of stabilizing virtual constraint controllers with 180 controller parameters.Comment: American Control Conference 201

Safe Robotic Grasping: Minimum Impact-Force Grasp Selection

This paper addresses the problem of selecting from a choice of possible grasps, so that impact forces will be minimised if a collision occurs while the robot is moving the grasped object along a post-grasp trajectory. Such considerations are important for safety in human-robot interaction, where even a certified "human-safe" (e.g. compliant) arm may become hazardous once it grasps and begins moving an object, which may have significant mass, sharp edges or other dangers. Additionally, minimising collision forces is critical to preserving the longevity of robots which operate in uncertain and hazardous environments, e.g. robots deployed for nuclear decommissioning, where removing a damaged robot from a contaminated zone for repairs may be extremely difficult and costly. Also, unwanted collisions between a robot and critical infrastructure (e.g. pipework) in such high-consequence environments can be disastrous. In this paper, we investigate how the safety of the post-grasp motion can be considered during the pre-grasp approach phase, so that the selected grasp is optimal in terms applying minimum impact forces if a collision occurs during a desired post-grasp manipulation. We build on the methods of augmented robot-object dynamics models and "effective mass" and propose a method for combining these concepts with modern grasp and trajectory planners, to enable the robot to achieve a grasp which maximises the safety of the post-grasp trajectory, by minimising potential collision forces. We demonstrate the effectiveness of our approach through several experiments with both simulated and real robots.Comment: To be appeared in IEEE/RAS IROS 201

Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning

Although reinforcement learning methods can achieve impressive results in simulation, the real world presents two major challenges: generating samples is exceedingly expensive, and unexpected perturbations or unseen situations cause proficient but specialized policies to fail at test time. Given that it is impractical to train separate policies to accommodate all situations the agent may see in the real world, this work proposes to learn how to quickly and effectively adapt online to new tasks. To enable sample-efficient learning, we consider learning online adaptation in the context of model-based reinforcement learning. Our approach uses meta-learning to train a dynamics model prior such that, when combined with recent data, this prior can be rapidly adapted to the local context. Our experiments demonstrate online adaptation for continuous control tasks on both simulated and real-world agents. We first show simulated agents adapting their behavior online to novel terrains, crippled body parts, and highly-dynamic environments. We also illustrate the importance of incorporating online adaptation into autonomous agents that operate in the real world by applying our method to a real dynamic legged millirobot. We demonstrate the agent's learned ability to quickly adapt online to a missing leg, adjust to novel terrains and slopes, account for miscalibration or errors in pose estimation, and compensate for pulling payloads.Comment: First 2 authors contributed equally. Website: https://sites.google.com/berkeley.edu/metaadaptivecontro

Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images

We introduce Embed to Control (E2C), a method for model learning and control of non-linear dynamical systems from raw pixel images. E2C consists of a deep generative model, belonging to the family of variational autoencoders, that learns to generate image trajectories from a latent space in which the dynamics is constrained to be locally linear. Our model is derived directly from an optimal control formulation in latent space, supports long-term prediction of image sequences and exhibits strong performance on a variety of complex control problems.Comment: Final NIPS versio

Experimental comparison of control strategies for trajectory tracking for mobile robots

The purpose of this paper is to implement, test and compare the performance of different control strategies for tracking trajectory for mobile robots. The control strategies used are based on linear algebra, PID controller and on a sliding mode controller. Each control scheme is developed taking into consideration the model of the robot. The linear algebra approaches take into account the complete kinematic model of the robot; and the PID and the sliding mode controller use a reduced order model, which is obtained considering the mobile robot platform as a black-box. All the controllers are tested and compared, firstly by simulations and then, by using a Pioneer 3DX robot in field experiments.Fil: Capito, Linda. Escuela Politécnica Nacional; EcuadorFil: Proaño, Pablo. Escuela Politécnica Nacional; EcuadorFil: Camacho, Oscar. Escuela Politécnica Nacional; EcuadorFil: Rosales, Andrés. Escuela Politécnica Nacional; EcuadorFil: Scaglia, Gustavo Juan Eduardo. Universidad Nacional de San Juan. Facultad de Ingeniería. Instituto de Ingeniería Química; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - San Juan; Argentin

Mathematical Analysis of Multi-Agent Systems

We review existing approaches to mathematical modeling and analysis of multi-agent systems in which complex collective behavior arises out of local interactions between many simple agents. Though the behavior of an individual agent can be considered to be stochastic and unpredictable, the collective behavior of such systems can have a simple probabilistic description. We show that a class of mathematical models that describe the dynamics of collective behavior of multi-agent systems can be written down from the details of the individual agent controller. The models are valid for Markov or memoryless agents, in which each agents future state depends only on its present state and not any of the past states. We illustrate the approach by analyzing in detail applications from the robotics domain: collaboration and foraging in groups of robots.Comment: latex, 15 figures, 42 page