One-Shot Learning of Manipulation Skills with Online Dynamics Adaptation and Neural Network Priors

One of the key challenges in applying reinforcement learning to complex robotic control tasks is the need to gather large amounts of experience in order to find an effective policy for the task at hand. Model-based reinforcement learning can achieve good sample efficiency, but requires the ability to learn a model of the dynamics that is good enough to learn an effective policy. In this work, we develop a model-based reinforcement learning algorithm that combines prior knowledge from previous tasks with online adaptation of the dynamics model. These two ingredients enable highly sample-efficient learning even in regimes where estimating the true dynamics is very difficult, since the online model adaptation allows the method to locally compensate for unmodeled variation in the dynamics. We encode the prior experience into a neural network dynamics model, adapt it online by progressively refitting a local linear model of the dynamics, and use model predictive control to plan under these dynamics. Our experimental results show that this approach can be used to solve a variety of complex robotic manipulation tasks in just a single attempt, using prior data from other manipulation behaviors

Spatio-Temporal Patterns act as Computational Mechanisms governing Emergent behavior in Robotic Swarms

open access articleOur goal is to control a robotic swarm without removing its swarm-like nature. In other words, we aim to intrinsically control a robotic swarm emergent behavior. Past attempts at governing robotic swarms or their selfcoordinating emergent behavior, has proven ineffective, largely due to the swarm’s inherent randomness (making it difficult to predict) and utter simplicity (they lack a leader, any kind of centralized control, long-range communication, global knowledge, complex internal models and only operate on a couple of basic, reactive rules). The main problem is that emergent phenomena itself is not fully understood, despite being at the forefront of current research. Research into 1D and 2D Cellular Automata has uncovered a hidden computational layer which bridges the micromacro gap (i.e., how individual behaviors at the micro-level influence the global behaviors on the macro-level). We hypothesize that there also lie embedded computational mechanisms at the heart of a robotic swarm’s emergent behavior. To test this theory, we proceeded to simulate robotic swarms (represented as both particles and dynamic networks) and then designed local rules to induce various types of intelligent, emergent behaviors (as well as designing genetic algorithms to evolve robotic swarms with emergent behaviors). Finally, we analysed these robotic swarms and successfully confirmed our hypothesis; analyzing their developments and interactions over time revealed various forms of embedded spatiotemporal patterns which store, propagate and parallel process information across the swarm according to some internal, collision-based logic (solving the mystery of how simple robots are able to self-coordinate and allow global behaviors to emerge across the swarm)

Adaptive Robot Framework: Providing Versatility and Autonomy to Manufacturing Robots Through FSM, Skills and Agents

207 p.The main conclusions that can be extracted from an analysis of the current situation and future trends of the industry,in particular manufacturing plants, are the following: there is a growing need to provide customization of products, ahigh variation of production volumes and a downward trend in the availability of skilled operators due to the ageingof the population. Adapting to this new scenario is a challenge for companies, especially small and medium-sizedenterprises (SMEs) that are suffering first-hand how their specialization is turning against them.The objective of this work is to provide a tool that can serve as a basis to face these challenges in an effective way.Therefore the presented framework, thanks to its modular architecture, allows focusing on the different needs of eachparticular company and offers the possibility of scaling the system for future requirements. The presented platform isdivided into three layers, namely: interface with robot systems, the execution engine and the application developmentlayer.Taking advantage of the provided ecosystem by this framework, different modules have been developed in order toface the mentioned challenges of the industry. On the one hand, to address the need of product customization, theintegration of tools that increase the versatility of the cell are proposed. An example of such tools is skill basedprogramming. By applying this technique a process can be intuitively adapted to the variations or customizations thateach product requires. The use of skills favours the reuse and generalization of developed robot programs.Regarding the variation of the production volumes, a system which permits a greater mobility and a faster reconfigurationis necessary. If in a certain situation a line has a production peak, mechanisms for balancing the loadwith a reasonable cost are required. In this respect, the architecture allows an easy integration of different roboticsystems, actuators, sensors, etc. In addition, thanks to the developed calibration and set-up techniques, the system canbe adapted to new workspaces at an effective time/cost.With respect to the third mentioned topic, an agent-based monitoring system is proposed. This module opens up amultitude of possibilities for the integration of auxiliary modules of protection and security for collaboration andinteraction between people and robots, something that will be necessary in the not so distant future.For demonstrating the advantages and adaptability improvement of the developed framework, a series of real usecases have been presented. In each of them different problematic has been resolved using developed skills,demonstrating how are adapted easily to the different casuistic

Intelligent Robots

There is an increasing need for integrating sensory feedback into the robot system. This will provide better flexibility and will improve the capacity of the robot to reason and make decisions in real time. This report discusses the current issues related to the development and application of intelligent robots. The report surveys the essential features of an intelligent robot. These features are sensing, off-line programming, task level programming, adaptive control and knowledge representation. Such a robot should be knowledge driven. It should know about objects and work plans, this knowledge should provide the capability for the robot to handle uncertainty in sensory data and to arbitrate between sensors in the event of conflicts

Vision-based Robotic Grasping in Simulation using Deep Reinforcement Learning

This thesis will investigate different robotic manipulation and grasping approaches. It will present an overview of robotic simulation environments, and offer an evaluation of PyBullet, CoppeliaSim, and Gazebo, comparing various features. The thesis further presents a background for current approaches to robotic manipulation and grasping by describing how the robotic movement and grasping can be organized. State-of-the-Art approaches for learning robotic grasping, both using supervised methods and reinforcement learning methods are presented. Two set of experiments will be conducted in PyBullet, illustrating how Deep Reinforcement Learning methods could be applied to train a 7 degrees of freedom robotic arm to grasp objects

Model checking learning agent systems using Promela with embedded C code and abstraction

As autonomous systems become more prevalent, methods for their verification will become more widely used. Model checking is a formal verification technique that can help ensure the safety of autonomous systems, but in most cases it cannot be applied by novices, or in its straight \off-the-shelf" form. In order to be more widely applicable it is crucial that more sophisticated techniques are used, and are presented in a way that is reproducible by engineers and verifiers alike. In this paper we demonstrate in detail two techniques that are used to increase the power of model checking using the model checker SPIN. The first of these is the use of embedded C code within Promela specifications, in order to accurately re ect robot movement. The second is to use abstraction together with a simulation relation to allow us to verify multiple environments simultaneously. We apply these techniques to a fairly simple system in which a robot moves about a fixed circular environment and learns to avoid obstacles. The learning algorithm is inspired by the way that insects learn to avoid obstacles in response to pain signals received from their antennae. Crucially, we prove that our abstraction is sound for our example system { a step that is often omitted but is vital if formal verification is to be widely accepted as a useful and meaningful approach

Towards a swarm robotic approach for cooperative object recognition

Social insects have inspired the behaviours of swarm robotic systems for the last 20 years. Interactions of the simple individuals in these swarms form solutions to relatively complex problems. A novel swarm robotic method is investigated for future robotic cooperative object recognition tasks. Previous multi-agent systems involve cameras and image analyses to identify objects. They cooperate only to improve their hypotheses of the shape's identity. The system proposed uses agents whose interactions with each other around the physical boundaries of the object's shape allow the distinguishing features found. The agents are a physical embodiment of the vision system, making them suitable for environments where it would not be possible to use a camera. A Simplified Hexagonal Model was developed to simulate and examine the strategies. The hexagonal cells of which can be empty, contain an agent (hBot) or part of an object shape. Initially the hBots are required to identify the valid object shapes from a set of two types of known shapes. To do this the hBots change state when in contact with an object and when touching other hBots of the same state level, where some states are only achieved when neighbouring certain object shapes. The agents are oblivious, anonymous and homogeneous. They also do not know their position or orientation and cannot distinguish between object shapes alone due to their limited sensor range. Further work increased the number of object shapes to provide a range of scenarios