Evolutionary Algorithms for Reinforcement Learning

There are two distinct approaches to solving reinforcement learning problems, namely, searching in value function space and searching in policy space. Temporal difference methods and evolutionary algorithms are well-known examples of these approaches. Kaelbling, Littman and Moore recently provided an informative survey of temporal difference methods. This article focuses on the application of evolutionary algorithms to the reinforcement learning problem, emphasizing alternative policy representations, credit assignment methods, and problem-specific genetic operators. Strengths and weaknesses of the evolutionary approach to reinforcement learning are presented, along with a survey of representative applications

Simulation, modelling and development of the metris RCA

In partnership with Metris UK we discuss the utilisation of modelling and simulation methods in the development of a revolutionary 7-axis Robot CMM Arm (RCA). An offline virtual model is described, facilitating pre-emptive collision avoidance and assessment of optimal placement of the RCA relative to scan specimens. Workspace accessibility of the RCA is examined under a range of geometrical assumptions and we discuss the effects of arbitrary offsets resulting from manufacturing tolerances. Degeneracy is identified in the number of ways a given pose may be attained and it is demonstrated how a simplified model may be exploited to solve the inverse kinematics problem of finding the “correct” set of joint angles. We demonstrate how the seventh axis may be utilised to avoid obstacles or otherwise awkward poses, giving the unit greater dexterity than traditional CMMs. The results of finite element analysis and static force modelling on the RCA are presented which provide an estimate of the forces exerted on the internal measurement arm in a range of poses

To mesh or not to mesh: flexible wireless indoor communication among mobile robots in industrial environments

Mobile robots such as automated guided vehicles become increasingly important in industry as they can greatly increase efficiency. For their operation such robots must rely on wireless communication, typically realized by connecting them to an existing enterprise network. In this paper we motivate that such an approach is not always economically viable or might result in performance issues. Therefore we propose a flexible and configurable mixed architecture that leverages on mesh capabilities whenever appropriate. Through experiments on a wireless testbed for a variety of scenarios, we analyse the impact of roaming, mobility and traffic separation and demonstrate the potential of our approach

Improved GelSight Tactile Sensor for Measuring Geometry and Slip

A GelSight sensor uses an elastomeric slab covered with a reflective membrane to measure tactile signals. It measures the 3D geometry and contact force information with high spacial resolution, and successfully helped many challenging robot tasks. A previous sensor, based on a semi-specular membrane, produces high resolution but with limited geometry accuracy. In this paper, we describe a new design of GelSight for robot gripper, using a Lambertian membrane and new illumination system, which gives greatly improved geometric accuracy while retaining the compact size. We demonstrate its use in measuring surface normals and reconstructing height maps using photometric stereo. We also use it for the task of slip detection, using a combination of information about relative motions on the membrane surface and the shear distortions. Using a robotic arm and a set of 37 everyday objects with varied properties, we find that the sensor can detect translational and rotational slip in general cases, and can be used to improve the stability of the grasp.Comment: IEEE/RSJ International Conference on Intelligent Robots and System

Search-based Motion Planning for Aggressive Flight in SE(3)

Quadrotors with large thrust-to-weight ratios are able to track aggressive trajectories with sharp turns and high accelerations. In this work, we develop a search-based trajectory planning approach that exploits the quadrotor maneuverability to generate sequences of motion primitives in cluttered environments. We model the quadrotor body as an ellipsoid and compute its flight attitude along trajectories in order to check for collisions against obstacles. The ellipsoid model allows the quadrotor to pass through gaps that are smaller than its diameter with non-zero pitch or roll angles. Without any prior information about the location of gaps and associated attitude constraints, our algorithm is able to find a safe and optimal trajectory that guides the robot to its goal as fast as possible. To accelerate planning, we first perform a lower dimensional search and use it as a heuristic to guide the generation of a final dynamically feasible trajectory. We analyze critical discretization parameters of motion primitive planning and demonstrate the feasibility of the generated trajectories in various simulations and real-world experiments.Comment: 8 pages, submitted to RAL and ICRA 201

Robots that can adapt like animals

As robots leave the controlled environments of factories to autonomously function in more complex, natural environments, they will have to respond to the inevitable fact that they will become damaged. However, while animals can quickly adapt to a wide variety of injuries, current robots cannot "think outside the box" to find a compensatory behavior when damaged: they are limited to their pre-specified self-sensing abilities, can diagnose only anticipated failure modes, and require a pre-programmed contingency plan for every type of potential damage, an impracticality for complex robots. Here we introduce an intelligent trial and error algorithm that allows robots to adapt to damage in less than two minutes, without requiring self-diagnosis or pre-specified contingency plans. Before deployment, a robot exploits a novel algorithm to create a detailed map of the space of high-performing behaviors: This map represents the robot's intuitions about what behaviors it can perform and their value. If the robot is damaged, it uses these intuitions to guide a trial-and-error learning algorithm that conducts intelligent experiments to rapidly discover a compensatory behavior that works in spite of the damage. Experiments reveal successful adaptations for a legged robot injured in five different ways, including damaged, broken, and missing legs, and for a robotic arm with joints broken in 14 different ways. This new technique will enable more robust, effective, autonomous robots, and suggests principles that animals may use to adapt to injury

Automated synthesis of monodisperse oligomers, featuring sequence control and tailored functionalization

Long, multifunctional sequence-defined oligomers were obtairred on solid support from a protecting-group-free two-step iterative protocol, based on the inherent reactivity of a readily available molecule containing an isocyanate and a thiolactone. Aminolysis of the latter entity with an amino alcohol liberates a thiol that reacts with an acrylate or acrylamide, present in the same medium. Subsequently, a new thiolactone can be reinstated by means of an alpha-isocyanato-gamma-thiolactone. Different acrylic compounds were used to incorporate diverse functionalities in the oligomers, which were built up to the level of decanters. The reaction conditions were closely monitored in order to fine-tune the applied strategy as well as facilitate the translation to an automated protocol