Collective Robot Reinforcement Learning with Distributed Asynchronous Guided Policy Search

In principle, reinforcement learning and policy search methods can enable robots to learn highly complex and general skills that may allow them to function amid the complexity and diversity of the real world. However, training a policy that generalizes well across a wide range of real-world conditions requires far greater quantity and diversity of experience than is practical to collect with a single robot. Fortunately, it is possible for multiple robots to share their experience with one another, and thereby, learn a policy collectively. In this work, we explore distributed and asynchronous policy learning as a means to achieve generalization and improved training times on challenging, real-world manipulation tasks. We propose a distributed and asynchronous version of Guided Policy Search and use it to demonstrate collective policy learning on a vision-based door opening task using four robots. We show that it achieves better generalization, utilization, and training times than the single robot alternative.Comment: Submitted to the IEEE International Conference on Robotics and Automation 201

A Global Optimisation Toolbox for Massively Parallel Engineering Optimisation

A software platform for global optimisation, called PaGMO, has been developed within the Advanced Concepts Team (ACT) at the European Space Agency, and was recently released as an open-source project. PaGMO is built to tackle high-dimensional global optimisation problems, and it has been successfully used to find solutions to real-life engineering problems among which the preliminary design of interplanetary spacecraft trajectories - both chemical (including multiple flybys and deep-space maneuvers) and low-thrust (limited, at the moment, to single phase trajectories), the inverse design of nano-structured radiators and the design of non-reactive controllers for planetary rovers. Featuring an arsenal of global and local optimisation algorithms (including genetic algorithms, differential evolution, simulated annealing, particle swarm optimisation, compass search, improved harmony search, and various interfaces to libraries for local optimisation such as SNOPT, IPOPT, GSL and NLopt), PaGMO is at its core a C++ library which employs an object-oriented architecture providing a clean and easily-extensible optimisation framework. Adoption of multi-threaded programming ensures the efficient exploitation of modern multi-core architectures and allows for a straightforward implementation of the island model paradigm, in which multiple populations of candidate solutions asynchronously exchange information in order to speed-up and improve the optimisation process. In addition to the C++ interface, PaGMO's capabilities are exposed to the high-level language Python, so that it is possible to easily use PaGMO in an interactive session and take advantage of the numerous scientific Python libraries available.Comment: To be presented at 'ICATT 2010: International Conference on Astrodynamics Tools and Techniques

Optimal options for rendezvous and impact missions to NEOs

In this paper some potentially interesting transfer options for missions to Near Earth Objects have been studied. Due to thehigh number of potential targets and to the large variety of possible missions that can be considered, especially if resorting to low-thrust propulsion, an extensive analysis of transfer options requires a preliminary approach oriented toward an effective global search, and an appropriately simplified trajectory transcription. Low-thrust options have been modeled through a novel shape-based approach and a global optimization method has been used to look for globally optimal transfers. Different targets have been identified and various mission scenarios have been considered: rendezvous, sample return missions both with and without Earth gravity assist and impact missions

Optimal coverage multi-path scheduling scheme with multiple mobile sinks for WSNs

Wireless Sensor Networks (WSNs) are usually formed with many tiny sensors which are randomly deployed within sensing field for target monitoring. These sensors can transmit their monitored data to the sink in a multi-hop communication manner. However, the ‘hot spots’ problem will be caused since nodes near sink will consume more energy during forwarding. Recently, mobile sink based technology provides an alternative solution for the long-distance communication and sensor nodes only need to use single hop communication to the mobile sink during data transmission. Even though it is difficult to consider many network metrics such as sensor position, residual energy and coverage rate etc., it is still very important to schedule a reasonable moving trajectory for the mobile sink. In this paper, a novel trajectory scheduling method based on coverage rate for multiple mobile sinks (TSCR-M) is presented especially for large-scale WSNs. An improved particle swarm optimization (PSO) combined with mutation operator is introduced to search the parking positions with optimal coverage rate. Then the genetic algorithm (GA) is adopted to schedule the moving trajectory for multiple mobile sinks. Extensive simulations are performed to validate the performance of our proposed method

Multiple NEA Rendezvous Mission: Solar Sailing Options

The scientific interest in near-Earth asteroids (NEAs) and the classification of some of those as potentially hazardous asteroid for the Earth stipulated the interest in NEA exploration. Close-up observations of these objects will increase drastically our knowledge about the overall NEA population. For this reason, a multiple NEA rendezvous mission through solar sailing is investigated, taking advantage of the propellantless nature of this groundbreaking propulsion technology. Considering a spacecraft based on the DLR/ESA Gossamer technology, this work focuses on the search of possible sequences of NEA encounters. The effectiveness of this approach is demonstrated through a number of fully-optimized trajectories. The results show that it is possible to visit five NEAs within 10 years with near-term solar-sail technology. Moreover, a study on a reduced NEA database demonstrates the reliability of the approach used, showing that 58% of the sequences found with an approximated trajectory model can be converted into real solar-sail trajectories. Lastly, this second study shows the effectiveness of the proposed automatic optimization algorithm, which is able to find solutions for a large number of mission scenarios without any input required from the user

Evolutionary neurocontrol: A novel method for low-thrust gravity-assist trajectory optimization

This article discusses evolutionary neurocontrol, a novel method for low-thrust gravity-assist trajectory optimization

Preliminary capture trajectory design for Europa tomography probe

The objective of this work is the preliminary design of a low-DV transfer from an initial elliptical orbit around Jupiter into a final circular orbit around the moon Europa. This type of trajectory represents an excellent opportunity for a low-cost mission to Europa, accomplished through a small orbiter, as in the proposed Europa Tomography Probe mission, a European contribution to NASA’s Europa Multiple-Flyby Mission (or Europa Clipper). The mission strategy is based on the v-infinity leveraging concept, and the use of resonant orbits to exploit multiple gravity-assist from the moon. Possible sequences of resonant orbits are selected with the help of the Tisserand graph. Suitable trajectories are provided by an optimization code based on the parallel running of several differential evolution algorithms. Different solutions are finally compared in terms of propellant consumption and flight time