Neural network controller against environment: A coevolutive approach to generalize robot navigation behavior

In this paper, a new coevolutive method, called Uniform Coevolution, is introduced to learn weights of a neural network controller in autonomous robots. An evolutionary strategy is used to learn high-performance reactive behavior for navigation and collisions avoidance. The introduction of coevolutive over evolutionary strategies allows evolving the environment, to learn a general behavior able to solve the problem in different environments. Using a traditional evolutionary strategy method, without coevolution, the learning process obtains a specialized behavior. All the behaviors obtained, with/without coevolution have been tested in a set of environments and the capability of generalization is shown for each learned behavior. A simulator based on a mini-robot Khepera has been used to learn each behavior. The results show that Uniform Coevolution obtains better generalized solutions to examples-based problems.Publicad

A general learning co-evolution method to generalize autonomous robot navigation behavior

Congress on Evolutionary Computation. La Jolla, CA, 16-19 July 2000.A new coevolutive method, called Uniform Coevolution, is introduced, to learn weights for a neural network controller in autonomous robots. An evolutionary strategy is used to learn high-performance reactive behavior for navigation and collision avoidance. The coevolutive method allows the evolution of the environment, to learn a general behavior able to solve the problem in different environments. Using a traditional evolutionary strategy method without coevolution, the learning process obtains a specialized behavior. All the behaviors obtained, with or without coevolution have been tested in a set of environments and the capability for generalization has been shown for each learned behavior. A simulator based on the mini-robot Khepera has been used to learn each behavior. The results show that Uniform Coevolution obtains better generalized solutions to example-based problems

Q Learning Behavior on Autonomous Navigation of Physical Robot

Behavior based architecture gives robot fast and reliable action. If there are many behaviors in robot, behavior coordination is needed. Subsumption architecture is behavior coordination method that give quick and robust response. Learning mechanism improve robot’s performance in handling uncertainty. Q learning is popular reinforcement learning method that has been used in robot learning because it is simple, convergent and off policy. In this paper, Q learning will be used as learning mechanism for obstacle avoidance behavior in autonomous robot navigation. Learning rate of Q learning affect robot’s performance in learning phase. As the result, Q learning algorithm is successfully implemented in a physical robot with its imperfect environment

An Idiotypic Immune Network as a Short Term Learning Architecture for Mobile Robots

A combined Short-Term Learning (STL) and Long-Term Learning (LTL) approach to solving mobile robot navigation problems is presented and tested in both real and simulated environments. The LTL consists of rapid simulations that use a Genetic Algorithm to derive diverse sets of behaviours. These sets are then transferred to an idiotypic Artificial Immune System (AIS), which forms the STL phase, and the system is said to be seeded. The combined LTL-STL approach is compared with using STL only, and with using a handdesigned controller. In addition, the STL phase is tested when the idiotypic mechanism is turned off. The results provide substantial evidence that the best option is the seeded idiotypic system, i.e. the architecture that merges LTL with an idiotypic AIS for the STL. They also show that structurally different environments can be used for the two phases without compromising transferabilityComment: 13 pages, 5 tables, 4 figures, 7th International Conference on Artificial Immune Systems (ICARIS2008), Phuket, Thailan

Using Centroidal Voronoi Tessellations to Scale Up the Multi-dimensional Archive of Phenotypic Elites Algorithm

The recently introduced Multi-dimensional Archive of Phenotypic Elites (MAP-Elites) is an evolutionary algorithm capable of producing a large archive of diverse, high-performing solutions in a single run. It works by discretizing a continuous feature space into unique regions according to the desired discretization per dimension. While simple, this algorithm has a main drawback: it cannot scale to high-dimensional feature spaces since the number of regions increase exponentially with the number of dimensions. In this paper, we address this limitation by introducing a simple extension of MAP-Elites that has a constant, pre-defined number of regions irrespective of the dimensionality of the feature space. Our main insight is that methods from computational geometry could partition a high-dimensional space into well-spread geometric regions. In particular, our algorithm uses a centroidal Voronoi tessellation (CVT) to divide the feature space into a desired number of regions; it then places every generated individual in its closest region, replacing a less fit one if the region is already occupied. We demonstrate the effectiveness of the new "CVT-MAP-Elites" algorithm in high-dimensional feature spaces through comparisons against MAP-Elites in maze navigation and hexapod locomotion tasks

Coevolutive adaptation of fitness landscape for solving the testing problem

IEEE International Conference on Systems, Man, and Cybernetics. Nashville, TN, 8-11 October 2000A general framework, called Uniform Coevolution, is introduced to overcome the testing problem in evolutionary computation methods. This framework is based on competitive evolution ideas where the solution and example sets are evolving by means of a competition to generate difficult test beds for the solutions in a gradual way. The method has been tested with two different problems: the robot navigation problem and the density parity problem in cellular automata. In both test cases using evolutive methods, the examples used in the learning process biased the solutions found. The main characteristics of the Uniform Coevolution method are that it smoothes the fitness landscape and, that it obtains “ideal learner examples”. Results using uniform coevolution show a high value of generality, compared with non co-evolutive approaches

Two-Timescale Learning Using Idiotypic Behaviour Mediation For A Navigating Mobile Robot

A combined Short-Term Learning (STL) and Long-Term Learning (LTL) approach to solving mobile-robot navigation problems is presented and tested in both the real and virtual domains. The LTL phase consists of rapid simulations that use a Genetic Algorithm to derive diverse sets of behaviours, encoded as variable sets of attributes, and the STL phase is an idiotypic Artificial Immune System. Results from the LTL phase show that sets of behaviours develop very rapidly, and significantly greater diversity is obtained when multiple autonomous populations are used, rather than a single one. The architecture is assessed under various scenarios, including removal of the LTL phase and switching off the idiotypic mechanism in the STL phase. The comparisons provide substantial evidence that the best option is the inclusion of both the LTL phase and the idiotypic system. In addition, this paper shows that structurally different environments can be used for the two phases without compromising transferability.Comment: 40 pages, 12 tables, Journal of Applied Soft Computin

Quantum Robot: Structure, Algorithms and Applications

A kind of brand-new robot, quantum robot, is proposed through fusing quantum theory with robot technology. Quantum robot is essentially a complex quantum system and it is generally composed of three fundamental parts: MQCU (multi quantum computing units), quantum controller/actuator, and information acquisition units. Corresponding to the system structure, several learning control algorithms including quantum searching algorithm and quantum reinforcement learning are presented for quantum robot. The theoretic results show that quantum robot can reduce the complexity of O(N^2) in traditional robot to O(N^(3/2)) using quantum searching algorithm, and the simulation results demonstrate that quantum robot is also superior to traditional robot in efficient learning by novel quantum reinforcement learning algorithm. Considering the advantages of quantum robot, its some potential important applications are also analyzed and prospected.Comment: 19 pages, 4 figures, 2 table