Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems

This paper was motivated by the problem of how to make robots fuse and transfer their experience so that they can effectively use prior knowledge and quickly adapt to new environments. To address the problem, we present a learning architecture for navigation in cloud robotic systems: Lifelong Federated Reinforcement Learning (LFRL). In the work, We propose a knowledge fusion algorithm for upgrading a shared model deployed on the cloud. Then, effective transfer learning methods in LFRL are introduced. LFRL is consistent with human cognitive science and fits well in cloud robotic systems. Experiments show that LFRL greatly improves the efficiency of reinforcement learning for robot navigation. The cloud robotic system deployment also shows that LFRL is capable of fusing prior knowledge. In addition, we release a cloud robotic navigation-learning website based on LFRL

Constraint capture and maintenance in engineering design

The Designers' Workbench is a system, developed by the Advanced Knowledge Technologies (AKT) consortium to support designers in large organizations, such as Rolls-Royce, to ensure that the design is consistent with the specification for the particular design as well as with the company's design rule book(s). In the principal application discussed here, the evolving design is described against a jet engine ontology. Design rules are expressed as constraints over the domain ontology. Currently, to capture the constraint information, a domain expert (design engineer) has to work with a knowledge engineer to identify the constraints, and it is then the task of the knowledge engineer to encode these into the Workbench's knowledge base (KB). This is an error prone and time consuming task. It is highly desirable to relieve the knowledge engineer of this task, and so we have developed a system, ConEditor+ that enables domain experts themselves to capture and maintain these constraints. Further we hypothesize that in order to appropriately apply, maintain and reuse constraints, it is necessary to understand the underlying assumptions and context in which each constraint is applicable. We refer to them as “application conditions” and these form a part of the rationale associated with the constraint. We propose a methodology to capture the application conditions associated with a constraint and demonstrate that an explicit representation (machine interpretable format) of application conditions (rationales) together with the corresponding constraints and the domain ontology can be used by a machine to support maintenance of constraints. Support for the maintenance of constraints includes detecting inconsistencies, subsumption, redundancy, fusion between constraints and suggesting appropriate refinements. The proposed methodology provides immediate benefits to the designers and hence should encourage them to input the application conditions (rationales)

Applying classifier systems to learn the reactions in mobile robots

The navigation problem involves how to reach a goal avoiding obstacles in dynamic environments. This problem can be faced considering reactions and sequences of actions. Classifier systems (CSs) have proven their ability of continuous learning, however, they have some problems in reactive systems. A modified CS, namely a reactive classifier system (RCS), is proposed to overcome those problems. Two special mechanisms are included in the RCS: the non-existence of internal cycles inside the CS (no internal cycles) and the fusion of environmental message with the messages posted to the message list in the previous instant (generation list through fusion). These mechanisms allow the learning of both reactions and sequences of actions. This learning process involves two main tasks: first, discriminate between rules and, second, the discovery of new rules to obtain a successful operation in dynamic environments. DiVerent experiments have been carried out using a mini-robot Khepera to find a generalized solution. The results show the ability of the system for continuous learning and adaptation to new situations.Publicad

An enhanced classifier system for autonomous robot navigation in dynamic environments

In many cases, a real robot application requires the navigation in dynamic environments. The navigation problem involves two main tasks: to avoid obstacles and to reach a goal. Generally, this problem could be faced considering reactions and sequences of actions. For solving the navigation problem a complete controller, including actions and reactions, is needed. Machine learning techniques has been applied to learn these controllers. Classifier Systems (CS) have proven their ability of continuos learning in these domains. However, CS have some problems in reactive systems. In this paper, a modified CS is proposed to overcome these problems. Two special mechanisms are included in the developed CS to allow the learning of both reactions and sequences of actions. The learning process has been divided in two main tasks: first, the discrimination between a predefined set of rules and second, the discovery of new rules to obtain a successful operation in dynamic environments. Different experiments have been carried out using a mini-robot Khepera to find a generalised solution. The results show the ability of the system to continuous learning and adaptation to new situations.Publicad

Reactive with tags classifier system applied to real robot navigation

7th IEEE International Conference on Emerging Technologies and Factory Automation. Barcelona, 18-21 October 1999.A reactive with tags classifier system (RTCS) is a special classifier system. This system combines the execution capabilities of symbolic systems and the learning capabilities of genetic algorithms. A RTCS is able to learn symbolic rules that allow to generate sequence of actions, chaining rules among different time instants, and react to new environmental situations, considering the last environmental situation to take a decision. The capacity of RTCS to learn good rules has been prove in robotics navigation problem. Results show the suitability of this approximation to the navigation problem and the coherence of extracted rules

Genetic Programming for Multibiometrics

Biometric systems suffer from some drawbacks: a biometric system can provide in general good performances except with some individuals as its performance depends highly on the quality of the capture. One solution to solve some of these problems is to use multibiometrics where different biometric systems are combined together (multiple captures of the same biometric modality, multiple feature extraction algorithms, multiple biometric modalities...). In this paper, we are interested in score level fusion functions application (i.e., we use a multibiometric authentication scheme which accept or deny the claimant for using an application). In the state of the art, the weighted sum of scores (which is a linear classifier) and the use of an SVM (which is a non linear classifier) provided by different biometric systems provide one of the best performances. We present a new method based on the use of genetic programming giving similar or better performances (depending on the complexity of the database). We derive a score fusion function by assembling some classical primitives functions (+, *, -, ...). We have validated the proposed method on three significant biometric benchmark datasets from the state of the art

Scalar and Pseudo-Scalar Higgs Production at Hadron Colliders

The evaluation of the NNLO QCD corrections to the production of a scalar and a pseudo-scalar Higgs boson is described.Comment: 5+2 pages, 4 figures (9 ps-files). Talk given at RADCOR/Loops and Legs 2002, September 8-13, 2002, Kloster Banz, German

Superheavy element production, nucleus-nucleus potential and mu-catalysis

The semi-microscopic potential between heavy nuclei is evaluated for various colliding ions in the approach of frozen densities in the framework of the extended Thomas-Fermi approximation with hbar^2 correction terms in the kinetic energy density functional. The proton and neutron densities of each nucleus are obtained in the Hartree-Fock-BCS approximation with SkM* parameter set of the Skyrme force. A simple expression for the nuclear interaction potential between spherical nuclei is presented. It is shown that muon bound with light projectile induces the superheavy elements production in nucleus-nucleus collisions.Comment: 9 pages, 4 figures, RevTex, Tours 5 Symposium on Nuclear Physics, Tours, August 200