Induction and the discovery of the causes of scurvy: a computational reconstruction

AbstractThe work presented here addresses the problem of inductive reasoning in medical discoveries. The discovery of the causes of scurvy is studied and simulated using computational means. An inductive algorithm is successful in simulating some essential steps in the progress of the understanding of the disease and also allows us to simulate the false reasoning of previous centuries through the introduction of some a priori knowledge inherited from pre-clinical medicine. These results confirm the good results obtained by other AI researchers with an inductive approach of discovery, and illustrate the importance of the social and cultural environment on the way the inductive inference is performed and on its outcome

Adversarial Imitation Learning On Aggregated Data

Inverse Reinforcement Learning (IRL) learns an optimal policy, given some expert demonstrations, thus avoiding the need for the tedious process of specifying a suitable reward function. However, current methods are constrained by at least one of the following requirements. The first one is the need to fully solve a forward Reinforcement Learning (RL) problem in the inner loop of the algorithm, which might be prohibitively expensive in many complex environments. The second one is the need for full trajectories from the experts, which might not be easily available. The third one is the assumption that the expert data is homogeneous rather than a collection from various experts or possibly alternative solutions to the same task. Such constraints make IRL approaches either not scalable or not usable on certain existing systems. In this work we propose an approach which removes these requirements through a dynamic, adaptive method called Adversarial Imitation Learning on Aggregated Data (AILAD). It learns conjointly both a non linear reward function and the associated optimal policy using an adversarial framework. The reward learner only uses aggregated data. Moreover, it generates diverse behaviors producing a distribution over the aggregated data matching that of the experts

Discovering Abstract Concepts to Aid Cross-Map Transfer for a Learning Agent

Abstract. The capacity to apply knowledge in a context different than the one in which it was learned has become crucial within the area of autonomous agents. This paper specifically addresses the issue of transfer of knowledge acquired through online learning in partially observable environments. We investigate the discovery of relevant abstract concepts which help the transfer of knowledge in the context of an environment characterized by its 2D geographical configuration. The architecture proposed is tested in a simple grid-world environment where two agents duel each other. Results show that an agent's performances are improved through learning, including when it is tested on a map it has not yet seen

Extending the Strada Framework to Design an AI for ORTS

International audienceStrategy games constitute a significant challenge for game AI, as they involve a large number of states, agents and actions. This makes indeed the decision and learning algorithms difficult to design and implement. Many commercial strategy games use scripts in order to simulate intelligence, combined with knowledge which is in principle not accessible to human players, such as the position of the enemy base or the offensive power of its army. Nevertheless, recent research on adaptive techniques has shown promising results. The goal of this paper is to present the extension such a research methodology, named Strada, so that it is made applicable to the real-time strategy platform ORTS. The adaptations necessary to make Strada applicable to ORTS are detailed and involve the use of dynamic tactical points and specific training scenario for the learning AI. Two sets of experiments are conducted to evaluate the performances of the new method

Jeux vidéo et Systèmes Multi-Agents

International audienc

Combining Reinforcement Learning with a Multi-Level Abstraction Method to Design a Powerful Game AI

Abstract This paper investigates the design of a challenging Game AI for a modern strategy game, which can be seen as a large-scale multiagent simulation of an historical military confrontation. As an alternative to the typical script-based approach used in industry, we test an approach where military units and leaders, organized in a hierarchy, learn to improve their collective behavior through playing repeated games. In order to allow the application of a reinforcement learning framework at each level of this complex hierarchical decision-making structure, we propose an abstraction mechanism that adapts semi-automatically the level of detail of the state and action representations to the level of the agent. We also study specifically various reward signals as well as interagent communication setups and show their impact on the Game AI performance, distinctively in offensive and defensive modes. The resulting Game AI achieves very good performance when compared with the existing commercial script-based solution

Combining Reinforcement Learning with a Multi-Level Abstraction Method to Design a Powerful Game AI

National audienceThis paper investigates the design of a challenging Game AI for a modern strategy game, which can be seen as a large-scale multiagent simulation of an historical military confrontation. As an alternative to the typical script-based approach used in industry, we test an approach where military units and leaders, organized in a hierarchy, learn to improve their collective behavior through playing repeated games. In order to allow the application of a reinforcement learning framework at each level of this complex hierarchical decision-making structure, we propose an abstraction mechanism that adapts semi-automatically the level of detail of the state and action representations to the level of the agent. We also study specifically various reward signals as well as inter-agent communication setups and show their impact on the Game AI performance, distinctively in offensive and defensive modes. The resulting Game AI achieves very good performance when compared with the existing commercial script-based solution

{Dynamic Level of Detail for Large Scale Agent-Based Urban Simulations}

International audienceLarge scale agent-based simulations typically face a trade-off between the level of detail in the representation of each agent and the scalability seen as the number of agents that can be simulated with the computing resources available. In this paper, we aim at bypassing this trade-off by considering that the level of detail is itself a parameter that can be adapted automatically and dynamically during the simulation, taking into account elements such as user focus, or specific events. We introduce a framework for such a methodology, and detail its deployment within an existing simulator dedicated to the simulation of urban infrastructures. We evaluate the approach experimentally along two criteria: (1) the impact of our methodology on the resources (CPU use), and (2) an estimate of the dissimilarity between the two modes of simulation, i.e. with and without applying our methodology. Initial experiments show that a major gain in CPU time can be obtained for a very limited loss of consistency