Finite automata games: basic concepts.

In this chapter we review the basic concepts on automata games, including best response, inference, equilibrium and complex system dynamics. We describe how the concept of Nash equilibrium is used to analyze the properties of automata systems and discuss its limitations. We explain why we think the topics of automata inference, the modeling of evolving automata, and the analysis of the relationship between emotions and reason, are interesting areas for further research

Combining latent learning with dynamic programming in the modular anticipatory classifier system

International audienceLearning Classifier Systems (LCS) are rule based Reinforcement Learning (RL) systems which use a generalization capability. In this paper, we highlight the differences between two kinds of LCSs. Some are used to directly perform RL while others latently learn a model of the interactions between the agent and its environment. Such a model can be used to speed up the core RL process. Thus, these two kinds of learning processes are complementary. We show here how the notion of generalization differs depending on whether the system anticipates (like Anticipatory Classifier System (ACS) and Yet Another Classifier System (YACS)) or not (like XCS). Moreover, we show some limitations of the formalism common to ACS and YACS, and propose a new system, called Modular Anticipatory Classifier System (MACS), which allows the latent learning process to take advantage of new regularities. We describe how the model can be used to perform active exploration and how this exploration may be aggregated with the policy resulting from the reinforcement learning process. The different algorithms are validated experimentally and some limitations in presence of uncertainties are highlighted