11 research outputs found
HyperNCA: Growing Developmental Networks with Neural Cellular Automata
In contrast to deep reinforcement learning agents, biological neural networks
are grown through a self-organized developmental process. Here we propose a new
hypernetwork approach to grow artificial neural networks based on neural
cellular automata (NCA). Inspired by self-organising systems and
information-theoretic approaches to developmental biology, we show that our
HyperNCA method can grow neural networks capable of solving common
reinforcement learning tasks. Finally, we explore how the same approach can be
used to build developmental metamorphosis networks capable of transforming
their weights to solve variations of the initial RL task.Comment: Paper accepted as a conference paper at ICLR 'From Cells to
Societies' workshop 202
EvoCraft: A New Challenge for Open-Endedness
This paper introduces EvoCraft, a framework for Minecraft designed to study
open-ended algorithms. We introduce an API that provides an open-source Python
interface for communicating with Minecraft to place and track blocks. In
contrast to previous work in Minecraft that focused on learning to play the
game, the grand challenge we pose here is to automatically search for
increasingly complex artifacts in an open-ended fashion. Compared to other
environments used to study open-endedness, Minecraft allows the construction of
almost any kind of structure, including actuated machines with circuits and
mechanical components. We present initial baseline results in evolving simple
Minecraft creations through both interactive and automated evolution. While
evolution succeeds when tasked to grow a structure towards a specific target,
it is unable to find a solution when rewarded for creating a simple machine
that moves. Thus, EvoCraft offers a challenging new environment for automated
search methods (such as evolution) to find complex artifacts that we hope will
spur the development of more open-ended algorithms. A Python implementation of
the EvoCraft framework is available at:
https://github.com/real-itu/Evocraft-py
MarioGPT: Open-Ended Text2Level Generation through Large Language Models
Procedural Content Generation (PCG) algorithms provide a technique to
generate complex and diverse environments in an automated way. However, while
generating content with PCG methods is often straightforward, generating
meaningful content that reflects specific intentions and constraints remains
challenging. Furthermore, many PCG algorithms lack the ability to generate
content in an open-ended manner. Recently, Large Language Models (LLMs) have
shown to be incredibly effective in many diverse domains. These trained LLMs
can be fine-tuned, re-using information and accelerating training for new
tasks. In this work, we introduce MarioGPT, a fine-tuned GPT2 model trained to
generate tile-based game levels, in our case Super Mario Bros levels. We show
that MarioGPT can not only generate diverse levels, but can be text-prompted
for controllable level generation, addressing one of the key challenges of
current PCG techniques. As far as we know, MarioGPT is the first text-to-level
model. We also combine MarioGPT with novelty search, enabling it to generate
diverse levels with varying play-style dynamics (i.e. player paths). This
combination allows for the open-ended generation of an increasingly diverse
range of content
Growing 3D Artefacts and Functional Machines with Neural Cellular Automata
Neural Cellular Automata (NCAs) have been proven effective in simulating
morphogenetic processes, the continuous construction of complex structures from
very few starting cells. Recent developments in NCAs lie in the 2D domain,
namely reconstructing target images from a single pixel or infinitely growing
2D textures. In this work, we propose an extension of NCAs to 3D, utilizing 3D
convolutions in the proposed neural network architecture. Minecraft is selected
as the environment for our automaton since it allows the generation of both
static structures and moving machines. We show that despite their simplicity,
NCAs are capable of growing complex entities such as castles, apartment blocks,
and trees, some of which are composed of over 3,000 blocks. Additionally, when
trained for regeneration, the system is able to regrow parts of simple
functional machines, significantly expanding the capabilities of simulated
morphogenetic systems. The code for the experiment in this paper can be found
at: https://github.com/real-itu/3d-artefacts-nca
Differentiable Logic Machines
The integration of reasoning, learning, and decision-making is key to build
more general AI systems. As a step in this direction, we propose a novel
neural-logic architecture that can solve both inductive logic programming (ILP)
and deep reinforcement learning (RL) problems. Our architecture defines a
restricted but expressive continuous space of first-order logic programs by
assigning weights to predicates instead of rules. Therefore, it is fully
differentiable and can be efficiently trained with gradient descent. Besides,
in the deep RL setting with actor-critic algorithms, we propose a novel
efficient critic architecture. Compared to state-of-the-art methods on both ILP
and RL problems, our proposition achieves excellent performance, while being
able to provide a fully interpretable solution and scaling much better,
especially during the testing phase
Périodes du groupe fondamental motivique de la droite projective moins zero, l’infini et les racines n-èmes de l’unité
Following F. Brown's point of view, we look at the Hopf algebra structure of motivic cyclotomic multiple zeta values, which are motivic periods of the fundamental groupoid of the projective line minus 0, infinity and N roots of unity. By application of a surjective period map (conjectured isomorphism), we deduce results (generating families, identities, etc.) on cyclotomic multiple zeta values, which are complex numbers. The coaction of this Hopf algebra (explicit combinatorial formula) is the dual of the action of a so-called motivic Galois group on these specific motivic periods. This entire study was motivated by the hope of a Galois theory for periods, which should extend the usual one for algebraic numbers.(i)In the first part, we focus on the case of motivic multiple zeta values (N = 1) and Euler sums (N = 2). In particular, we present new bases for motivic multiple zeta values: one via motivic Euler sums, and another (depending on an analytic conjecture) which is known as the Hoffman star basis; under a general motivic identity that we conjecture, these bases are identical.
(ii)In the second part, we apply some Galois descents ideas to the study of these periods, and examine how multiple zeta values relative to N' roots of unity are embedded into those relative to N roots, when N' divide N. After giving some general criteria for any N, we focus on the cases N=2,3,4, 6, 8, for which the motivic fundamental group generates the category of mixed Tate motives on the ring of integer of the N cyclotomic field ramified in N (unramified if N=6). For those N, we are able to construct Galois descents explicitly, and extend P. Deligne's results.En s'inspirant du point de vue adopté par Francis Brown, nous examinons la structure d'algèbre de Hopf des multizêtas motiviques cyclotomiques, qui sont des périodes motiviques du groupoïde fondamental de la droite projective moins 0, l'infini et les racines Nèmes de l'unité. Par application d'un morphisme période surjectif (conjecturé isomorphisme), nous pouvons déduire des résultats (identités, familles génératrices, etc.) sur les multizêtas cyclotomiques (complexes). La coaction de cette algèbre de Hopf (formule combinatoire explicite) est duale à l'action d'un dénommé groupe de Galois motivique sur ces périodes motiviques. Ces recherches sont ainsi motivées par l'espoir d'une théorie de Galois pour les périodes, étendant la théorie de Galois usuelle pour les nombres algébriques. (i) Nous présentons de nouvelles relations entre les sommes d'Euler (N=2) motiviques et deux nouvelles bases (conjecturées identiques) pour les multizêtas motiviques (N=1): Hoffman star (sous une conjecture analytique) et une seconde via les sommes d'Euler motiviques. (ii) Nous appliquons des idées de descentes galoisiennes à l'étude de ces périodes, en regardant notamment comment les multizêtas motiviques relatifs aux racines N' èmes de l'unité se plongent dans ceux associés aux racines Nèmes, lorsque N' divise N. Après avoir fourni des critères généraux, nous nous tournons vers les cas N égal à 2,3,4,6, 8, pour lesquels le groupoïde fondamental motivique engendre la catégorie des motifs de Tate mixtes sur l'anneau des entiers du Nème corps cyclotomique ramifié en N (non ramifié pour 6). Pour ces valeurs, nous explicitons les descentes galoisiennes, et étendons les résultats de Pierre Delign
Périodes du groupe fondamental motivique de la droite projective moins zero, l’infini et les racines n-èmes de l’unité
En s'inspirant du point de vue adopté par Francis Brown, nous examinons la structure d'algèbre de Hopf des multizêtas motiviques cyclotomiques, qui sont des périodes motiviques du groupoïde fondamental de la droite projective moins 0, l'infini et les racines Nèmes de l'unité. Par application d'un morphisme période surjectif (conjecturé isomorphisme), nous pouvons déduire des résultats (identités, familles génératrices, etc.) sur les multizêtas cyclotomiques (complexes). La coaction de cette algèbre de Hopf (formule combinatoire explicite) est duale à l'action d'un dénommé groupe de Galois motivique sur ces périodes motiviques. Ces recherches sont ainsi motivées par l'espoir d'une théorie de Galois pour les périodes, étendant la théorie de Galois usuelle pour les nombres algébriques. (i) Nous présentons de nouvelles relations entre les sommes d'Euler (N=2) motiviques et deux nouvelles bases (conjecturées identiques) pour les multizêtas motiviques (N=1): Hoffman star (sous une conjecture analytique) et une seconde via les sommes d'Euler motiviques. (ii) Nous appliquons des idées de descentes galoisiennes à l'étude de ces périodes, en regardant notamment comment les multizêtas motiviques relatifs aux racines N' èmes de l'unité se plongent dans ceux associés aux racines Nèmes, lorsque N' divise N. Après avoir fourni des critères généraux, nous nous tournons vers les cas N égal à 2,3,4,6, 8, pour lesquels le groupoïde fondamental motivique engendre la catégorie des motifs de Tate mixtes sur l'anneau des entiers du Nème corps cyclotomique ramifié en N (non ramifié pour 6). Pour ces valeurs, nous explicitons les descentes galoisiennes, et étendons les résultats de Pierre DeligneFollowing F. Brown's point of view, we look at the Hopf algebra structure of motivic cyclotomic multiple zeta values, which are motivic periods of the fundamental groupoid of the projective line minus 0, infinity and N roots of unity. By application of a surjective period map (conjectured isomorphism), we deduce results (generating families, identities, etc.) on cyclotomic multiple zeta values, which are complex numbers. The coaction of this Hopf algebra (explicit combinatorial formula) is the dual of the action of a so-called motivic Galois group on these specific motivic periods. This entire study was motivated by the hope of a Galois theory for periods, which should extend the usual one for algebraic numbers.(i)In the first part, we focus on the case of motivic multiple zeta values (N = 1) and Euler sums (N = 2). In particular, we present new bases for motivic multiple zeta values: one via motivic Euler sums, and another (depending on an analytic conjecture) which is known as the Hoffman star basis; under a general motivic identity that we conjecture, these bases are identical.\u2028(ii)In the second part, we apply some Galois descents ideas to the study of these periods, and examine how multiple zeta values relative to N' roots of unity are embedded into those relative to N roots, when N' divide N. After giving some general criteria for any N, we focus on the cases N=2,3,4, 6, 8, for which the motivic fundamental group generates the category of mixed Tate motives on the ring of integer of the N cyclotomic field ramified in N (unramified if N=6). For those N, we are able to construct Galois descents explicitly, and extend P. Deligne's results
Learning Fair Policies in Decentralized Cooperative Multi-Agent Reinforcement Learning
We consider the problem of learning fair policies in (deep) cooperative
multi-agent reinforcement learning (MARL). We formalize it in a principled way
as the problem of optimizing a welfare function that explicitly encodes two
important aspects of fairness: efficiency and equity. As a solution method, we
propose a novel neural network architecture, which is composed of two
sub-networks specifically designed for taking into account the two aspects of
fairness. In experiments, we demonstrate the importance of the two sub-networks
for fair optimization. Our overall approach is general as it can accommodate
any (sub)differentiable welfare function. Therefore, it is compatible with
various notions of fairness that have been proposed in the literature (e.g.,
lexicographic maximin, generalized Gini social welfare function, proportional
fairness). Our solution method is generic and can be implemented in various
MARL settings: centralized training and decentralized execution, or fully
decentralized. Finally, we experimentally validate our approach in various
domains and show that it can perform much better than previous methods.Comment: International Conference on Machine Learnin