Deep Reinforcement Learning for Multi-Agent Interaction

The development of autonomous agents which can interact with other agents to accomplish a given task is a core area of research in artificial intelligence and machine learning. Towards this goal, the Autonomous Agents Research Group develops novel machine learning algorithms for autonomous systems control, with a specific focus on deep reinforcement learning and multi-agent reinforcement learning. Research problems include scalable learning of coordinated agent policies and inter-agent communication; reasoning about the behaviours, goals, and composition of other agents from limited observations; and sample-efficient learning based on intrinsic motivation, curriculum learning, causal inference, and representation learning. This article provides a broad overview of the ongoing research portfolio of the group and discusses open problems for future directions.Comment: Published in AI Communications Special Issue on Multi-Agent Systems Research in the U

RIG-I and dsRNA-Induced IFNβ Activation

Except for viruses that initiate RNA synthesis with a protein primer (e.g., picornaviruses), most RNA viruses initiate RNA synthesis with an NTP, and at least some of their viral pppRNAs remain unblocked during the infection. Consistent with this, most viruses require RIG-I to mount an innate immune response, whereas picornaviruses require mda-5. We have examined a SeV infection whose ability to induce interferon depends on the generation of capped dsRNA (without free 5′ tri-phosphate ends), and found that this infection as well requires RIG-I and not mda-5. We also provide evidence that RIG-I interacts with poly-I/C in vivo, and that heteropolymeric dsRNA and poly-I/C interact directly with RIG-I in vitro, but in different ways; i.e., poly-I/C has the unique ability to stimulate the helicase ATPase of RIG-I variants which lack the C-terminal regulatory domain

Credit scoring using neural networks and SURE posterior probability calibration

In this article we compare the performances of a logistic regression and a feed forward neural network for credit scoring purposes. Our results show that the logistic regression gives quite good results on the dataset and the neural network can improve a little the performance. We also consider different sets of features in order to assess their importance in terms of prediction accuracy. We find that temporal features (i.e. repeated measures over time) can be an important source of information resulting in an increase in the overall model accuracy. Finally, we introduce a new technique for the calibration of predicted probabilities based on Stein's unbiased risk estimate (SURE). This calibration technique can be applied to very general calibration functions. In particular, we detail this method for the sigmoid function as well as for the Kumaraswamy function, which includes the identity as a particular case. We show that the SURE calibration technique is able to calibrate the predicted probabilities as well as the classical Platt method

Collection of procedurally generated MiniGrid environments datasets

This dataset contains procedurally generated MiniGrid environments (two dimensional gridworlds) to benchmark and train Reinforcement Learning agents. These datasets can also be used to train generative models of MiniGrid environments. For more information on how to make use of these datasets refer to the README of our dred code repository at https://github.com/uoe-agents/dred

: Application géomatique en région Provence Alpes Côte d'Azur

International audienceDans ce poster, nous présentons le modèle gravitaire sous différents modèles en mettant en évidence les effets de distance et les occasions interposées. Le modèle généralisé de Newton est utilisé et donne des estimations relativement correctes des flux intercommunaux en région PACA

Modelling the Response of Atoll Reef Islands to Multi-Millennial Sea Level Rise from the Last Glacial Maximum to the Coming 10kyr: the Case of Mururoa Atoll (Tuamotu, French Polynesia)

International audienceComposed of biodetritic sediments and lying just a few meters above present sea-level, atoll reef islands are liable to be highly exposed to coastal flooding and shoreline erosion. Nevertheless, the analysis of multi-decadal shoreline change has shown that most reef islands either remain stable in area or are expanding within the context of current sea level rise. This article addresses the key issue of future atoll-island persistence using a simple morphodynamic model based on the computation of sediment production and fluxes, vertical coral growth and reef island accretion, with special reference to Mururoa Atoll (French Polynesia). The model parameters are calibrated from previously gained stratigraphic frameworks and sediment production rates. While a proper validation is a challenge with the scares data available, the model fits well with the atoll-rim and atoll-islands evolution schemes of Mururoa Atoll since the last glacial maximum. Multi-millennial projections of sea-level rise (Clark et al., 2016) are used to examine future reef island response to rising sea-level. Assuming that all sediment volumes available on the atoll rim maintain in place and that the sediment production remains unaffected by ocean warming and acidification, the reef is interpreted as able to catch up sea level rise in the near future. Even in this very optimistic evolution scheme, the new reef edifice would be filnally drown in a high carbon emission scenario. The present study, along with others, strongly suggests that the persistence of reef islands in the future requires the conservation of already available sediments together with a continued production of coral detritus, not only from the outer slopes, but also on the atoll rim as water depths increase

Estimer des flux de navetteurs avec un modèle gravitaire. Application géomatique en Région Provence-Alpes-Côte d'Azur (France)

International audienceThis paper investigates the modeling of journeys to work commuting flows using gravity models. Based on a study for the Provence-Alpes Côte d'Azur region, a model that meets the principle of parsimony, based on publicly available data, was designed to meet the need for forecasting, while ensuring transparency and reproducibility of the method. Different models are processed, tuning several input variables such as the masses of geographical entities, the way to compute short distances as the crow flies, the time budget constraint from Zahavi, and successive resampling. The results show that it is possible to build a robust model of correct quality to estimate the commuting flows, using INSEE data and a log-linear gravity computation, while reducing the sample size.Cet article réinvestit la modélisation des flux de déplacement domicile-travail au moyen des modèles gravitaires de flux. À partir d'une demande émanant de la région Provence-Alpes Côte d'Azur, un modèle répondant au principe de parcimonie, basé sur des données publiques ouvertes et facilement mobilisables, a été établi afin de répondre au besoin de prévision tout en assurant la transparence et la réplicabilité de la démarche. À partir de ce modèle, différentes hypothèses (type de variable sur les masses des entités géographiques, variations des calculs des courtes distances à vol d'oiseau, contrainte de portée spatiale due au budget-temps de déplacement, ré-échantillonnage aléatoire) ont été modélisées dans une optique de prospective et d'aide à la décision. Les résultats montrent qu'il est possible d'obtenir un modèle assez robuste avec les données disponibles de l'INSEE et un modèle gravitaire log-linéaire, tout en réduisant sensiblement la taille de l'échantillon traité

Official Data Repository for DRED: Zero-Shot Transfer in Reinforcement Learning via Data-Regularised Environment Design

Autonomous agents trained using deep reinforcement learning (RL) often lack the ability to successfully generalise to new environments, even when these environments share characteristics with the ones they have encountered during training. In this work, we investigate how the sampling of individual environment instances, or levels, affects the zero-shot generalisation (ZSG) ability of RL agents. We discover that, for deep actor-critic architectures sharing their base layers, prioritising levels according to their value loss minimises the mutual information between the agent's internal representation and the set of training levels in the generated training data. This provides a novel theoretical justification for the regularisation achieved by certain adaptive sampling strategies. We then turn our attention to unsupervised environment design (UED) methods, which assume control over level generation. We find that existing UED methods can significantly shift the training distribution, which translates to low ZSG performance. To prevent both overfitting and distributional shift, we introduce data-regularised environment design (DRED). DRED generates levels using a generative model trained to approximate the ground truth distribution of an initial set of level parameters. Through its grounding, DRED achieves significant improvements in ZSG over adaptive level sampling strategies and UED methods. Our code and experimental data are available at: Garcin et al, "DRED: Zero-Shot Transfer in Reinforcement Learning via Data-Regularised Environment Design", 2024, (https://github.com/uoe-agents/dred)