Transcriptional signature of human pro-inflammatory TH17 cells identifies reduced IL10 gene expression in multiple sclerosis

We have previously reported the molecular signature of murine pathogenic TH17 cells that induce experimental autoimmune encephalomyelitis (EAE) in animals. Here we show that human peripheral blood IFN-γ+IL-17+ (TH1/17) and IFN-γ−IL-17+ (TH17) CD4+ T cells display distinct transcriptional profiles in high-throughput transcription analyses. Compared to TH17 cells, TH1/17 cells have gene signatures with marked similarity to mouse pathogenic TH17 cells. Assessing 15 representative signature genes in patients with multiple sclerosis, we find that TH1/17 cells have elevated expression of CXCR3 and reduced expression of IFNG, CCL3, CLL4, GZMB, and IL10 compared to healthy controls. Moreover, higher expression of IL10 in TH17 cells is found in clinically stable vs. active patients. Our results define the molecular signature of human pro-inflammatory TH17 cells, which can be used to both identify pathogenic TH17 cells and to measure the effect of treatment on TH17 cells in human autoimmune diseases

Model-Assisted Approaches for Relational Reinforcement Learning (Model-assisterende methoden voor het leren uit beloningen in complexe omgevingen)

Automatisch leren (''machine learning'') is het onderzoeksveld binnen kunstmatige intelligentie dat zich bezig houdt met het ontwikkelen van computer programma's die kunnen leren uit ervaring. Een belangrijke toepassing hiervan is het leren uit beloningen (''reinforcement learning'' (RL)) waarbij de programma's dienen te leren door middel van interactie met hun omgeving en dit op basis van beloningen of straffen die zij ontvangen als informatie over hun vertoonde gedrag, d.w.z. positieve of negatieve numerieke waarden. Om dergelijke technieken toe te passen in complexe omgevingen is er veel onderzoek gedaan naar de integratie van allerlei vormen van abstractie en generalisatie in deze leertechniek. E\'en van deze vormen, dewelke recent veel interesse geniet, is het gebruik maken van relationele representaties bij het voorstellen van toestanden, acties en het gedrag van het systeem. In dit proefschrift zullen we technieken onderzoeken die verder bouwen op deze vorm van abstractie en dan met name systemen ontwikkelen die extra informatie over de omgeving kunnen leren en deze informatie vervolgens gebruiken om sneller een goed gedrag te leren. In een eerste deel zullen we drie zulke systemen voorstellen. Een eerste systeem combineert relationele representaties en temporele abstractie. Vervolgens zullen we een model-gebaseerd leersysteem voorstellen dat de dynamica van de omgeving kan leren. Het derde leersysteem onderzoekt de invloed van meerdere leersystemen in dezelfde omgeving. We zullen hierbij aantonen hoe relationele representaties gebruikt kunnen worden om de leersystemen van elkaar te laten leren en hoe deze tevens kunnen helpen bij de communicatie tussen deze verschillende systemen. In een tweede deel zullen twee technieken voorgesteld worden die betere modellen kunnen leren. Een eerste techniek is een nieuwe leermethode voor het incrementeel leren van relationele regressie bomen en een tweede techniek die probabilistisch logische modellen kan leren.status: publishe

On intelligent agents in relational reinforcement learning

status: publishe

Applying machine learning in the real world

status: publishe

Informed reinforcement learning

status: publishe

Searching for compound goals using relevancy zones in the game of Go

In complex games with a high branching factor, global alphabeta search is computationally infeasible. One way to overcome this problem is by using selective goal-directed search algorithms. These goal-directed searches can use relevancy zones to determine which part of the board influences the goal. In this paper, we propose a general method that uses these relevancy zones for searching for compound goals. A compound goal is constructed from less complex atomic goals, using the standard connectives. In contrast to other approaches that treat goals separately in the search phase, compound goal search obtains exact results.status: publishe

Multi-agent relational reinforcement learning. Explorations in multi-state coordination tasks

Abstract. In this paper we report on using a relational state space in multi-agent reinforcement learning. There is growing evidence in the Reinforcement Learning research community that a relational representation of the state space has many benefits over a propositional one. Complex tasks as planning or information retrieval on the web can be represented more naturally in relational form. Yet, this relational structure has not been exploited for multi-agent reinforcement learning tasks and has only been studied in a single agent context so far. In this paper we explore the powerful possibilities of using Relational Reinforcement Learning (RRL) in complex multi-agent coordination tasks. More precisely, we consider an abstract multi-state coordination problem, which canbeconsideredasavariationandextensionofrepeatedstatelessDispersion Games. Our approach shows that RRL allows to represent a complex state space in a multi-agent environment more compactly and allows for fast convergence of learning agents. Moreover, with this technique, agents are able to make complex interactive models (in the sense of learning from an expert), to predict what other agents will do and generalize over this model. This enables to solve complex multi-agent planning tasks, in which agents need to be adaptive and learn, with more powerful tools.

Transfer learning in reinforcement learning problems through partial policy recycling

In this paper we investigate the relation between transfer learning in reinforcement learning with function approximation and supervised learning with concept drift. We present a new incremental relational regression tree algorithm that is capable of dealing with concept drift through tree restructuring and show that it enables a reinforcement learner, more precisely a Q-learner, to transfer knowledge from one task to another by recycling those parts of the generalized Q-function that still hold interesting information for the new task. We illustrate the performance of the algorithm in experiments with both supervised learning tasks with concept drift and reinforcement learning tasks that allow the transfer of knowledge from easier, related tasks.Acceptance rate = 23%status: publishe