Sampled Policy Gradient for Learning to Play the Game Agar.io

In this paper, a new offline actor-critic learning algorithm is introduced: Sampled Policy Gradient (SPG). SPG samples in the action space to calculate an approximated policy gradient by using the critic to evaluate the samples. This sampling allows SPG to search the action-Q-value space more globally than deterministic policy gradient (DPG), enabling it to theoretically avoid more local optima. SPG is compared to Q-learning and the actor-critic algorithms CACLA and DPG in a pellet collection task and a self play environment in the game Agar.io. The online game Agar.io has become massively popular on the internet due to intuitive game design and the ability to instantly compete against players around the world. From the point of view of artificial intelligence this game is also very intriguing: The game has a continuous input and action space and allows to have diverse agents with complex strategies compete against each other. The experimental results show that Q-Learning and CACLA outperform a pre-programmed greedy bot in the pellet collection task, but all algorithms fail to outperform this bot in a fighting scenario. The SPG algorithm is analyzed to have great extendability through offline exploration and it matches DPG in performance even in its basic form without extensive sampling

09051 Abstracts Collection -- Knowledge representation for intelligent music processing

From the twenty-fifth to the thirtieth of January, 2009, the Dagstuhl Seminar 09051 on ``Knowledge representation for intelligent music processing\u27\u27 was held in Schloss Dagstuhl~--~Leibniz Centre for Informatics. During the seminar, several participants presented their current research, and ongoing work and open problems were discussed. Abstracts of the presentations and demos given during the seminar as well as plenary presentations, reports of workshop discussions, results and ideas are put together in this paper. The first section describes the seminar topics and goals in general, followed by plenary `stimulus\u27 papers, followed by reports and abstracts arranged by workshop followed finally by some concluding materials providing views of both the seminar itself and also forward to the longer-term goals of the discipline. Links to extended abstracts, full papers and supporting materials are provided, if available. The organisers thank David Lewis for editing these proceedings

Multi-Source Transfer Learning for Deep Model-Based Reinforcement Learning

A crucial challenge in reinforcement learning is to reduce the number of interactions with the environment that an agent requires to master a given task. Transfer learning proposes to address this issue by re-using knowledge from previously learned tasks. However, determining which source task qualifies as optimal for knowledge extraction, as well as the choice regarding which algorithm components to transfer, represent severe obstacles to its application in reinforcement learning. The goal of this paper is to alleviate these issues with modular multi-source transfer learning techniques. Our proposed methodologies automatically learn how to extract useful information from source tasks, regardless of the difference in state-action space and reward function. We support our claims with extensive and challenging cross-domain experiments for visual control.Comment: 15 pages, 6 figures, 8 tables. arXiv admin note: text overlap with arXiv:2108.0652

An analysis of rotation matrix and colour constancy data augmentation in classifying images of animals

In this paper, we examine a novel data augmentation (DA) method that transforms an image into a new image containing multiple rotated copies of the original image. The DA method creates a grid of cells, in which each cell contains a different randomly rotated image and introduces a natural background in the newly created image. We investigate the use of deep learning to assess the classification performance on the rotation matrix or original dataset with colour constancy versions of the datasets. For the colour constancy methods, we use two well-known retinex techniques: the multi-scale retinex and the multi-scale retinex with colour restoration for enhancing both original (ORIG) and rotation matrix (ROT) images. We perform experiments on three datasets containing images of animals, from which the first dataset is collected by us and contains aerial images of cows or non-cow backgrounds. To classify the Aerial UAV images, we use a convolutional neural network (CNN) architecture and compare two loss functions (hinge loss and cross-entropy loss). Additionally, we compare the CNN to classical feature-based techniques combined with a k-nearest neighbour classifier or a support vector machine. The best approach is then used to examine the colour constancy DA variants, ORIG and ROT-DA alone for three datasets (Aerial UAV, Bird-600 and Croatia fish). The results show that the rotation matrix data augmentation is very helpful for the Aerial UAV dataset. Furthermore, the colour constancy data augmentation is helpful for the Bird-600 dataset. Finally, the results show that the fine-tuned CNNs significantly outperform the CNNs trained from scratch on the Croatia fish and the Bird-600 datasets, and obtain very high accuracies on the Aerial UAV and Bird-600 datasets

The rationales of resilience in English and Dutch flood risk policies

We compared the governance of flood risk in England and the Netherlands, focusing on the general policies, instruments used and underlying principles. Both physical and political environments are important in explaining how countries evolved towards very different rationales of resilience. Answering questions as ‘who decides’, ‘who should act’ and ‘who is responsible and liable for flood damage’ systematically, results in a quite fundamental difference in what resilience means, and how this affects the governance regime. In the Netherlands, there is nationwide collective regime with a technocracy based on the merit of water expertise, legitimated by a social contract of government being responsible and the general public accepting and supporting this. In England there also is a technocracy, but this is part of a general-political and economic-rational decision-making process, with responsibilities spread over state, insurance companies, individuals and communities. The rationales are connected to specific conceptions of the public interest, leading to specific governance principles. In both countries, flood risk strategies are discussed in the light of climate change effects, but resilience strategies show more persistence, although combined with gradual adaptation of practices on lower scales, than great transformations