Advances in Grid Computing

This book approaches the grid computing with a perspective on the latest achievements in the field, providing an insight into the current research trends and advances, and presenting a large range of innovative research papers. The topics covered in this book include resource and data management, grid architectures and development, and grid-enabled applications. New ideas employing heuristic methods from swarm intelligence or genetic algorithm and quantum encryption are considered in order to explain two main aspects of grid computing: resource management and data management. The book addresses also some aspects of grid computing that regard architecture and development, and includes a diverse range of applications for grid computing, including possible human grid computing system, simulation of the fusion reaction, ubiquitous healthcare service provisioning and complex water systems

Accelerating Cooperative Planning for Automated Vehicles with Learned Heuristics and Monte Carlo Tree Search

Efficient driving in urban traffic scenarios requires foresight. The observation of other traffic participants and the inference of their possible next actions depending on the own action is considered cooperative prediction and planning. Humans are well equipped with the capability to predict the actions of multiple interacting traffic participants and plan accordingly, without the need to directly communicate with others. Prior work has shown that it is possible to achieve effective cooperative planning without the need for explicit communication. However, the search space for cooperative plans is so large that most of the computational budget is spent on exploring the search space in unpromising regions that are far away from the solution. To accelerate the planning process, we combined learned heuristics with a cooperative planning method to guide the search towards regions with promising actions, yielding better solutions at lower computational costs

Toward evolutionary and developmental intelligence

Given the phenomenal advances in artificial intelligence in specific domains like visual object recognition and game playing by deep learning, expectations are rising for building artificial general intelligence (AGI) that can flexibly find solutions in unknown task domains. One approach to AGI is to set up a variety of tasks and design AI agents that perform well in many of them, including those the agent faces for the first time. One caveat for such an approach is that the best performing agent may be just a collection of domain-specific AI agents switched for a given domain. Here we propose an alternative approach of focusing on the process of acquisition of intelligence through active interactions in an environment. We call this approach evolutionary and developmental intelligence (EDI). We first review the current status of artificial intelligence, brain-inspired computing and developmental robotics and define the conceptual framework of EDI. We then explore how we can integrate advances in neuroscience, machine learning, and robotics to construct EDI systems and how building such systems can help us understand animal and human intelligence

Learning to Play Chess from Textbooks (LEAP): a Corpus for Evaluating Chess Moves based on Sentiment Analysis

Learning chess strategies has been investigated widely, with most studies focussing on learning from previous games using search algorithms. Chess textbooks encapsulate grandmaster knowledge, explain playing strategies and require a smaller search space compared to traditional chess agents. This paper examines chess textbooks as a new knowledge source for enabling machines to learn how to play chess -- a resource that has not been explored previously. We developed the LEAP corpus, a first and new heterogeneous dataset with structured (chess move notations and board states) and unstructured data (textual descriptions) collected from a chess textbook containing 1164 sentences discussing strategic moves from 91 games. We firstly labelled the sentences based on their relevance, i.e., whether they are discussing a move. Each relevant sentence was then labelled according to its sentiment towards the described move. We performed empirical experiments that assess the performance of various transformer-based baseline models for sentiment analysis. Our results demonstrate the feasibility of employing transformer-based sentiment analysis models for evaluating chess moves, with the best performing model obtaining a weighted micro F_1 score of 68%. Finally, we synthesised the LEAP corpus to create a larger dataset, which can be used as a solution to the limited textual resource in the chess domain.Comment: 27 pages, 10 Figures, 9 Tabel

Diversifying AI: Towards Creative Chess with AlphaZero

In recent years, Artificial Intelligence (AI) systems have surpassed human intelligence in a variety of computational tasks. However, AI systems, like humans, make mistakes, have blind spots, hallucinate, and struggle to generalize to new situations. This work explores whether AI can benefit from creative decision-making mechanisms when pushed to the limits of its computational rationality. In particular, we investigate whether a team of diverse AI systems can outperform a single AI in challenging tasks by generating more ideas as a group and then selecting the best ones. We study this question in the game of chess, the so-called drosophila of AI. We build on AlphaZero (AZ) and extend it to represent a league of agents via a latent-conditioned architecture, which we call AZ_db. We train AZ_db to generate a wider range of ideas using behavioral diversity techniques and select the most promising ones with sub-additive planning. Our experiments suggest that AZ_db plays chess in diverse ways, solves more puzzles as a group and outperforms a more homogeneous team. Notably, AZ_db solves twice as many challenging puzzles as AZ, including the challenging Penrose positions. When playing chess from different openings, we notice that players in AZ_db specialize in different openings, and that selecting a player for each opening using sub-additive planning results in a 50 Elo improvement over AZ. Our findings suggest that diversity bonuses emerge in teams of AI agents, just as they do in teams of humans and that diversity is a valuable asset in solving computationally hard problems

Agentes com aprendizagem automática para jogos de computador

In recent years, new Reinforcement Learning algorithms have been developed. These algorithms use Deep Neural Networks to represent the agent’s knowledge. After surpassing previous Artificial Intelligence (AI) milestones, such as Chess and Go, these Deep Reinforcement Learning (DRL) methods were able to surpass the human level in very complex games like Dota 2, where long-term planning is required and in which professional teams of human players train daily to win e-sports competitions. These algorithms start from scratch, do not use examples of human behavior, and can be applied in various domains. Learning from experience, new and better behaviors were discovered, indicating a lot of potential in these algorithms. However, they require a lot of computational power and training time. Computer games are used in an AI course at the University of Aveiro as an application domain of the AI knowledge acquired by students. The students should develop software agents for these games and try to get the best scores. The objective of this dissertation is to develop agents using the latest DRL techniques and to compare their performance with the agents developed by students. To begin with, DRL agents were developed for a simpler game like Tic-Tac-Toe, where various learning options will be addressed until a robust agent capable of playing against multiple opponents is created. Then, DRL agents capable of playing the version of Pac-Man used in the University of Aveiro course, in the 2018/19 academic year, were developed through the realization of various experiments where the parameters used in the learning process were modified in order to obtain better scores. The developed agent, that obtained the best score, is able to play in all game configurations used in the evaluation of the course and reached the top 7 ranking, among more than 50 agents developed by students that used hard-coded strategies with pathfinding algorithms.Nos últimos anos, novos algoritmos de Aprendizagem por Reforço foram desenvolvidos. Estes algoritmos usam Redes Neuronais Profundas para representar o conhecimento do agente. Após ultrapassarem marcos anteriores da Inteligência Artificial (AI), como o Xadrez e o Go, esses métodos de Aprendizagem Profunda por Reforço (DRL) foram capazes de superar o nível humano em jogos muito complexos como o Dota 2, onde é necessário um planeamento a longo prazo e nos quais equipas profissionais de jogadores humanos treinam diariamente para ganhar competições de desportos eletrónicos. Estes algoritmos começam do zero, não usam exemplos de comportamento humano e podem ser aplicados em vários domínios. Aprendendo pela experiência, novos e melhores comportamentos foram descobertos, indicando um grande potencial nestes algoritmos. No entanto, eles exigem muito poder computacional e tempo de treino. Os jogos de computador são utilizados numa disciplina de AI da Universidade de Aveiro como domínio de aplicação dos conhecimentos de AI adquiridos pelos alunos. Os alunos devem desenvolver agentes de software para esses jogos e tentar obter as melhores pontuações. O objetivo desta dissertação é desenvolver agentes usando as mais recentes técnicas de DRL e comparar o seu desempenho com o dos agentes desenvolvidos pelos alunos. Para começar, os agentes com DRL foram desenvolvidos para um jogo mais simples como o Jogo do Galo, onde várias opções de aprendizagem foram abordadas até ser criado um agente robusto capaz de jogar contra vários oponentes. Posteriormente, foram desenvolvidos agentes com DRL capazes de jogar a versão do Pac-Man utilizada na disciplina da Universidade de Aveiro, no ano letivo de 2018/19, através da realização de diversas experiências onde os parâmetros utilizados no processo de aprendizagem foram modificados de forma a obter melhores pontuações. O agente desenvolvido, que obteve a melhor pontuação, consegue jogar em todas as configurações de jogo utilizadas na avaliação da disciplina e alcançou o top 7 das classificações, entre mais de 50 agentes desenvolvidos por alunos que utilizaram estratégias embutidas no código com algoritmos de pesquisa.Mestrado em Engenharia Informátic

iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement Learning

Navigating safely and efficiently in dense and heterogeneous traffic scenarios is challenging for autonomous vehicles (AVs) due to their inability to infer the behaviors or intentions of nearby drivers. In this work, we introduce a distributed multi-agent reinforcement learning (MARL) algorithm that can predict trajectories and intents in dense and heterogeneous traffic scenarios. Our approach for intent-aware planning, iPLAN, allows agents to infer nearby drivers' intents solely from their local observations. We model two distinct incentives for agents' strategies: Behavioral Incentive for high-level decision-making based on their driving behavior or personality and Instant Incentive for motion planning for collision avoidance based on the current traffic state. Our approach enables agents to infer their opponents' behavior incentives and integrate this inferred information into their decision-making and motion-planning processes. We perform experiments on two simulation environments, Non-Cooperative Navigation and Heterogeneous Highway. In Heterogeneous Highway, results show that, compared with centralized training decentralized execution (CTDE) MARL baselines such as QMIX and MAPPO, our method yields a 4.3% and 38.4% higher episodic reward in mild and chaotic traffic, with 48.1% higher success rate and 80.6% longer survival time in chaotic traffic. We also compare with a decentralized training decentralized execution (DTDE) baseline IPPO and demonstrate a higher episodic reward of 12.7% and 6.3% in mild traffic and chaotic traffic, 25.3% higher success rate, and 13.7% longer survival time