A software toolkit for web-based virtual environments based on a shared database

We propose a software toolkit for developing complex web-based user interfaces, incorporating such things as multi-user facilities, virtual environments (VEs), and interface agents. The toolkit is based on a novel software architecture that combines ideas from multi-agent platforms and user interface (UI) architectures. It provides a distributed shared database with publish-subscribe facilities. This enables UI components to observe the state and activities of any other components in the system easily. The system runs in a web-based environment. The toolkit is comprised of several programming and other specification languages, providing a complete suite of systems design languages. We illustrate the toolkit by means of a couple of examples

ViZDoom Competitions: Playing Doom from Pixels

This paper presents the first two editions of Visual Doom AI Competition, held in 2016 and 2017. The challenge was to create bots that compete in a multi-player deathmatch in a first-person shooter (FPS) game, Doom. The bots had to make their decisions based solely on visual information, i.e., a raw screen buffer. To play well, the bots needed to understand their surroundings, navigate, explore, and handle the opponents at the same time. These aspects, together with the competitive multi-agent aspect of the game, make the competition a unique platform for evaluating the state of the art reinforcement learning algorithms. The paper discusses the rules, solutions, results, and statistics that give insight into the agents' behaviors. Best-performing agents are described in more detail. The results of the competition lead to the conclusion that, although reinforcement learning can produce capable Doom bots, they still are not yet able to successfully compete against humans in this game. The paper also revisits the ViZDoom environment, which is a flexible, easy to use, and efficient 3D platform for research for vision-based reinforcement learning, based on a well-recognized first-person perspective game Doom

Artificial Intelligence and Systems Theory: Applied to Cooperative Robots

This paper describes an approach to the design of a population of cooperative robots based on concepts borrowed from Systems Theory and Artificial Intelligence. The research has been developed under the SocRob project, carried out by the Intelligent Systems Laboratory at the Institute for Systems and Robotics - Instituto Superior Tecnico (ISR/IST) in Lisbon. The acronym of the project stands both for "Society of Robots" and "Soccer Robots", the case study where we are testing our population of robots. Designing soccer robots is a very challenging problem, where the robots must act not only to shoot a ball towards the goal, but also to detect and avoid static (walls, stopped robots) and dynamic (moving robots) obstacles. Furthermore, they must cooperate to defeat an opposing team. Our past and current research in soccer robotics includes cooperative sensor fusion for world modeling, object recognition and tracking, robot navigation, multi-robot distributed task planning and coordination, including cooperative reinforcement learning in cooperative and adversarial environments, and behavior-based architectures for real time task execution of cooperating robot teams

Towards next generation coordination infrastructures

Coordination infrastructures play a central role in the engineering of multiagent systems. Since the advent of agent technology, research on coordination infrastructures has produced a significant number of infrastructures with varying features. In this paper, we review the the state-of-the-art coordination infrastructures with the purpose of identifying open research challenges that next generation coordination infrastructures should address. Our analysis concludes that next generation coordination infrastructures must address a number of challenges: (i) to become socially aware, by facilitating human interaction within a MAS; (ii) to assist agents in their decision making by providing decision support that helps them reduce the scope of reasoning and facilitates the achievement of their goals; and (iii) to increase openness to support on-line, fully decentralised design and execution. Furthermore, we identify some promising approaches in the literature, together with the research issues worth investigating, to cope with such challenges. © Cambridge University Press, 2015.The work presented in this paper has been partially funded by projects EVE (TIN2009-14702-C02-01), AT (CSD2007-0022), and the Generalitat of Catalunya grant 2009-SGR-1434Peer Reviewe

Towards next generation coordination infrastructures

Coordination infrastructures play a central role in the engineering of multiagent systems. Since the advent of agent technology, research on coordination infrastructures has produced a significant number of infrastructures with varying features. In this paper, we review the the state-of-the-art coordination infrastructures with the purpose of identifying open research challenges that next generation coordination infrastructures should address. Our analysis concludes that next generation coordination infrastructures must address a number of challenges: (i) to become socially aware, by facilitating human interaction within a MAS; (ii) to assist agents in their decision making by providing decision support that helps them reduce the scope of reasoning and facilitates the achievement of their goals; and (iii) to increase openness to support on-line, fully decentralised design and execution. Furthermore, we identify some promising approaches in the literature, together with the research issues worth investigating, to cope with such challenges

Relational Representations in Reinforcement Learning: Review and Open Problems

This paper is about representation in RL.We discuss some of the concepts in representation and generalization in reinforcement learning and argue for higher-order representations, instead of the commonly used propositional representations. The paper contains a small review of current reinforcement learning systems using higher-order representations, followed by a brief discussion. The paper ends with research directions and open problems.\u