An Algorithmic Framework for Strategic Fair Division

We study the paradigmatic fair division problem of allocating a divisible good among agents with heterogeneous preferences, commonly known as cake cutting. Classical cake cutting protocols are susceptible to manipulation. Do their strategic outcomes still guarantee fairness? To address this question we adopt a novel algorithmic approach, by designing a concrete computational framework for fair division---the class of Generalized Cut and Choose (GCC) protocols}---and reasoning about the game-theoretic properties of algorithms that operate in this model. The class of GCC protocols includes the most important discrete cake cutting protocols, and turns out to be compatible with the study of fair division among strategic agents. In particular, GCC protocols are guaranteed to have approximate subgame perfect Nash equilibria, or even exact equilibria if the protocol's tie-breaking rule is flexible. We further observe that the (approximate) equilibria of proportional GCC protocols---which guarantee each of the $n$ agents a $1/n$-fraction of the cake---must be (approximately) proportional. Finally, we design a protocol in this framework with the property that its Nash equilibrium allocations coincide with the set of (contiguous) envy-free allocations

Argue to agree: A case-based argumentation approach

[EN] The capability of reaching agreements is a necessary feature that large computer systems where agents interoperate must include. In these systems, agents represent self-motivated entities that have a social context, including dependency relations among them, and different preferences and beliefs. Without agreement there is no cooperation and thus, complex tasks which require the interaction of agents with different points of view cannot be performed. In this work, we propose a case-based argumentation approach for Multi-Agent Systems where agents reach agreements by arguing and improve their argumentation skills from experience. A set of knowledge resources and a reasoning process that agents can use to manage their positions and arguments are presented. These elements are implemented and validated in a customer support application.This work is supported by the Spanish government grants [CONSOLIDER-INGENIO 2010 CSD2007-00022, TIN2008-04446, and TIN2009-13839-C03-01] and by the GVA project [PROMETEO 2008/051].Heras Barberá, SM.; Jordán Prunera, JM.; Botti, V.; Julian Inglada, VJ. (2013). Argue to agree: A case-based argumentation approach. International Journal of Approximate Reasoning. 54(1):82-108. https://doi.org/10.1016/j.ijar.2012.06.005S8210854

Reasoning about norms under uncertainty in dynamic environments

The behaviour of norm-autonomous agents is determined by their goals and the norms that are explicitly represented inside their minds. Thus, they require mechanisms for acquiring and accepting norms, determining when norms are relevant to their case, and making decisions about norm compliance. Up un- til now the existing proposals on norm-autonomous agents assume that agents interact within a deterministic environment that is certainly perceived. In prac- tice, agents interact by means of sensors and actuators under uncertainty with non-deterministic and dynamic environments. Therefore, the existing propos- als are unsuitable or, even, useless to be applied when agents have a physical presence in some real-world environment. In response to this problem we have developed the n-BDI architecture. In this paper, we propose a multi -context graded BDI architecture (called n-BDI) that models norm-autonomous agents able to deal with uncertainty in dynamic environments. The n-BDI architecture has been experimentally evaluated and the results are shown in this paper.This paper was partially funded by the Spanish government under Grant CONSOLIDER-INGENIO 2010 CSD2007-00022 and the Valencian government under Project PROMETEOH/2013/019.Criado Pacheco, N.; Argente, E.; Noriega, P.; Botti Navarro, VJ. (2014). Reasoning about norms under uncertainty in dynamic environments. International Journal of Approximate Reasoning. 55(9):2049-2070. https://doi.org/10.1016/j.ijar.2014.02.004S2049207055

Modeling Mutual Influence in Multi-Agent Reinforcement Learning

In multi-agent systems (MAS), agents rarely act in isolation but tend to achieve their goals through interactions with other agents. To be able to achieve their ultimate goals, individual agents should actively evaluate the impacts on themselves of other agents' behaviors before they decide which actions to take. The impacts are reciprocal, and it is of great interest to model the mutual influence of agent's impacts with one another when they are observing the environment or taking actions in the environment. In this thesis, assuming that the agents are aware of each other's existence and their potential impact on themselves, I develop novel multi-agent reinforcement learning (MARL) methods that can measure the mutual influence between agents to shape learning. The first part of this thesis outlines the framework of recursive reasoning in deep multi-agent reinforcement learning. I hypothesize that it is beneficial for each agent to consider how other agents react to their behavior. I start from Probabilistic Recursive Reasoning (PR2) using level-1 reasoning and adopt variational Bayes methods to approximate the opponents' conditional policies. Each agent shapes the individual Q-value by marginalizing the conditional policies in the joint Q-value and finding the best response to improving their policies. I further extend PR2 to Generalized Recursive Reasoning (GR2) with different hierarchical levels of rationality. GR2 enables agents to possess various levels of thinking ability, thereby allowing higher-level agents to best respond to less sophisticated learners. The first part of the thesis shows that eliminating the joint Q-value to an individual Q-value via explicitly recursive reasoning would benefit the learning. In the second part of the thesis, in reverse, I measure the mutual influence by approximating the joint Q-value based on the individual Q-values. I establish Q-DPP, an extension of the Determinantal Point Process (DPP) with partition constraints, and apply it to multi-agent learning as a function approximator for the centralized value function. An attractive property of using Q-DPP is that when it reaches the optimum value, it can offer a natural factorization of the centralized value function, representing both quality (maximizing reward) and diversity (different behaviors). In the third part of the thesis, I depart from the action-level mutual influence and build a policy-space meta-game to analyze agents' relationship between adaptive policies. I present a Multi-Agent Trust Region Learning (MATRL) algorithm that augments single-agent trust region policy optimization with a weak stable fixed point approximated by the policy-space meta-game. The algorithm aims to find a game-theoretic mechanism to adjust the policy optimization steps that force the learning of all agents toward the stable point

The heuristic conception of inference to the best explanation

An influential suggestion about the relationship between Bayesianism and inference to the best explanation holds that IBE functions as a heuristic to approximate Bayesian reasoning. While this view promises to unify Bayesianism and IBE in a very attractive manner, important elements of the view have not yet been spelled out in detail. I present and argue for a heuristic conception of IBE on which IBE serves primarily to locate the most probable available explanatory hypothesis to serve as a working hypothesis in an agent’s further investigations. Along the way, I criticize what I consider to be an overly ambitious conception of the heuristic role of IBE, according to which IBE serves as a guide to absolute probability values. My own conception, by contrast, requires only that IBE can function as a guide to the comparative probability values of available hypotheses. This is shown to be a much more realistic role for IBE given the nature and limitations of the explanatory considerations with which IBE operates