Biases for Emergent Communication in Multi-agent Reinforcement Learning

We study the problem of emergent communication, in which language arises because speakers and listeners must communicate information in order to solve tasks. In temporally extended reinforcement learning domains, it has proved hard to learn such communication without centralized training of agents, due in part to a difficult joint exploration problem. We introduce inductive biases for positive signalling and positive listening, which ease this problem. In a simple one-step environment, we demonstrate how these biases ease the learning problem. We also apply our methods to a more extended environment, showing that agents with these inductive biases achieve better performance, and analyse the resulting communication protocols.Comment: Accepted at NeurIPS 201

Contributions to artificial intelligence: the IIIA perspective

La intel·ligència artificial (IA) és un camp científic i tecnològic relativament nou dedicat a l'estudi de la intel·ligència mitjançant l'ús d'ordinadors com a eines per produir comportament intel·ligent. Inicialment, l'objectiu era essencialment científic: assolir una millor comprensió de la intel·ligència humana. Aquest objectiu ha estat, i encara és, el dels investigadors en ciència cognitiva. Dissortadament, aquest fascinant però ambiciós objectiu és encara molt lluny de ser assolit i ni tan sols podem dir que ens hi haguem acostat significativament. Afortunadament, però, la IA també persegueix un objectiu més aplicat: construir sistemes que ens resultin útils encara que la intel·ligència artificial de què estiguin dotats no tingui res a veure amb la intel·ligència humana i, per tant, aquests sistemes no ens proporcionarien necessàriament informació útil sobre la naturalesa de la intel·ligència humana. Aquest objectiu, que s'emmarca més aviat dins de l'àmbit de l'enginyeria, és actualment el que predomina entre els investigadors en IA i ja ha donat resultats impresionants, tan teòrics com aplicats, en moltíssims dominis d'aplicació. A més, avui dia, els productes i les aplicacions al voltant de la IA representen un mercat anual de desenes de milers de milions de dòlars. Aquest article resumeix les principals contribucions a la IA fetes pels investigadors de l'Institut d'Investigació en Intel·ligència Artificial del Consell Superior d'Investigacions Científiques durant els darrers cinc anys.Artificial intelligence is a relatively new scientific and technological field which studies the nature of intelligence by using computers to produce intelligent behaviour. Initially, the main goal was a purely scientific one, understanding human intelligence, and this remains the aim of cognitive scientists. Unfortunately, such an ambitious and fascinating goal is not only far from being achieved but has yet to be satisfactorily approached. Fortunately, however, artificial intelligence also has an engineering goal: building systems that are useful to people even if the intelligence of such systems has no relation whatsoever with human intelligence, and therefore being able to build them does not necessarily provide any insight into the nature of human intelligence. This engineering goal has become the predominant one among artificial intelligence researchers and has produced impressive results, ranging from knowledge-based systems to autonomous robots, that have been applied to many different domains. Furthermore, artificial intelligence products and services today represent an annual market of tens of billions of dollars worldwide. This article summarizes the main contributions to the field of artificial intelligence made at the IIIA-CSIC (Artificial Intelligence Research Institute of the Spanish Scientific Research Council) over the last five years

Байєсівські мережі в системах підтримки прийняття рішень

Пропонується докладне висвітлення сучасних підходів до моделювання процесів довільної природи за допомогою байєсівських мереж (БМ) і дерев рішень. Байєсівська мережа – ймовірнісна модель, преставлена у формі спрямованого ациклічного графа, вершинами якого є змінні досліджуваного процесу. БМ – потужний сучасний інструмент моделювання процесів та об’єктів, які функціонують в умовах наявності невизначеностей довільної природи. Їх успішно використовують для розв’язання задач прогнозування, передбачення, медичної і технічної діагностики, прийняття управлінських рішень, автоматичного керування і т. ін. Розглянуто теорію побудови байєсівських мереж, яка включає задачі навчання структури мережі та формування ймовірнісного висновку на її основі. Наведено практичні методики побудови (оцінювання) структури мережі на основі статистичних даних і експертних оцінок. Докладно описано відповідні алгоритмічні процедури. Окремо розглянуто варіанти використання дискретних і неперервних змінних, а також можливості створення гібридної мережі. Наведено кілька методів обчислення ймовірнісного висновку за допомогою побудованої мережі, у тому числі методи формування точного і наближеного висновків. Докладно розглянуто приклади розв’язання практичних задач за допомогою мереж Байєса. Зокрема, задачі моделювання, прогнозування і розпізнавання образів. Наведено перелік відомих програмних продуктів та їх виробників для побудови та застосування байєсівських мереж, частина з яких є повністю доступними для використання у мережі Інтернет. Деякі системи можна доповнювати новими програмними модулями. Книга рекомендується як навчальний посібник для студентів, аспірантів та викладачів, а також для інженерів, які спеціалізуються у галузі розв’язання задач ймовірнісного математичного моделювання, прогнозування, передбачення і розпізнавання образів процесів довільної природи, інформація стосовно який представлена статистичними даними та експертними оцінками

Spectral decomposition method of dialog state tracking via collective matrix factorization

Revised versionThe task of dialog management is commonly decomposed into two sequential subtasks: dialog state tracking and dialog policy learning. In an end-to-end dialog system, the aim of dialog state tracking is to accurately estimate the true dialog state from noisy observations produced by the speech recognition and the natural language understanding modules. The state tracking task is primarily meant to support a dialog policy. From a probabilistic perspective, this is achieved by maintaining a posterior distribution over hidden dialog states composed of a set of context dependent variables. Once a dialog policy is learned, it strives to select an optimal dialog act given the estimated dialog state and a defined reward function. This paper introduces a novel method of dialog state tracking based on a bilinear algebric decomposition model that provides an efficient inference schema through collective matrix factorization. We evaluate the proposed approach on the second Dialog State Tracking Challenge (DSTC-2) dataset and we show that the proposed tracker gives encouraging results compared to the state-of-the-art trackers that participated in this standard benchmark. Finally, we show that the prediction schema is computationally efficient in comparison to the previous approaches

Learning Score-Optimal Chordal Markov Networks via Branch and Bound

Graphical models are commonly used to encode conditional independence assumptions between random variables. Here we focus on undirected graphical models called chordal Markov networks. Specifically, we will consider the chordal Markov network structure learning problem (CMSL), where the aim is to find (or "learn") a graph structure that best fits the given data with respect to a given decomposable scoring function. We introduce a branch and bound search algorithm for CMSL which represents chordal Markov network structures as decomposable DAGs. We show how revisiting equivalent solution candidates can be avoided in the search by detecting symmetries among graph structures. For the symmetry breaking we apply specific rules by van Beek and Hoffman (CP 2015), and also propose a new rule that takes advantage of the special nature of decomposable DAGs. In addition, we show how we can achieve on-the-fly score pruning for CMSL. We also propose methods for obtaining strong upper bounds for CMSL that help us close branches in the search tree. We implement a dynamic programming algorithm to find the optimal Bayesian network structures and then use the scores of those graphs as upper bounds. We also show how we can relax the requirement for decomposability in decomposable DAGs in order to achieve even stronger upper bounds. Furthermore, we propose a method for obtaining an initial lower bound in CMSL by turning a Bayesian network structure into a chordal Markov network structure. Empirically we show that our approach is competitive with the recently proposed CMSL algorithms by being able to sometimes scale up to 20 variables within 24 hours with unbounded treewidth. We also report that our branch and bound requires considerably less memory than the fastest of the recently proposed algorithms for CMSL

Activity, context, and plan recognition with computational causal behavior models

Objective of this thesis is to answer the question "how to achieve efficient sensor-based reconstruction of causal structures of human behaviour in order to provide assistance?". To answer this question, the concept of Computational Causal Behaviour Models (CCBMs) is introduced. CCBM allows the specification of human behaviour by means of preconditions and effects and employs Bayesian filtering techniques to reconstruct action sequences from noisy and ambiguous sensor data. Furthermore, a novel approximative inference algorithm – the Marginal Filter – is introduced

Achieving Causal Fairness in Machine Learning

Fairness is a social norm and a legal requirement in today\u27s society. Many laws and regulations (e.g., the Equal Credit Opportunity Act of 1974) have been established to prohibit discrimination and enforce fairness on several grounds, such as gender, age, sexual orientation, race, and religion, referred to as sensitive attributes. Nowadays machine learning algorithms are extensively applied to make important decisions in many real-world applications, e.g., employment, admission, and loans. Traditional machine learning algorithms aim to maximize predictive performance, e.g., accuracy. Consequently, certain groups may get unfairly treated when those algorithms are applied for decision-making. Therefore, it is an imperative task to develop fairness-aware machine learning algorithms such that the decisions made by them are not only accurate but also subject to fairness requirements. In the literature, machine learning researchers have proposed association-based fairness notions, e.g., statistical parity, disparate impact, equality of opportunity, etc., and developed respective discrimination mitigation approaches. However, these works did not consider that fairness should be treated as a causal relationship. Although it is well known that association does not imply causation, the gap between association and causation is not paid sufficient attention by the fairness researchers and stakeholders. The goal of this dissertation is to study fairness in machine learning, define appropriate fairness notions, and develop novel discrimination mitigation approaches from a causal perspective. Based on Pearl\u27s structural causal model, we propose to formulate discrimination as causal effects of the sensitive attribute on the decision. We consider different types of causal effects to cope with different situations, including the path-specific effect for direct/indirect discrimination, the counterfactual effect for group/individual discrimination, and the path-specific counterfactual effect for general cases. In the attempt to measure discrimination, the unidentifiable situations pose an inevitable barrier to the accurate causal inference. To address this challenge, we propose novel bounding methods to accurately estimate the strength of unidentifiable fairness notions, including path-specific fairness, counterfactual fairness, and path-specific counterfactual fairness. Based on the estimation of fairness, we develop novel and efficient algorithms for learning fair classification models. Besides classification, we also investigate the discrimination issues in other machine learning scenarios, such as ranked data analysis