Towards a Universal Test of Social Intelligence

[EN] Under the view of artificial intelligence, an intelligent agent is an autonomous entity which interacts in an environment through observations and actions, trying to achieve one or more goals with the aid of several signals called rewards. The creation of intelligent agents is proliferating during the last decades, and the evaluation of their intelligence is a fundamental issue for their understanding, construction and improvement. Social intelligence is recently obtaining special attention in the creation of intelligent agents due to the current view of human intelligence as highly social. Social intelligence in natural and artificial systems is usually measured by the evaluation of associated traits or tasks that are deemed to represent some facets of social behaviour. The amalgamation of these traits or tasks is then used to configure an operative notion of social intelligence. However, this operative notion does not truly represent what social intelligence is and a definition following this principle will not be precise. Instead, in this thesis we investigate the evaluation of social intelligence in a more formal and general way, by actually considering the evaluee's interaction with other agents. In this thesis we analyse the implications of evaluating social intelligence using a test that evaluates general intelligence. For this purpose, we include other agents into an initially single-agent environment to figure out the issues that appear when evaluating an agent in the context of other agents. From this analysis we obtain useful information for the evaluation of social intelligence. From the lessons learned, we identify the components that should be considered in order to measure social intelligence, and we provide a formal and parametrised definition of social intelligence. This definition calculates an agent's social intelligence as its expected performance in a set of environments with a set of other agents arranged in teams and participating in line-ups, with rewards being re-understood appropriately. This is conceived as a tool to define social intelligence testbeds where we can generate several degrees of competitive and cooperative behaviours. We test this definition by experimentally analysing the influence of teams and agent line-ups for several multi-agent systems with variants of Q-learning agents. However, not all testbeds are appropriate for the evaluation of social intelligence. To facilitate the analysis of a social intelligence testbed, we provide some formal property models about social intelligence in order to characterise the testbed and thus assess its suitability. Finally, we use the presented properties to characterise some social games and multi-agent environments, we make a comparison between them and discuss their strengths and weaknesses in order to evaluate social intelligence.[ES] Bajo la visión de la inteligencia artificial, un agente inteligente es una entidad autónoma la cual interactúa en un entorno a través de observaciones y acciones, tratando de lograr uno o más objetivos con la ayuda de varias señales llamadas recompensas. La creación de agentes inteligentes está proliferando durante las últimas décadas, y la evaluación de su inteligencia es un asunto fundamental para su entendimiento, construcción y mejora. Recientemente la inteligencia social está obteniendo especial atención en la creación de agentes inteligentes debido a la visión actual de la inteligencia humana como altamente social. Normalmente la inteligencia social en sistemas naturales y artificiales se mide mediante la evaluación de rasgos asociados o tareas que se consideran que representan algunas facetas del comportamiento social. La agrupación de estos rasgos o tareas se utiliza entonces para configurar una noción operacional de inteligencia social. Sin embargo, esta noción operacional no representa fielmente a la inteligencia social y no sería posible una definición siguiendo este principio. En su lugar, en esta tesis investigamos la evaluación de la inteligencia social de un modo más formal y general, considerando la interacción del agente a evaluar con otros agentes. En esta tesis analizamos las implicaciones de evaluar la inteligencia social utilizando un test que evalúe la inteligencia general. Con este objetivo incluimos otros agentes en un entorno inicialmente diseñado para un único agente con el fin de averiguar qué cuestiones aparecen cuando evaluamos a un agente en un contexto con otros agentes. A partir de este análisis obtenemos información útil para la evaluación de la inteligencia social. A partir de las lecciones aprendidas identificamos los componentes que deberían considerarse al medir la inteligencia social y proporcionamos una definición formal y parametrizada de esta inteligencia social. Esta definición calcula la inteligencia social de un agente como su rendimiento esperado en un conjunto de entornos y con un conjunto de otros agentes organizados en equipos y distribuidos en alineaciones, reinterpretando apropiadamente las recompensas. Esto se concibe como una herramienta para definir bancos de prueba de inteligencia social donde podamos generar varios grados de comportamientos competitivos y cooperativos. Probamos esta definición analizando experimentalmente la influencia de los equipos y las alineaciones de agentes en varios sistemas multiagente con variantes de agentes Q-learning. Sin embargo, no todos los bancos de prueba son apropiados para la evaluación de la inteligencia social. Para facilitar el análisis de un banco de pruebas de inteligencia social, proporcionamos algunos modelos de propiedades formales sobre la inteligencia social con el objetivo de caracterizar el banco de pruebas y así valorar su idoneidad. Finalmente, usamos las propiedades presentadas para caracterizar algunos juegos sociales y entornos multiagente, hacemos una comparación entre ellos y discutimos sus puntos fuertes y débiles para ser usados en la evaluación de la inteligencia social.[CA] Davall la visió de la intel·ligència artificial, un agent intel·ligent és una entitat autònoma la qual interactua en un entorn a través d'observacions i accions, tractant d'aconseguir un o més objectius amb l'ajuda de diverses senyals anomenades recompenses. La creació d'agents intel·ligents està proliferant durant les últimes dècades, i l'avaluació de la seua intel·ligència és un assumpte fonamental per al seu enteniment, construcció i millora. Recentment la intel·ligència social està obtenint especial atenció en la creació d'agents intel·ligents a causa de la visió actual de la intel·ligència humana com altament social. Normalment la intel·ligència social en sistemes naturals i artificials es mesura per mitjà de l'avaluació de trets associats o tasques que es consideren que representen algunes facetes del comportament social. L'agrupació d'aquests trets o tasques s'utilitza llavors per a configurar una noció operacional d'intel·ligència social. No obstant això, aquesta noció operacional no representa fidelment a la intel·ligència social i no seria possible una definició seguint aquest principi. En el seu lloc, en aquesta tesi investiguem l'avaluació de la intel·ligència social d'una manera més formal i general, considerant la interacció de l'agent a avaluar amb altres agents. En aquesta tesi analitzem les implicacions d'avaluar la intel·ligència social utilitzant un test que avalue la intel·ligència general. Amb aquest objectiu incloem altres agents en un entorn inicialment dissenyat per a un únic agent amb la finalitat d'esbrinar quines qüestions apareixen quan avaluem un agent en un context amb altres agents. A partir d'aquesta anàlisi obtenim informació útil per a l'avaluació de la intel·ligència social. A partir de les lliçons apreses identifiquem els components que haurien de considerar-se al mesurar la intel·ligència social i proporcionem una definició formal i parametrizada d'aquesta intel·ligència social. Aquesta definició calcula la intel·ligència social d'un agent com el seu rendiment esperat en un conjunt d'entorns i amb un conjunt d'altres agents organitzats en equips i distribuïts en alineacions, reinterpretant apropiadament les recompenses. Açò es concep com una ferramenta per a definir bancs de prova d'intel·ligència social on podem generar diversos graus de comportaments competitius i cooperatius. Provem aquesta definició analitzant experimentalment la influència dels equips i les alineacions d'agents en diversos sistemes multiagent amb variants d'agents Q-learning. No obstant això, no tots els bancs de prova són apropiats per a l'avaluació de la intel·ligència social. Per a facilitar l'anàlisi d'un banc de proves d'intel·ligència social, proporcionem alguns models de propietats formals sobre la intel·ligència social amb l'objectiu de caracteritzar el banc de proves i així valorar la seua idoneïtat. Finalment, usem les propietats presentades per a caracteritzar alguns jocs socials i entorns multiagent, fem una comparació entre ells i discutim els seus punts forts i dèbils per a ser usats en l'avaluació de la intel·ligència social.Insa Cabrera, J. (2016). Towards a Universal Test of Social Intelligence [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/66080TESI

CLEAN learning to improve coordination and scalability in multiagent systems

Recent advances in multiagent learning have led to exciting new capabilities spanning fields as diverse as planetary exploration, air traffic control, military reconnaissance, and airport security. Such algorithms provide a tangible benefit over traditional control algorithms in that they allow fast responses, adapt to dynamic environments, and generally scale well. Unfortunately, because many existing multiagent learning methods are extensions of single agent approaches, they are inhibited by three key issues: i) they treat the actions of other agents as "environmental noise" in an attempt to simplify the problem complexity, ii) they are slow to converge in large systems as the joint action space grows exponentially in the number of agents, and iii) they frequently rely upon the presence of an accurate system model being readily available. This work addresses these three issues sequentially. First, we improve overall learning performance compared to existing state-of-the-art techniques in the field by embracing the exploration in learning rather than ignoring it or approximating it away. Within multiagent systems, exploration by individual agents significantly alters the dynamics of the environment in which all agents learn. To address this, we introduce the concept of "private" exploration, which enables each agent to present a stationary baseline policy to other agents in order to allow other agents in the system to learn more efficiently. In particular, we introduce Coordinated Learning without Exploratory Action Noise (CLEAN) rewards which improve coordination and performance by utilizing the concept of private exploration in order to remove the negative impact of traditional "public" exploration strategies from learning in multiagent systems. Next, we leverage the fundamental properties of CLEAN rewards that enable private exploration to allow agents to explore multiple potential actions concurrently in a "batch mode" in order to significantly improve learning speed over the state-of-the-art. Finally, we improve the real-world applicability of the proposed techniques by reducing their requirements. Specifically, the CLEAN rewards developed require an accurate partial model (i.e., an accurate model of the system objective) of the system in order to be computed. Unfortunately, many real-world systems are too complex to be modeled or are not known in advance, so an accurate system model is not available a priori. We address this shortcoming by employing model-based reinforcement learning techniques to enable agents to construct their own approximate model of the system objective based upon their observations and use this approximate model to calculate their CLEAN rewards.Keywords: Multiagent Coordination, Multiagent Learning, UAV Communication Network, Fractionated Satellites, UAV Swarms, Distributed Control, Multiagent Scalability, Learning based control, Reward Shaping, Cubesats, Multiagent systems, Solar Power UAVs, Satellite Constellation

Decision shaping and strategy learning in multi-robot interactions

Recent developments in robot technology have contributed to the advancement of autonomous behaviours in human-robot systems; for example, in following instructions received from an interacting human partner. Nevertheless, increasingly many systems are moving towards more seamless forms of interaction, where factors such as implicit trust and persuasion between humans and robots are brought to the fore. In this context, the problem of attaining, through suitable computational models and algorithms, more complex strategic behaviours that can influence human decisions and actions during an interaction, remains largely open. To address this issue, this thesis introduces the problem of decision shaping in strategic interactions between humans and robots, where a robot seeks to lead, without however forcing, an interacting human partner to a particular state. Our approach to this problem is based on a combination of statistical modeling and synthesis of demonstrated behaviours, which enables robots to efficiently adapt to novel interacting agents. We primarily focus on interactions between autonomous and teleoperated (i.e. human-controlled) NAO humanoid robots, using the adversarial soccer penalty shooting game as an illustrative example. We begin by describing the various challenges that a robot operating in such complex interactive environments is likely to face. Then, we introduce a procedure through which composable strategy templates can be learned from provided human demonstrations of interactive behaviours. We subsequently present our primary contribution to the shaping problem, a Bayesian learning framework that empirically models and predicts the responses of an interacting agent, and computes action strategies that are likely to influence that agent towards a desired goal. We then address the related issue of factors affecting human decisions in these interactive strategic environments, such as the availability of perceptual information for the human operator. Finally, we describe an information processing algorithm, based on the Orient motion capture platform, which serves to facilitate direct (as opposed to teleoperation-mediated) strategic interactions between humans and robots. Our experiments introduce and evaluate a wide range of novel autonomous behaviours, where robots are shown to (learn to) influence a variety of interacting agents, ranging from other simple autonomous agents, to robots controlled by experienced human subjects. These results demonstrate the benefits of strategic reasoning in human-robot interaction, and constitute an important step towards realistic, practical applications, where robots are expected to be not just passive agents, but active, influencing participants