Making friends on the fly : advances in ad hoc teamwork

textGiven the continuing improvements in design and manufacturing processes in addition to improvements in artificial intelligence, robots are being deployed in an increasing variety of environments for longer periods of time. As the number of robots grows, it is expected that they will encounter and interact with other robots. Additionally, the number of companies and research laboratories producing these robots is increasing, leading to the situation where these robots may not share a common communication or coordination protocol. While standards for coordination and communication may be created, we expect that any standards will lag behind the state-of-the-art protocols and robots will need to additionally reason intelligently about their teammates with limited information. This problem motivates the area of ad hoc teamwork in which an agent may potentially cooperate with a variety of teammates in order to achieve a shared goal. We argue that agents that effectively reason about ad hoc teamwork need to exhibit three capabilities: 1) robustness to teammate variety, 2) robustness to diverse tasks, and 3) fast adaptation. This thesis focuses on addressing all three of these challenges. In particular, this thesis introduces algorithms for quickly adapting to unknown teammates that enable agents to react to new teammates without extensive observations. The majority of existing multiagent algorithms focus on scenarios where all agents share coordination and communication protocols. While previous research on ad hoc teamwork considers some of these three challenges, this thesis introduces a new algorithm, PLASTIC, that is the first to address all three challenges in a single algorithm. PLASTIC adapts quickly to unknown teammates by reusing knowledge it learns about previous teammates and exploiting any expert knowledge available. Given this knowledge, PLASTIC selects which previous teammates are most similar to the current ones online and uses this information to adapt to their behaviors. This thesis introduces two instantiations of PLASTIC. The first is a model-based approach, PLASTIC-Model, that builds models of previous teammates' behaviors and plans online to determine the best course of action. The second uses a policy-based approach, PLASTIC-Policy, in which it learns policies for cooperating with past teammates and selects from among these policies online. Furthermore, we introduce a new transfer learning algorithm, TwoStageTransfer, that allows transferring knowledge from many past teammates while considering how similar each teammate is to the current ones. We theoretically analyze the computational tractability of PLASTIC-Model in a number of scenarios with unknown teammates. Additionally, we empirically evaluate PLASTIC in three domains that cover a spread of possible settings. Our evaluations show that PLASTIC can learn to communicate with unknown teammates using a limited set of messages, coordinate with externally-created teammates that do not reason about ad hoc teams, and act intelligently in domains with continuous states and actions. Furthermore, these evaluations show that TwoStageTransfer outperforms existing transfer learning algorithms and enables PLASTIC to adapt even better to new teammates. We also identify three dimensions that we argue best describe ad hoc teamwork scenarios. We hypothesize that these dimensions are useful for analyzing similarities among domains and determining which can be tackled by similar algorithms in addition to identifying avenues for future research. The work presented in this thesis represents an important step towards enabling agents to adapt to unknown teammates in the real world. PLASTIC significantly broadens the robustness of robots to their teammates and allows them to quickly adapt to new teammates by reusing previously learned knowledge.Computer Science

A Survey of Ad Hoc Teamwork: Definitions, Methods, and Open Problems

Ad hoc teamwork is the well-established research problem of designing agents that can collaborate with new teammates without prior coordination. This survey makes a two-fold contribution. First, it provides a structured description of the different facets of the ad hoc teamwork problem. Second, it discusses the progress that has been made in the field so far, and identifies the immediate and long-term open problems that need to be addressed in the field of ad hoc teamwork

The Role of Models and Communication in the Ad Hoc Multiagent Team Decision Problem

Abstract Ad hoc teams are formed of members who have little or no information regarding one another. In order to achieve a shared goal, agents are tasked with learning the capabilities of their teammates such that they can coordinate effectively. Typically, the capabilities of the agent teammates encountered are constrained by the particular domain specifications. However, for wide application, it is desirable to develop systems that are able to coordinate with general ad hoc agents independent of the choice of domain. We propose examining ad hoc multiagent teamwork from a generalized perspective and discuss existing domains within the context of our framework. Furthermore, we consider how communication of agent intentions can provide a means of reducing teammate model uncertainty at key junctures, requiring an agent to consider its own information deficiencies in order to form communicative acts improving team coordination

Ad-hoc teamwork with behavior-switching agents

As autonomous AI agents proliferate in the real world, they will increasingly need to cooperate with each other to achieve complex goals without always being able to coordinate in advance. This kind of cooperation, in which agents have to learn to cooperate on the fly, is called ad hoc teamwork. Many previous works investigating this setting assumed that teammates behave according to one of many predefined types that is fixed throughout the task. This assumption of stationarity in behaviors, is a strong assumption which cannot be guaranteed in many real-world settings. In this work, we relax this assumption and investigate settings in which teammates can change their types during the course of the task. This adds complexity to the planning problem as now an agent needs to recognize that a change has occurred in addition to figuring out what is the new type of the teammate it is interacting with. In this paper, we present a novel Convolutional-Neural-Network-based Change Point Detection (CPD) algorithm for ad hoc teamwork. When evaluating our algorithm on the modified predator prey domain, we find that it outperforms existing Bayesian CPD algorithms.Electrical and Computer Engineerin

Combining a Meta-Policy and Monte-Carlo Planning for Scalable Type-Based Reasoning in Partially Observable Environments

The design of autonomous agents that can interact effectively with other agents without prior coordination is a core problem in multi-agent systems. Type-based reasoning methods achieve this by maintaining a belief over a set of potential behaviours for the other agents. However, current methods are limited in that they assume full observability of the state and actions of the other agent or do not scale efficiently to larger problems with longer planning horizons. Addressing these limitations, we propose Partially Observable Type-based Meta Monte-Carlo Planning (POTMMCP) - an online Monte-Carlo Tree Search based planning method for type-based reasoning in large partially observable environments. POTMMCP incorporates a novel meta-policy for guiding search and evaluating beliefs, allowing it to search more effectively to longer horizons using less planning time. We show that our method converges to the optimal solution in the limit and empirically demonstrate that it effectively adapts online to diverse sets of other agents across a range of environments. Comparisons with the state-of-the art method on problems with up to $10^{14}$ states and $10^8$ observations indicate that POTMMCP is able to compute better solutions significantly faster.Comment: 24 page

Information Uses and Learning Outcomes During Guided Discovery in a Blended E-Learning Game Design Program for Secondary Computer Science Education

This study investigates middle school and high school students’ online information uses and social constructivist engagement during a blended e-learning program of game design for computer science education. Students use a learning management system (LMS) pre-populated with curriculum and resources, participating in an in-school class, daily for credit and a grade for a year, with non-expert teachers. This blended e-learning model aims to contribute to scaling of CS education, towards meeting the needs of teacher shortages in this domain. The study draws on Google Analytics data to describe student activity patterns and investigate relationships between measured patterns and learning outcomes. Findings show two activity factors emerging in student resource uses (less advanced, more advanced), and correlations between uses of more advanced resource, and outcomes. Further, student uses of the “team page,” the locus of their social constructivist game design engagement online, are highly correlated with outcomes. The research offers some support for effectiveness of such blended learning approaches in supporting CS education in this age group through knowledge-building, while also showing areas for improvement in instructional design, including direct scaffolding of information literacy instruction in such contexts

Recruiting employees to work in teams: The impact of perceptions, KSAs, and recruitment source on pre-hire recruitment variables

Teams perform essential roles in many modern organizations and are therefore the tied to organizational success. The purpose of the current study was to examine the recruitment of employees to work in teams through an investigation into the impact of perceptions of teams, teamwork KSAs, and recruitment source on pre-hire recruitment variables in team and individual positions. A 2 x 3 repeated measures design presented participants with team and individual job postings on three online recruitment sources (organizational websites, online site visits, and referrals). Results support the idea that perceptions of teams do influence pre-hire recruitment variables to team and individual positions. However, relationships were not observed between teamwork KSAs and pre-hire recruitment variables with the exception of perceptions of organizational honesty. Furthermore, results indicated that differences do exist between recruitment sources with organizational websites leading to higher per-hire recruitment variables than online site visits and referrals

Use of IC information in Japanese financial firms

Purpose – The purpose of this paper is to explore the perceptions of: how Japanese financial firms (JFF) acquire and use company intellectual capital (IC) information in their common routine equity investment decisions, how this activity contributes to knowledge creation in the JFFs, and how investee company knowledge creation is affected by the JFFs.<p></p> Design/methodology/approach – The research employed a multi-case design, using four JFF cases. The investigation was performed in terms of Nonaka and Toyama's “theory of the knowledge creating firm”.<p></p> Findings – IC information contributed to earnings estimates and company valuation. Emotional information contributed to JFF feelings and confidence in their information use and valuation. JFF knowledge was an important component of the key interacting and informed contexts used by JFFs. This generated opportunities to improve disclosure and accountability between JFFs and their investee companies. Common patterns of behaviour across the JFFs were counterbalanced by variety and differences noted in JFF behaviour.<p></p> Practical implications – The findings provide important insights into how JFF knowledge creating patterns could limit or progress a common language of communication between companies and markets on the subject of IC. This could impact on the quality of corporate disclosure and accountability processes.<p></p> Originality/value – The paper demonstrates that there is a need for further use of qualitative studies of financial market behavior. Especially in the area of understanding the communication of IC between firms and financial markets, the potential of using sociology of finance approaches appears to be considerable