Agent Teaming Situation Awareness (ATSA): A Situation Awareness Framework for Human-AI Teaming

The rapid advancements in artificial intelligence (AI) have led to a growing trend of human-AI teaming (HAT) in various fields. As machines continue to evolve from mere automation to a state of autonomy, they are increasingly exhibiting unexpected behaviors and human-like cognitive/intelligent capabilities, including situation awareness (SA). This shift has the potential to enhance the performance of mixed human-AI teams over all-human teams, underscoring the need for a better understanding of the dynamic SA interactions between humans and machines. To this end, we provide a review of leading SA theoretical models and a new framework for SA in the HAT context based on the key features and processes of HAT. The Agent Teaming Situation Awareness (ATSA) framework unifies human and AI behavior, and involves bidirectional, and dynamic interaction. The framework is based on the individual and team SA models and elaborates on the cognitive mechanisms for modeling HAT. Similar perceptual cycles are adopted for the individual (including both human and AI) and the whole team, which is tailored to the unique requirements of the HAT context. ATSA emphasizes cohesive and effective HAT through structures and components, including teaming understanding, teaming control, and the world, as well as adhesive transactive part. We further propose several future research directions to expand on the distinctive contributions of ATSA and address the specific and pressing next steps.Comment: 52 pages,5 figures, 1 tabl

Human-agent collectives

We live in a world where a host of computer systems, distributed throughout our physical and information environments, are increasingly implicated in our everyday actions. Computer technologies impact all aspects of our lives and our relationship with the digital has fundamentally altered as computers have moved out of the workplace and away from the desktop. Networked computers, tablets, phones and personal devices are now commonplace, as are an increasingly diverse set of digital devices built into the world around us. Data and information is generated at unprecedented speeds and volumes from an increasingly diverse range of sources. It is then combined in unforeseen ways, limited only by human imagination. People’s activities and collaborations are becoming ever more dependent upon and intertwined with this ubiquitous information substrate. As these trends continue apace, it is becoming apparent that many endeavours involve the symbiotic interleaving of humans and computers. Moreover, the emergence of these close-knit partnerships is inducing profound change. Rather than issuing instructions to passive machines that wait until they are asked before doing anything, we will work in tandem with highly inter-connected computational components that act autonomously and intelligently (aka agents). As a consequence, greater attention needs to be given to the balance of control between people and machines. In many situations, humans will be in charge and agents will predominantly act in a supporting role. In other cases, however, the agents will be in control and humans will play the supporting role. We term this emerging class of systems human-agent collectives (HACs) to reflect the close partnership and the flexible social interactions between the humans and the computers. As well as exhibiting increased autonomy, such systems will be inherently open and social. This means the participants will need to continually and flexibly establish and manage a range of social relationships. Thus, depending on the task at hand, different constellations of people, resources, and information will need to come together, operate in a coordinated fashion, and then disband. The openness and presence of many distinct stakeholders means participation will be motivated by a broad range of incentives rather than diktat. This article outlines the key research challenges involved in developing a comprehensive understanding of HACs. To illuminate this agenda, a nascent application in the domain of disaster response is presented

Human-Machine Teaming for UAVs: An Experimentation Platform

Full automation is often not achievable or desirable in critical systems with high-stakes decisions. Instead, human-AI teams can achieve better results. To research, develop, evaluate, and validate algorithms suited for such teaming, lightweight experimentation platforms that enable interactions between humans and multiple AI agents are necessary. However, there are limited examples of such platforms for defense environments. To address this gap, we present the Cogment human-machine teaming experimentation platform, which implements human-machine teaming (HMT) use cases that features heterogeneous multi-agent systems and can involve learning AI agents, static AI agents, and humans. It is built on the Cogment platform and has been used for academic research, including work presented at the ALA workshop at AAMAS this year [1]. With this platform, we hope to facilitate further research on human-machine teaming in critical systems and defense environments.Comment: 9 pages, 6 figures Presented at Conference on Artificial Intelligence for Defense (CAID) 202

Design for Acceptance and Intuitive Interaction: Teaming Autonomous Aerial Systems with Non-experts

In recent years, rapid developments in artificial intelligence (AI) and robotics have enabled transportation systems such as delivery drones to strive for ever-higher levels of autonomy and improve infrastructure in many industries. Consequently, the significance of interaction between autonomous systems and humans with little or no experience is steadily rising. While acceptance of delivery drones remains low among the general public, a solution for intuitive interaction with autonomous drones to retrieve packages is urgently needed so that non-experts can also benefit from the technology. We apply a design science research approach and develop a mobile application as a solution instantiation for both challenges. We conduct one expert and one non-expert design cycle to integrate necessary domain knowledge and ensure acceptance of the artifact by potential non-expert users. The results show that teaming of non-experts with complex autonomous systems requires rethinking common design requirements, such as ensuring transparency of AI-based decisions

Exploring Multimedia Web Conferencing

Internet changed the perspective on meetings and also on decision making processes. Virtualization of meetings has become a common way for collaboration among employees, customers, partners, trainees and trainers, etc. Web conferencing allows the collaboration between teams' members to achieve common goals. Without the need of travelling and meeting organization, the web conferencing applications permit the participation of people from different location. Web conferencing applications are multimedia systems that allow various remote collaborations with multiple types of resources. The paper presents an exploratory study on multimedia web conferencing systems, its advantages and disadvantages and also a use case, meant to highlight several of this technology benefits and problems.multimedia web conferencing, web collaboration, virtual teams, decision support

Analytics for Autonomous C4ISR within e-Government: a Research Agenda

e-Government enables big data analytics to support decision processes in governing. C4ISR (Command, Control, Communications, Computers, Intelligence, Surveillance and Reconnaissance) is essentially e-Government scoped to military decision processes. The value of big data and its challenges are common to both. High variety and demand for veracity compel domain expertise-specific data analysis, and increasing volume and velocity hinder data analytics at scale. These conditions challenge even highly automated methods for comprehensive cross-domain analytics, and motivate cognitive approaches such as underlie Autonomous Systems (AS) aimed at C4ISR. A C4ISR framework is examined by parts, linking each C to ISR capability, and a taxonomy of analytics is extended to include cognitive autonomy enablers. Coupling these frameworks, the authors propose an extension of cognitive approaches for autonomy in C4ISR to e-Government in general and outline a research agenda for attaining it

Exploring a GPT-based large language model for variable autonomy in a VR-based human-robot teaming simulation

In a rapidly evolving digital landscape autonomous tools and robots are becoming commonplace. Recognizing the significance of this development, this paper explores the integration of Large Language Models (LLMs) like Generative pre-trained transformer (GPT) into human-robot teaming environments to facilitate variable autonomy through the means of verbal human-robot communication. In this paper, we introduce a novel simulation framework for such a GPT-powered multi-robot testbed environment, based on a Unity Virtual Reality (VR) setting. This system allows users to interact with simulated robot agents through natural language, each powered by individual GPT cores. By means of OpenAI’s function calling, we bridge the gap between unstructured natural language input and structured robot actions. A user study with 12 participants explores the effectiveness of GPT-4 and, more importantly, user strategies when being given the opportunity to converse in natural language within a simulated multi-robot environment. Our findings suggest that users may have preconceived expectations on how to converse with robots and seldom try to explore the actual language and cognitive capabilities of their simulated robot collaborators. Still, those users who did explore were able to benefit from a much more natural flow of communication and human-like back-and-forth. We provide a set of lessons learned for future research and technical implementations of similar systems