Towards the Verification of Human-Robot Teams

Human-Agent collaboration is increasingly important. Not only do high-profile activities such as NASA missions to Mars intend to employ such teams, but our everyday activities involving interaction with computational devices falls into this category. In many of these scenarios, we are expected to trust that the agents will do what we expect and that the agents and humans will work together as expected. But how can we be sure? In this paper, we bring together previous work on the verification of multi-agent systems with work on the modelling of human-agent teamwork. Specifically, we target human-robot teamwork. This paper provides an outline of the way we are using formal verification techniques in order to analyse such collaborative activities. A particular application is the analysis of human-robot teams intended for use in future space exploration

Believable exploration : investigating human exploration behavior to inform the design of believable agents in video games

University of Technology Sydney. Faculty of Engineering and Information Technology.By nature, human beings are curious about their environment. Arriving in a new place, they observe, recognize and interact with their surroundings. People collect information about the new place, and locate objects in that space that help them to make further decisions. This is a typical scenario of spatial exploration. Spatial exploration is common human behavior, where humans explore unknown environments to acquire information and resources. It is pervasively seen in real-world and virtual environments, from exploring new living/working spaces to charting the oceans or venturing beyond the boundaries of our planet. Just as humans explore ‘real’ environments, they also investigate artificial environments in video games. Computer agents, which perceive surrounding environments with limited visual range, often appear in exploration activities, acting as tools or partners for explorers. Despite the broad range of human activities that employ exploration behavior, this element has been insufficiently investigated and understood. Additionally, even though it is commonly accepted that believable agents benefit people in human-computer interaction systems, the research into creating computer agents with believable exploration behavior has been neglected. To solve these issues, I extract the patterns of human exploration behavior in virtual environments, and explore the methodologies of developing believable agents, which explore spatial environments in human-like ways. In the pursuit of this goal, this thesis makes the following four contributions to the emerging field of believable agent exploration: 1) I employed video games as a testbed to investigate human behavior of spatial exploration. Human players played specialized exploration games, verbalized their behavior during playing and discussed their thoughts in the post-play interview. Behavioral patterns were extracted based on replays of playing, think-aloud data and interview data via thematic analysis. 2) Differences of exploration behavior between human and computer agents were identified through a third-person-observation assessment of believability. 3) A heuristic agent was developed, which mimics human exploration methods reflected via the behavioral patterns. Three heuristics, as components of the heuristic agent, were designed to filter potential options when the agent decides where to explore in each step. 4) An integrated agent was developed by filling the behavior gaps between human and computer agents, where an integrated architecture embedded expectations of human-like exploration from mid-level players. Both the heuristic agent and the integrated agent passed the third-person-observation assessment of believability. Therefore, findings in this thesis contribute to fill the gaps in the fields of understanding human exploration behavior as well as developing believable agent

A Conceptual Model of Exploration Wayfinding: An Integrated Theoretical Framework and Computational Methodology

This thesis is an attempt to integrate contending cognitive approaches to modeling wayfinding behavior. The primary goal is to create a plausible model for exploration tasks within indoor environments. This conceptual model can be extended for practical applications in the design, planning, and Social sciences. Using empirical evidence a cognitive schema is designed that accounts for perceptual and behavioral preferences in pedestrian navigation. Using this created schema, as a guiding framework, the use of network analysis and space syntax act as a computational methods to simulate human exploration wayfinding in unfamiliar indoor environments. The conceptual model provided is then implemented in two ways. First of which is by updating an existing agent-based modeling software directly. The second means of deploying the model is using a spatial interaction model that distributed visual attraction and movement permeability across a graph-representation of building floor plans

Geography: Critical Factors in the Analysis of Complex Systems

Geography is a disciple of discovery and exploration. From earliest human endeavor until today, it remains the key to understanding human interaction with the landscape. A conceptual framework of geographic factors provides a holistic analytical approach to the complex systems experienced by humankind across the globe. Physical, Cultural, Economic, and Political variables combine to create the environment of individuals and nations. A holistic and comprehensive framework of geographical variables is needed for a systematic study of geostrategic issues for the purposes of policy making and strategic planning. Geographic scale, its impact on human action and incorporation into human culture, is pervasive. These factors of geography and their variables must be applicable at many scales of human interaction and experience. The complex system of human geo-strategic interaction demands this. Humans are a product of the natural environment, fundamentally a part of the planet. This basic context energizes the processes flowing within the geographic variables. Human nature, acting in the spatial context, is the engine of human generated change that moves through time and is measured on the landscape. It is possible to model this reality and study the interaction. This interaction is observable and informative. The purpose of this dissertation is to identify geographic variables that inform a systematic approach to the analysis of geostrategic issues. These geographic factors have been drawn from the legacy of geographic thought and imagination. The factors and their corresponding variables operate holistically in cycles of action measured across space and time. By use of basic statistical analysis, lines of enquiry can be identified for the expanded use of Agent-Based Models for the purpose of inferential predictive analysis. The unique contribution of this dissertation is a novel conceptual construct for analysis of complex systems in a geostrategic context. The contribution is four-fold: First the organization of geographic factors into a linked field of key variables. Second the creation of a multi-modal process of these variables through a nested set of operational imperatives. Thirdly the construction of the operational imperatives process cycle which informs the resulting fourth contribution of a predictive path of inquiry and analysis

Towards a framework for architecting heterogeneous teams of humans and robots for space exploration

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 2006.Includes bibliographical references (p. 113-121).Human-robotic systems will play a critical role in space exploration, should NASA embark on missions to the Moon and Mars. A unified framework to optimally leverage the capabilities of humans and robots in space exploration will be an invaluable tool for mission planning. Although there is a growing body of literature on human robotic interactions (HRI), there is not yet a framework that lends itself both to a formal representation of heterogeneous teams of humans and robots, and to an evaluation of such teams across a series of common, task-based metrics. My objective in this thesis is to lay the foundations of a unified framework for architecting human-robotic systems for optimal task performance given a set of metrics. First, I review literature from different fields including HRI and human-computer interaction, and synthesize multiple considerations for architecting heterogeneous teams of humans and robots. I then present methods to systematically and formally capture the characteristics that describe a human-robotic system to provide a basis for evaluating human-robotic systems against a common set of metrics.(cont.) I propose an analytical formulation of common metrics to guide the design and evaluate the performance of human-robot systems, and I then apply the analytical formulation to a case study of a multi-agent human-robot system developed at NASA. Finally, I discuss directions for further research aimed at developing this framework.by Julie Ann Arnold.S.M

Hieros: Hierarchical Imagination on Structured State Space Sequence World Models

One of the biggest challenges to modern deep reinforcement learning (DRL) algorithms is sample efficiency. Many approaches learn a world model in order to train an agent entirely in imagination, eliminating the need for direct environment interaction during training. However, these methods often suffer from either a lack of imagination accuracy, exploration capabilities, or runtime efficiency. We propose Hieros, a hierarchical policy that learns time abstracted world representations and imagines trajectories at multiple time scales in latent space. Hieros uses an S5 layer-based world model, which predicts next world states in parallel during training and iteratively during environment interaction. Due to the special properties of S5 layers, our method can train in parallel and predict next world states iteratively during imagination. This allows for more efficient training than RNN-based world models and more efficient imagination than Transformer-based world models. We show that our approach outperforms the state of the art in terms of mean and median normalized human score on the Atari 100k benchmark, and that our proposed world model is able to predict complex dynamics very accurately. We also show that Hieros displays superior exploration capabilities compared to existing approaches.Comment: Submitted to ICLR 2024, 23 pages, 11 figures, code available at: https://github.com/Snagnar/Hiero

Efficiently Combining Human Demonstrations and Interventions for Safe Training of Autonomous Systems in Real-Time

This paper investigates how to utilize different forms of human interaction to safely train autonomous systems in real-time by learning from both human demonstrations and interventions. We implement two components of the Cycle-of-Learning for Autonomous Systems, which is our framework for combining multiple modalities of human interaction. The current effort employs human demonstrations to teach a desired behavior via imitation learning, then leverages intervention data to correct for undesired behaviors produced by the imitation learner to teach novel tasks to an autonomous agent safely, after only minutes of training. We demonstrate this method in an autonomous perching task using a quadrotor with continuous roll, pitch, yaw, and throttle commands and imagery captured from a downward-facing camera in a high-fidelity simulated environment. Our method improves task completion performance for the same amount of human interaction when compared to learning from demonstrations alone, while also requiring on average 32% less data to achieve that performance. This provides evidence that combining multiple modes of human interaction can increase both the training speed and overall performance of policies for autonomous systems.Comment: 9 pages, 6 figure