Fluid coordination of human-robot teams

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 2011.Cataloged from PDF version of thesis.Includes bibliographical references (p. 235-239).I envision a future where collaboration between humans and robots will be indispensable to our work in numerous domains, ranging from surgery to space exploration. The success of these systems will depend in part on the ability of robots to integrate within existing human teams. The goal of this thesis is to develop robot partners that we can work with easily and naturally, inspired by the way we work with other people. My hypothesis is that human-robot team performance improves when a robot teammate emulates the effective coordination behaviors observed in human teams. I design and evaluate Chaski, a robot plan execution system that uses insights from human-human teaming to make human-robot teaming more natural and fluid. Chaski is a task-level executive that enables a robot to robustly anticipate and adapt to other team members. Chaski also emulates a human's response to implicit communications, including verbal and gestural cues, and explicit commands. Development of such an executive is challenging because the robot must be able to make decisions very quickly in response to a human's actions. In the past, the ability of robots to demonstrate these capabilities has been limited by the time-consuming computations required to anticipate a large set of possible futures. These computations result in execution delays that endanger the robot's ability to fulfill its role on the team. I significantly improve the ability of a robot to adapt on-the-fly by generalizing the state-of-the-art in dynamic plan execution to support just-in-time task assignment and scheduling. My methods provide a novel way to represent the robot's plan compactly. This compact representation enables the plan to be incrementally updated very quickly. I empirically demonstrate that, compared to prior work in this area, my methods increase the speed of online computation by one order of magnitude on average. I also show that 89% of moderately-sized benchmark plans are updated within human reaction time using Chaski, compared to 24% for prior art. I evaluate Chaski in human subject experiments in which a person works with a mobile and dexterous robot to collaboratively assemble structures using building blocks. I measure team performances outcomes for robots controlled by Chaski compared to robots that are verbally commanded, step-by-step by the human teammate. I show that Chaski reduces the human's idle time by 85%, a statistically significant difference. This result supports the hypothesis that human-robot team performance is improved when a robot emulates the effective coordination behaviors observed in human teams.by Julie A. Shah.Ph.D

10451 Abstracts Collection -- Runtime Verification, Diagnosis, Planning and Control for Autonomous Systems

From November 7 to 12, 2010, the Dagstuhl Seminar 10451 ``Runtime Verification, Diagnosis, Planning and Control for Autonomous Systems\u27\u27 was held in Schloss Dagstuhl~--~Leibniz Center for Informatics. During the seminar, 35 participants presented their current research and discussed ongoing work and open problems. This document puts together abstracts of the presentations given during the seminar, and provides links to extended abstracts or full papers, if available

Flexible schedule optimization for human-robot collaboration

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 99-101).Robots are increasingly entering domains typically thought of as human-only. This convergence of human and robotic agents leads to a need for new technology to enable safe and efficient collaboration. The goal of this thesis is to develop a task allocation and scheduling algorithm for teams of robots working with or around teams of humans in intense domains where tight, fluid choreography of robotic schedules is required to guarantee the safety of all involved while maintaining high levels of productivity. Three algorithms are presented in this work: the Adaptive Preferences Algorithm, the Multi-Agent Optimization Algorithm, and Tercio. Tercio, the culminatory algorithm, is capable of assigning robots to tasks and producing near-optimal schedules for ten agents and hundreds of tasks in seconds while making guarantees about process specifications such as worker safety and deadline satisfaction. This work extends dynamic scheduling methods to incorporate flexible windows with an optimization framework featuring a mixed integer program and a satisficing hueristic scheduler. By making use of Tercio, a manufacturing facility or other high-intensity domain may fluidly command a team of robots to complete tasks in a quick, efficient manner while maintaining an ability to respond seamlessly to disturbances at execution. This greatly increases both productivity, by decreasing the time spent recompiling solutions, and responsiveness to humans in the area. These improvements in performance are displayed with multiple live demonstrations and simulations of teams of robots responding to disturbances. Tercio acts as an enabling step towards the ultimate goal of fully coordinated factories of dozens to hundreds of robots accomplishing many thousands of tasks in a safe, predictable, efficient manner.by Ronald J. Wilcox.S.M

Human–agent team dynamics: a review and future research opportunities

Humans teaming with intelligent autonomous agents is becoming indispensable in work environments. However, human–agent teams pose significant challenges, as team dynamics are complex arising from the task and social aspects of human–agent interactions. To improve our understanding of human–agent team dynamics, in this article, we conduct a systematic literature review. Drawing on Mathieu et al.’s (2019) teamwork model developed for all-human teams, we map the landscape of research to human–agent team dynamics, including structural features, compositional features, mediating mechanisms, and the interplay of the above features and mechanisms. We reveal that the development of human–agent team dynamics is still nascent, with a particular focus on information sharing, trust development, agents’ human likeness behaviors, shared cognitions, situation awareness, and function allocation. Gaps remain in many areas of team dynamics, such as team processes, adaptability, shared leadership, and team diversity. We offer various interdisciplinary pathways to advance research on human–agent teams

Autonomous, Context-Sensitive, Task Management Systems and Decision Support Tools I: Human-Autonomy Teaming Fundamentals and State of the Art

Recent advances in artificial intelligence, machine learning, data mining and extraction, and especially in sensor technology have resulted in the availability of a vast amount of digital data and information and the development of advanced automated reasoners. This creates the opportunity for the development of a robust dynamic task manager and decision support tool that is context sensitive and integrates information from a wide array of on-board and off aircraft sourcesa tool that monitors systems and the overall flight situation, anticipates information needs, prioritizes tasks appropriately, keeps pilots well informed, and is nimble and able to adapt to changing circumstances. This is the first of two companion reports exploring issues associated with autonomous, context-sensitive, task management and decision support tools. In the first report, we explore fundamental issues associated with the development of an integrated, dynamic, flight information and automation management system. We discuss human factors issues pertaining to information automation and review the current state of the art of pilot information management and decision support tools. We also explore how effective human-human team behavior and expectations could be extended to teams involving humans and automation or autonomous systems

Cognitive Decay And Memory Recall During Long Duration Spaceflight

This dissertation aims to advance the efficacy of Long-Duration Space Flight (LDSF) pre-flight and in-flight training programs, acknowledging existing knowledge gaps in NASA\u27s methodologies. The research\u27s objective is to optimize the cognitive workload of LDSF crew members, enhance their neurocognitive functionality, and provide more meaningful work experiences, particularly for Mars missions.The study addresses identified shortcomings in current training and learning strategies and simulation-based training systems, focusing on areas requiring quantitative measures for astronaut proficiency and training effectiveness assessment. The project centers on understanding cognitive decay and memory loss under LDSF-related stressors, seeking to establish when such cognitive decline exceeds acceptable performance levels throughout mission phases. The research acknowledges the limitations of creating a near-orbit environment due to resource constraints and the need to develop engaging tasks for test subjects. Nevertheless, it underscores the potential impact on future space mission training and other high-risk professions. The study further explores astronaut training complexities, the challenges encountered in LDSF missions, and the cognitive processes involved in such demanding environments. The research employs various cognitive and memory testing events, integrating neuroimaging techniques to understand cognition\u27s neural mechanisms and memory. It also explores Rasmussen\u27s S-R-K behaviors and Brain Network Theory’s (BNT) potential for measuring forgetting, cognition, and predicting training needs. The multidisciplinary approach of the study reinforces the importance of integrating insights from cognitive psychology, behavior analysis, and brain connectivity research. Research experiments were conducted at the University of North Dakota\u27s Integrated Lunar Mars Analog Habitat (ILMAH), gathering data from selected subjects via cognitive neuroscience tools and Electroencephalography (EEG) recordings to evaluate neurocognitive performance. The data analysis aimed to assess brain network activations during mentally demanding activities and compare EEG power spectra across various frequencies, latencies, and scalp locations. Despite facing certain challenges, including inadequacies of the current adapter boards leading to analysis failure, the study provides crucial lessons for future research endeavors. It highlights the need for swift adaptation, continual process refinement, and innovative solutions, like the redesign of adapter boards for high radio frequency noise environments, for the collection of high-quality EEG data. In conclusion, while the research did not reveal statistically significant differences between the experimental and control groups, it furnished valuable insights and underscored the need to optimize astronaut performance, well-being, and mission success. The study contributes to the ongoing evolution of training methodologies, with implications for future space exploration endeavors

Decisional issues during human-robot joint action

In the future, robots will become our companions and co-workers. They will gradually appear in our environment, to help elderly or disabled people or to perform repetitive or unsafe tasks. However, we are still far from a real autonomous robot, which would be able to act in a natural, efficient and secure manner with humans. To endow robots with the capacity to act naturally with human, it is important to study, first, how humans act together. Consequently, this manuscript starts with a state of the art on joint action in psychology and philosophy before presenting the implementation of the principles gained from this study to human-robot joint action. We will then describe the supervision module for human-robot interaction developed during the thesis. Part of the work presented in this manuscript concerns the management of what we call a shared plan. Here, a shared plan is a a partially ordered set of actions to be performed by humans and/or the robot for the purpose of achieving a given goal. First, we present how the robot estimates the beliefs of its humans partners concerning the shared plan (called mental states) and how it takes these mental states into account during shared plan execution. It allows it to be able to communicate in a clever way about the potential divergent beliefs between the robot and the humans knowledge. Second, we present the abstraction of the shared plans and the postponing of some decisions. Indeed, in previous works, the robot took all decisions at planning time (who should perform which action, which object to use…) which could be perceived as unnatural by the human during execution as it imposes a solution preferentially to any other. This work allows us to endow the robot with the capacity to identify which decisions can be postponed to execution time and to take the right decision according to the human behavior in order to get a fluent and natural robot behavior. The complete system of shared plans management has been evaluated in simulation and with real robots in the context of a user study. Thereafter, we present our work concerning the non-verbal communication needed for human-robot joint action. This work is here focused on how to manage the robot head, which allows to transmit information concerning what the robot's activity and what it understands of the human actions, as well as coordination signals. Finally, we present how to mix planning and learning in order to allow the robot to be more efficient during its decision process. The idea, inspired from neuroscience studies, is to limit the use of planning (which is adapted to the human-aware context but costly) by letting the learning module made the choices when the robot is in a "known" situation. The first obtained results demonstrate the potential interest of the proposed solution