HUMAN CONTROL OF COOPERATING ROBOTS

Advances in robotic technologies and artificial intelligence are allowing robots to emerge fromresearch laboratories into our lives. Experiences with field applications show that we haveunderestimated the importance of human-robot interaction (HRI) and that new problems arise inHRI as robotic technologies expand. This thesis classifies HRI along four dimensions - human,robot, task, and world and illustrates that previous HRI classifications can be successfullyinterpreted as either about one of these elements or about the relationship between two or moreof these elements. Current HRI studies of single-operator single-robot (SOSR) control andsingle-operator multiple-robots (SOMR) control are reviewed using this approach.Human control of multiple robots has been suggested as a way to improve effectiveness inrobot control. Unlike previous studies that investigated human interaction either in low-fidelitysimulations or based on simple tasks, this thesis investigates human interaction with cooperatingrobot teams within a realistically complex environment. USARSim, a high-fidelity game-enginebasedrobot simulator, and MrCS, a distributed multirobot control system, were developed forthis purpose. In the pilot experiment, we studied the impact of autonomy level. Mixed initiativecontrol yielded performance superior to fully autonomous and manual control.To avoid limitation to particular application fields, the present thesis focuses on commonHRI evaluations that enable us to analyze HRI effectiveness and guide HRI design independentlyof the robotic system or application domain. We introduce the interaction episode (IEP), whichwas inspired by our pilot human-multirobot control experiment, to extend the Neglect ToleranceHUMAN CONTROL OF COOPERATING ROBOTSJijun Wang, Ph.D.University of Pittsburgh, 2007vmodel to support general multiple robots control for complex tasks. Cooperation Effort (CE),Cooperation Demand (CD), and Team Attention Demand (TAD) are defined to measure thecooperation in SOMR control. Two validation experiments were conducted to validate the CDmeasurement under tight and weak cooperation conditions in a high-fidelity virtual environment.The results show that CD, as a generic HRI metric, is able to account for the various factors thataffect HRI and can be used in HRI evaluation and analysis

Recent Advances in Multi Robot Systems

To design a team of robots which is able to perform given tasks is a great concern of many members of robotics community. There are many problems left to be solved in order to have the fully functional robot team. Robotics community is trying hard to solve such problems (navigation, task allocation, communication, adaptation, control, ...). This book represents the contributions of the top researchers in this field and will serve as a valuable tool for professionals in this interdisciplinary field. It is focused on the challenging issues of team architectures, vehicle learning and adaptation, heterogeneous group control and cooperation, task selection, dynamic autonomy, mixed initiative, and human and robot team interaction. The book consists of 16 chapters introducing both basic research and advanced developments. Topics covered include kinematics, dynamic analysis, accuracy, optimization design, modelling, simulation and control of multi robot systems

Assessing user specifications of robot behaviour for material transport tasks

Robots are an established component of many existing manufacturing processes. In the majority of cases, these robots operate in segregated areas, separated from human workers. However, as robots' capabilities in sensing and autonomy improve, they are expected to increasingly operate in human environments, and interact with novice, untrained users. When robots operate in human or shared environments, their tasks and behaviours need to be specified. This is typically performed by a human operator or supervisor, who may specify constraints on robot behaviour to make the robot more predictable or align its behaviour with user expectations. However, these constraints may impact robot task performance. This thesis develops a user interface to obtain robot specifications, and proposes metrics for quantifying the specification quality, and to investigate how users create robot specifications. The metrics relate the robot's performance in the specified environment to its performance in a fully-unconstrained environment, and capture the trade-offs that users make in ensuring that the robot accomplishes its tasks while minimizing the loss of performance. The proposed approach is evaluated in a series of user studies. The first user study sought to understand how novice users provide specifications for an autonomous robot operating in a shared warehouse environment, and to validate the metrics by applying them to user-created specifications. The metrics were then modified based on the results of the pilot study, and employed in a second, larger study. The second study trialed a modified interface and interaction scheme that implemented an interactive preference learning system, aimed at modifying specifications to improve robot performance. The modified metrics were then used to assess the quality of specifications following the preference learning system. The two studies show that inexperienced users create a wide variety of behaviour-limiting specifications, and that they generally have difficulty creating efficient specifications, or assessing their own performance. Furthermore, the preference learning process succeeds in improving specification quality by making them more efficient and more similar between different users. Moreover, users that created specifications of worse initial quality benefit the most from the interactive learning process, as those specifications see a larger improvement

The Impact of Automation and Stress on Human Performance in UAV Operation

The United States Air Force (USAF) has increasing needs for unmanned aerial vehicle (UAV) operators. Automation may enable a single operator to manage multiple UAVs at the same time. Multi-UAV operation may require a unique set of skills and the need for new operators calls for targeting new populations for recruitment. The objective of this research is to develop a simulation environment for studying the role of individual differences in UAV operation under different task configurations and investigate predictors of performance and stress. Primarily, the study examined the impact of levels of automation (LOAs), as well as task demands, on task performance, stress and operator reliance on automation. Two intermediate LOAs were employed for two surveillance tasks included in the simulation of UAV operation. Task demand was manipulated via the high and low frequency of events associated with additional tasks included in the simulation. The task demand and LOA manipulations influenced task performance generally as expected. The task demand manipulations elicited higher subjective distress and workload. LOAs did not affect operator workload but affected reliance behavior. Also, this study examined the role of individual differences in simulated UAV operation. A variety of individual difference factors were associated with task performance and with subjective stress response. Video gaming experience was linked to lower distress and better performance, suggesting possible transfer of skills. Some gender differences were revealed in stress response, task performance, but all the gender effects became insignificant with gaming experience controlled. Generally, the effects of personality were consistent with previous studies, except some novel findings with the performance metrics. Additionally, task demand was found to moderate the influence of personality factors on stress response and performance metrics. Specifically, conscientiousness was associated with higher subjective engagement and performance when demands were higher. This study supports future research which aims to improve the dynamic interfaces in UAV operation, optimize operator reliance on automation, and identify individuals with the highest aptitude for multi-UAV control

Route Planning and Operator Allocation in Robot Fleets

In this thesis, we address various challenges related to optimal planning and task allocation in a robot fleet supervised by remote human operators. The overarching goal is to enhance the performance and efficiency of the robot fleets by planning routes and scheduling operator assistance while accounting for limited human availability. The thesis consists of three main problems, each of which focuses on a specific aspect of the system. The first problem pertains to optimal planning for a robot in a collaborative human-robot team, where the human supervisor is intermittently available to assist the robot to complete its tasks faster. Specifically, we address the challenge of computing the fastest route between two configurations in an environment with time constraints on how long the robot can wait for assistance at intermediate configurations. We consider the application of robot navigation in a city environment, where different routes can have distinct speed limits and different time constraints on how long a robot is allowed to wait. Our proposed approach utilizes the concepts of budget and critical departure times, enabling optimal solution and enhanced scalability compared to existing methods. Extensive comparisons with baseline algorithms on a city road network demonstrate its effectiveness and ability to achieve high-quality solutions. Furthermore, we extend the problem to the multi-robot case, where the challenge lies in prioritizing robots when multiple service requests arrive simultaneously. To address this challenge, we present a greedy algorithm that efficiently prioritizes service requests in a batch and has a remarkably good performance compared to the optimal solution. The next problem focuses on allocating human operators to robots in a fleet, considering each robot's specified route and the potential for failures and getting stuck. Conventional techniques used to solve such problems face scalability issues due to exponential growth of state and action spaces with the number of robots and operators. To overcome these, we derive conditions for a technical requirement called indexability, thereby enabling the use of the Whittle index heuristic. Our key insight is to leverage the structure of the value function of individual robots, resulting in conditions that can be easily verified separately for each state of each robot. We apply these conditions to two types of transitions commonly seen in supervised robot fleets. Through numerical simulations, we demonstrate the efficacy of Whittle index policy as a near-optimal scalable approach that outperforms existing scalable methods. Finally, we investigate the impact of interruptions on human supervisors overseeing a fleet of robots. Human supervisors in such systems are primarily responsible for monitoring robots, but can also be assigned with secondary tasks. These tasks can act as interruptions and can be categorized as either intrinsic, i.e., being directly related to the monitoring task, or extrinsic, i.e., being unrelated. Through a user study involving $39$ participants, the findings reveal that task performance remains relatively unaffected by interruptions, and is primarily dependent on the number of robots being monitored. However, extrinsic interruptions led to a significant increase in perceived workload, creating challenges in switching between tasks. These results highlight the importance of managing user workload by limiting extrinsic interruptions in such supervision systems. Overall, this thesis contributes to the field of robot planning and operator allocation in collaborative human-robot teams. By incorporating human assistance, addressing scalability challenges, and understanding the impact of interruptions, we aim to enhance the performance and usability of robot fleets. Our work introduces optimal planning methods and efficient allocation strategies, empowering the seamless operation of robot fleets in real-world scenarios. Additionally, we provide valuable insights into user workload, shedding light on the interactions between humans and robots in such systems. We hope that our research promotes the widespread adoption of robot fleets and facilitates their integration into various domains, ultimately driving advancements in the field

Comparing the Usability of RoboFlag Interface Alternatives*

Abstract- The RoboFlag system was designed as a testbed to study distributed control of multiple vehicle teams with humans in the loop. This work analyzed the RoboFlag version 2.0 interface to identify usability issues. The existing interface was modified to create two new interfaces. The first interface focused on improved usability, while the second focused on improved situation awareness. A user evaluation was conducted to determine if both new interfaces improved the system usability over the original interface. Twenty-four participants completed the evaluation. This paper reports the design considerations, the experimental apparatus, and the usability based statistical analysis. The results indicate that both new interfaces provide improved usability over the RoboFlag version 2.0 interface