Inferring Robot Task Plans from Human Team Meetings: A Generative Modeling Approach with Logic-Based Prior

We aim to reduce the burden of programming and deploying autonomous systems to work in concert with people in time-critical domains, such as military field operations and disaster response. Deployment plans for these operations are frequently negotiated on-the-fly by teams of human planners. A human operator then translates the agreed upon plan into machine instructions for the robots. We present an algorithm that reduces this translation burden by inferring the final plan from a processed form of the human team's planning conversation. Our approach combines probabilistic generative modeling with logical plan validation used to compute a highly structured prior over possible plans. This hybrid approach enables us to overcome the challenge of performing inference over the large solution space with only a small amount of noisy data from the team planning session. We validate the algorithm through human subject experimentation and show we are able to infer a human team's final plan with 83% accuracy on average. We also describe a robot demonstration in which two people plan and execute a first-response collaborative task with a PR2 robot. To the best of our knowledge, this is the first work that integrates a logical planning technique within a generative model to perform plan inference.Comment: Appears in Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence (AAAI-13

Using a test automation tool for robotic process automation: An empirical study

Robotic Process Automation (RPA) uses software robots that interact with systems through their user interface, reducing costs and improving efficiency in automating processes. Despite being a recent term, it is being progressively adopted in companies, being used in many areas, such as IT, Insurance and Human Resources. Although RPA is relatively inexpensive to implement, the cost of licensing is high. To reduce costs, there are open source tools that might be capable of automating process, despite being used for other purposes, such as Test Automation. Therefore, this research aims to test if it is feasible to use a Test Automation tool to automate business processes and identify advantages and disadvantages of using a Test Automation tool as a RPA tool. To accomplish that, a Case Study (CS) was performed in a real company where the same business process was automated, using both a Test Automation tool and a RPA tool. This research presents the comparison analysis and results of an experiment designed with two approaches: a Test Automation tool vs a RPA tool. The results show that despite there were some challenges of using a Test Automation to automate the selected process than with a RPA tool, using Test Automation tools may be useful for companies with low financial resources that aim to find low cost alternatives to RPA tools to automate processes, taking into account the advantages and disadvantages of using a Test Automation tool as a RPA tool provided in this research.Robotic Process Automation (RPA) utiliza software robots que interagem com os sistemas através da sua interface gráfica, reduzindo custos e melhorando a eficiência na automação de processos. Apesar de ser um termo recente, está a ser cada vez mais usado nas empresas, sendo usado em várias áreas, como o IT, seguradoras e recursos humanos. Embora o RPA tenha um custo relativamente baixo de implementação, o custo das licenças é elevado. Para reduzir custos, há ferramentas open source que podem ser capazes de automatizar processos, apesar de serem usadas para outros fins, como Test Automation. Consequentemente, esta investigação pretende testar se é possível usar uma ferramenta de Test Automation para automação de processos e identificar vantagens e desvantagens de usar uma ferramenta de Test Automation como ferramenta de RPA. Esta investigação apresenta a análise da comparação e resultados de uma experiência feita com duas abordagens: uma ferramenta de Test Automation e uma ferramenta de RPA. Os resultados mostram que apesar de terem sido identificados mais desafios ao usar uma ferramenta de Test Automation na automação do processo escolhido do que ao usar uma ferramenta de RPA, o uso de ferramentas de Test Automation pode ser útil em empresas com poucos recursos financeiros que procuram alternativas low cost às ferramentas de RPA para automação de processos, tendo em conta as vantagens e desvantagens do uso de uma ferramenta de Test Automation como ferramenta de RPA descritas nesta investigação

Asynchronous Visualization of Spatiotemporal Information for Multiple Moving Targets

In the modern information age, the quantity and complexity of spatiotemporal data is increasing both rapidly and continuously. Sensor systems with multiple feeds that gather multidimensional spatiotemporal data will result in information clusters and overload, as well as a high cognitive load for users of these systems. To meet future safety-critical situations and enhance time-critical decision-making missions in dynamic environments, and to support the easy and effective managing, browsing, and searching of spatiotemporal data in a dynamic environment, we propose an asynchronous, scalable, and comprehensive spatiotemporal data organization, display, and interaction method that allows operators to navigate through spatiotemporal information rather than through the environments being examined, and to maintain all necessary global and local situation awareness. To empirically prove the viability of our approach, we developed the Event-Lens system, which generates asynchronous prioritized images to provide the operator with a manageable, comprehensive view of the information that is collected by multiple sensors. The user study and interaction mode experiments were designed and conducted. The Event-Lens system was discovered to have a consistent advantage in multiple moving-target marking-task performance measures. It was also found that participants’ attentional control, spatial ability, and action video gaming experience affected their overall performance

Teamwork in controlling multiple robots

Simultaneously controlling increasing numbers of robots requires multiple operators working together as a team. Helping operators allocate attention among different robots and determining how to construct the human-robot team to promote performance and reduce workload are critical questions that must be answered in these settings. To this end, we investigated the effect of team structure and search guidance on operators' performance, subjective workload, work processes and communication. To investigate team structure in an urban search and rescue setting, we compared a pooled condition, in which team members shared control of 24 robots, with a sector condition, in which each team member control half of all the robots. For search guidance, a notification was given when the operator spent too much time on one robot and either suggested or forced the operator to change to another robot. A total of 48 participants completed the experiment with two persons forming one team. The results demonstrate that automated search guidance neither increased nor decreased performance. However, suggested search guidance decreased average task completion time in Sector teams. Search guidance also influenced operators' teleoperation behaviors. For team structure, pooled teams experienced lower subjective workload than sector teams. Pooled teams communicated more than sector teams, but sector teams teleoperated more than pool teams.United States. Office of Naval ResearchUnited States. Air Force Office of Scientific Researc

Human-robot Interaction For Multi-robot Systems

Designing an effective human-robot interaction paradigm is particularly important for complex tasks such as multi-robot manipulation that require the human and robot to work together in a tightly coupled fashion. Although increasing the number of robots can expand the area that the robots can cover within a bounded period of time, a poor human-robot interface will ultimately compromise the performance of the team of robots. However, introducing a human operator to the team of robots, does not automatically improve performance due to the difficulty of teleoperating mobile robots with manipulators. The human operator’s concentration is divided not only among multiple robots but also between controlling each robot’s base and arm. This complexity substantially increases the potential neglect time, since the operator’s inability to effectively attend to each robot during a critical phase of the task leads to a significant degradation in task performance. There are several proven paradigms for increasing the efficacy of human-robot interaction: 1) multimodal interfaces in which the user controls the robots using voice and gesture; 2) configurable interfaces which allow the user to create new commands by demonstrating them; 3) adaptive interfaces which reduce the operator’s workload as necessary through increasing robot autonomy. This dissertation presents an evaluation of the relative benefits of different types of user interfaces for multi-robot systems composed of robots with wheeled bases and three degree of freedom arms. It describes a design for constructing low-cost multi-robot manipulation systems from off the shelf parts. User expertise was measured along three axes (navigation, manipulation, and coordination), and participants who performed above threshold on two out of three dimensions on a calibration task were rated as expert. Our experiments reveal that the relative expertise of the user was the key determinant of the best performing interface paradigm for that user, indicating that good user modiii eling is essential for designing a human-robot interaction system that will be used for an extended period of time. The contributions of the dissertation include: 1) a model for detecting operator distraction from robot motion trajectories; 2) adjustable autonomy paradigms for reducing operator workload; 3) a method for creating coordinated multi-robot behaviors from demonstrations with a single robot; 4) a user modeling approach for identifying expert-novice differences from short teleoperation traces

Assessing the Role of Critical Value Factors (CVFs) on Users’ Resistance of Urban Search and Rescue Robotics

Natural and manmade disasters have brought urban search and rescue (USAR) robots to the technology forefront as a means of providing additional support for search and rescue workers. The loss of life among victims and rescue workers necessitates the need for a wider acceptance of this assistive technology. Disasters, such as hurricane Harvey in 2017, hurricane Sandy in 2012, the 2012 United States tornadoes that devastated 17 states, the 2011 Australian floods, the 2011 Japan and 2010 Haiti earthquakes, the 2010 West Virginia coal mine explosions, the 2009 Typhoon caused mudslides in Taiwan, the 2001 Collapse of the World Trade Center, the 2005 Hurricane Katrina, the 1995 Oklahoma City bombing, and the 1995 Kobe Japan earthquake all benefited from the use of USAR. While there has been a push for use of USAR for disaster, user resistance to such technology is still significantly understudied. This study applied a mixed quantitative and qualitative approach to identify important system characteristics and critical value factors (CVFs) that contribute to team members’ resistance to use such technology. The populations for this study included 2,500 USAR team members from the Houston Professional Fire Fighters Association (HPFFA), and the expected sample size of approximately 250 respondents. The main goal of this quantitative study was to examine system characteristics and CVFs that contribute to USAR team members’ resistance to use such technology. System characteristics and CVFs are associated with USAR. Furthermore, the study utilized multivariate linear regression (MLR) and multivariate analysis of covariance (ANCOVA) to determine if, and to what extent, CVFs and computer self-efficacy (CSE) interact to influence USAR team members’ resistance to use such technology. This quantitative study will test for significant differences on CVF’s, CSE, and resistance to use such technology based on age, gender, prior experience with USAR events, years of USAR experience, and organizational role. The contribution of this study was to reduce USAR team members’ resistance to use such technology in an effort minimize risk to USAR team members while maintaining their lifesaving capability

Interactive Multi-Stage Robotic Positioner for Intra-Operative MRI-Guided Stereotactic Neurosurgery

Magnetic resonance imaging (MRI) demonstrates clear advantages over other imaging modalities in neurosurgery with its ability to delineate critical neurovascular structures and cancerous tissue in high-resolution 3D anatomical roadmaps. However, its application has been limited to interventions performed based on static pre/post-operative imaging, where errors accrue from stereotactic frame setup, image registration, and brain shift. To leverage the powerful intra-operative functions of MRI, e.g., instrument tracking, monitoring of physiological changes and tissue temperature in MRI-guided bilateral stereotactic neurosurgery, a multi-stage robotic positioner is proposed. The system positions cannula/needle instruments using a lightweight (203 g) and compact (Ø97 × 81 mm) skull-mounted structure that fits within most standard imaging head coils. With optimized design in soft robotics, the system operates in two stages: i) manual coarse adjustment performed interactively by the surgeon (workspace of ±30°), ii) automatic fine adjustment with precise (<0.2° orientation error), responsive (1.4 Hz bandwidth), and high-resolution (0.058°) soft robotic positioning. Orientation locking provides sufficient transmission stiffness (4.07 N/mm) for instrument advancement. The system's clinical workflow and accuracy is validated with lab-based (<0.8 mm) and MRI-based testing on skull phantoms (<1.7 mm) and a cadaver subject (<2.2 mm). Custom-made wireless omni-directional tracking markers facilitated robot registration under MRI