Hierarchic interactive path planning in virtual reality

To save time and money while designing new products, industry needs tools to design, test and validate the product using virtual prototypes. These vir- tual prototypes must enable to test the product at all Product Life-cycle Management (PLM) stages. Many operations in PLM involve human manipulation of product com- ponents in cluttered environment (product assembly, disassembly or maintenance). Virtual Reality (VR) enables real operators to perform these tests with virtual proto- types. This work introduces a novel path planning architecture allowing collaboration between a VR user and an automatic path planning system. It is based on an origi- nal environment model including semantic, topological and geometric information, and an automatic path planning process split in two phases: coarse (semantic and topological information) and fine (semantic and geometric information) planning. The collaboration between VR user and automatic path planner is made of 3 main aspects. First, the VR user is guided along a pre-computed path through a haptic device whereas he VR user can go away from the proposed path to explore possible better ways. Second the authority of automatic planning system is balanced to let the user free to explore alternatives (geometric layer). Third the intents of VR user are predicted (on topological layer) to be integrated in the re-planning process. Exper- iments are provided to illustrate the multi-layer representation of the environment, the path planning process, the control sharing and the intent prediction

A hierarchic approach for path planning in virtual reality.

This work considers path-planning processes for manipu- lation tasks such as assembly, maintenance or disassem- bly in a virtual reality (VR) context. The approach con- sists in providing a collaborative system associating a user immersed in VR and an automatic path planning process. It is based on semantic, topological and geometric representations of the environment and the planning process is split in two phases: coarse and fine planning. The automatic planner suggests a path to the user and guides him trough a haptic device. The user can escape from the proposed solution if he wants to explore a possible better way. In this case, the interactive system detects the users intention and computes in real-time a new path starting from the users guess. Experiments illustrate the different aspects of the approach: multi-representation of the en- vironment, path planning process, users intent prediction and control sharing

A multi-layer approach for interactive path planning control.

This work considers path-planning processes for manipulation tasks such as assembly, maintenance or disassembly in a Virtual Reality (VR) context. The approach consists in providing a collaborative system associating a user immersed in VR and an automatic path planning process. It is based on semantic, topological and geometric representations of the environment and the planning process is split in two phases: coarse and fine planning. The automatic planner suggests a path to the user and guides him trough a haptic device. The user can escape from the proposed solution if he wants to explore a possible better way. In this case, the interactive system detects the user’s intention in real-time and computes a new path starting from the user’s guess. Experiments illustrate the different aspects of the approach: multi-representation of the environment, path planning process, user’s intent prediction and control sharing

Trust-Based Control of (Semi)Autonomous Mobile Robotic Systems

Despite great achievements made in (semi)autonomous robotic systems, human participa-tion is still an essential part, especially for decision-making about the autonomy allocation of robots in complex and uncertain environments. However, human decisions may not be optimal due to limited cognitive capacities and subjective human factors. In human-robot interaction (HRI), trust is a major factor that determines humans use of autonomy. Over/under trust may lead to dispro-portionate autonomy allocation, resulting in decreased task performance and/or increased human workload. In this work, we develop automated decision-making aids utilizing computational trust models to help human operators achieve a more effective and unbiased allocation. Our proposed decision aids resemble the way that humans make an autonomy allocation decision, however, are unbiased and aim to reduce human workload, improve the overall performance, and result in higher acceptance by a human. We consider two types of autonomy control schemes for (semi)autonomous mobile robotic systems. The first type is a two-level control scheme which includes switches between either manual or autonomous control modes. For this type, we propose automated decision aids via a computational trust and self-confidence model. We provide analytical tools to investigate the steady-state effects of the proposed autonomy allocation scheme on robot performance and human workload. We also develop an autonomous decision pattern correction algorithm using a nonlinear model predictive control to help the human gradually adapt to a better allocation pattern. The second type is a mixed-initiative bilateral teleoperation control scheme which requires mixing of autonomous and manual control. For this type, we utilize computational two-way trust models. Here, mixed-initiative is enabled by scaling the manual and autonomous control inputs with a function of computational human-to-robot trust. The haptic force feedback cue sent by the robot is dynamically scaled with a function of computational robot-to-human trust to reduce humans physical workload. Using the proposed control schemes, our human-in-the-loop tests show that the trust-based automated decision aids generally improve the overall robot performance and reduce the operator workload compared to a manual allocation scheme. The proposed decision aids are also generally preferred and trusted by the participants. Finally, the trust-based control schemes are extended to the single-operator-multi-robot applications. A theoretical control framework is developed for these applications and the stability and convergence issues under the switching scheme between different robots are addressed via passivity based measures

General Concepts for Human Supervision of Autonomous Robot Teams

For many dangerous, dirty or dull tasks like in search and rescue missions, deployment of autonomous teams of robots can be beneficial due to several reasons. First, robots can replace humans in the workspace. Second, autonomous robots reduce the workload of a human compared to teleoperated robots, and therefore multiple robots can in principle be supervised by a single human. Third, teams of robots allow distributed operation in time and space. This thesis investigates concepts of how to efficiently enable a human to supervise and support an autonomous robot team, as common concepts for teleoperation of robots do not apply because of the high mental workload. The goal is to find a way in between the two extremes of full autonomy and pure teleoperation, by allowing to adapt the robots’ level of autonomy to the current situation and the needs of the human supervisor. The methods presented in this thesis make use of the complementary strengths of humans and robots, by letting the robots do what they are good at, while the human should support the robots in situations that correspond to the human strengths. To enable this type of collaboration between a human and a robot team, the human needs to have an adequate knowledge about the current state of the robots, the environment, and the mission. For this purpose, the concept of situation overview (SO) has been developed in this thesis, which is composed of the two components robot SO and mission SO. Robot SO includes information about the state and activities of each single robot in the team, while mission SO deals with the progress of the mission and the cooperation between the robots. For obtaining SO a new event-based communication concept is presented in this thesis, that allows the robots to aggregate information into discrete events using methods from complex event processing. The quality and quantity of the events that are actually sent to the supervisor can be adapted during runtime by defining positive and negative policies for (not) sending events that fulfill specific criteria. This reduces the required communication bandwidth compared to sending all available data. Based on SO, the supervisor is enabled to efficiently interact with the robot team. Interactions can be initiated either by the human or by the robots. The developed concept for robot-initiated interactions is based on queries, that allow the robots to transfer decisions to another process or the supervisor. Various modes for answering the queries, ranging from fully autonomous to pure human decisions, allow to adapt the robots’ level of autonomy during runtime. Human-initiated interactions are limited to high-level commands, whereas interactions on the action level (e. g., teleoperation) are avoided, to account for the specific strengths of humans and robots. These commands can in principle be applied to quite general classes of task allocation methods for autonomous robot teams, e. g., in terms of specific restrictions, which are introduced into the system as constraints. In that way, the desired allocations emerge implicitly because of the introduced constraints, and the task allocation method does not need to be aware of the human supervisor in the loop. This method is applicable to different task allocation approaches, e. g., instantaneous or time-extended task assignments, and centralized or distributed algorithms. The presented methods are evaluated by a number of different experiments with physical and simulated scenarios from urban search and rescue as well as robot soccer, and during robot competitions. The results show that with these methods a human supervisor can significantly improve the robot team performance

Mixed initiative control of autonomous vehicles

While there is extensive work on motion planning and control for navigation tasks with guarantees, there is no systematic way for human operators to modify the resulting plans without losing the guarantees. In this paper we propose a systematic way of composing behaviors resulting from human inputs with behaviors derived from navigation functions. The proposed controller is based on a new class of navigation function based controllers that posses weak Input-to-State stability properties. The resulting system has analytically guaranteed safety and convergence properties. The feasibility of the proposed methodology is demonstrated through simulation examples and hardware experiments