An ontology-based approach towards coupling task and path planning for the simulation of manipulation tasks

This work deals with the simulation and the validation of complex manipulation tasks under strong geometric constraints in virtual environments. The targeted applications relate to the industry 4.0 framework; as up-to-date products are more and more integrated and the economic competition increases, industrial companies express the need to validate, from design stage on, not only the static CAD models of their products but also the tasks (e.g., assembly or maintenance) related to their Product Lifecycle Management (PLM). The scientific community looked at this issue from two points of view: - Task planning decomposes a manipulation task to be realized into a sequence of primitive actions (i.e., a task plan) - Path planning computes collision-free trajectories, notably for the manipulated objects. It traditionally uses purely geometric data, which leads to classical limitations (possible high computational processing times, low relevance of the proposed trajectory concerning the task to be performed, or failure); recent works have shown the interest of using higher abstraction level data. Joint task and path planning approaches found in the literature usually perform a classical task planning step, and then check out the feasibility of path planning requests associated with the primitive actions of this task plan. The link between task and path planning has to be improved, notably because of the lack of loopback between the path planning level and the task planning level: - The path planning information used to question the task plan is usually limited to the motion feasibility where richer information such as the relevance or the complexity of the proposed path would be needed; - path planning queries traditionally use purely geometric data and/or “blind” path planning methods (e.g., RRT), and no task-related information is used at the path planning level Our work focuses on using task level information at the path planning level. The path planning algorithm considered is RRT; we chose such a probabilistic algorithm because we consider path planning for the simulation and the validation of complex tasks under strong geometric constraints. We propose an ontology-based approach to use task level information to specify path planning queries for the primitive actions of a task plan. First, we propose an ontology to conceptualize the knowledge about the 3D environment in which the simulated task takes place. The environment where the simulated task takes place is considered as a closed part of 3D Cartesian space cluttered with mobile/fixed obstacles (considered as rigid bodies). It is represented by a digital model relying on a multilayer architecture involving semantic, topologic and geometric data. The originality of the proposed ontology lies in the fact that it conceptualizes heterogeneous knowledge about both the obstacles and the free space models. Second, we exploit this ontology to automatically generate a path planning query associated to each given primitive action of a task plan. Through a reasoning process involving the primitive actions instantiated in the ontology, we are able to infer the start and the goal configurations, as well as task-related geometric constraints. Finally, a multi-level path planner is called to generate the corresponding trajectory. The contributions of this work have been validated by full simulation of several manipulation tasks under strong geometric constraints. The results obtained demonstrate that using task-related information allows better control on the RRT path planning algorithm involved to check the motion feasibility for the primitive actions of a task plan, leading to lower computational time and more relevant trajectories for primitive actions

An ontology-based approach towards coupling task and path planning for the simulation of manipulation tasks

Simulating complex industrial manipulation tasks (e.g., assembly, disassembly and maintenance tasks) under strong geometric constraints in a virtual environment, requires the joint usage of task and path planning, not only to compute a sequence of primitive actions (i.e., a task plan) at task planning level to identify the order to manipulate different objects (e.g., assembly order), but also to generate and validate motions for each of these primitive actions in a virtual environment by computing valid collision-free paths for these actions at path planning level. Although task and path planning have been respectively welly discussed by artificial intelligence and robotic domain, the link between them still remains an open issue, in particular because path planning for a primitive action often uses purely geometric data. This purely geometric path planning suffers from the classical failures (i.e., high-possibility of failure, high processing time and low path relevance) of automated path planning techniques when dealing with complex geometric models. Thus, it can possibly lead to high computational time of the joint task and path planning process and can probably produce a poor implementation of a task plan. Instead of geometric data, involving higher abstraction level information related to a task to be performed in the path planning of a primitive action could lead to a better relevance of simulations. In this work, we propose an ontology-based approach to generate a specific path planning query for a primitive action, using a well-structured task-oriented knowledge model. This specific path planning query aims at obtaining an increased control on the path planning process of the targeted primitive action

Service-oriented agent architecture for autonomous maritime vehicles

Advanced ocean systems are increasing their capabilities and the degree of autonomy more and more in order to perform more sophisticated maritime missions. Remotely operated vehicles are no longer cost-effective since they are limited by economic support costs, and the presence and skills of the human operator. Alternatively, autonomous surface and underwater vehicles have the potential to operate with greatly reduced overhead costs and level of operator intervention. This Thesis proposes an Intelligent Control Architecture (ICA) to enable multiple collaborating marine vehicles to autonomously carry out underwater intervention missions. The ICA is generic in nature but aimed at a case study where a marine surface craft and an underwater vehicle are required to work cooperatively. They are capable of cooperating autonomously towards the execution of complex activities since they have different but complementary capabilities. The architectural foundation to achieve the ICA lays on the flexibility of service-oriented computing and agent technology. An ontological database captures the operator skills, platform capabilities and, changes in the environment. The information captured, stored as knowledge, enables reasoning agents to plan missions based on the current situation. The ICA implementation is verified in simulation, and validated in trials by means of a team of autonomous marine robots. This Thesis also presents architectural details and evaluation scenarios of the ICA, results of simulations and trials from different maritime operations, and future research directions

Practical applications of multi-agent systems in electric power systems

The transformation of energy networks from passive to active systems requires the embedding of intelligence within the network. One suitable approach to integrating distributed intelligent systems is multi-agent systems technology, where components of functionality run as autonomous agents capable of interaction through messaging. This provides loose coupling between components that can benefit the complex systems envisioned for the smart grid. This paper reviews the key milestones of demonstrated agent systems in the power industry and considers which aspects of agent design must still be addressed for widespread application of agent technology to occur

Semantic coupling of path planning and a primitive action of a task plan for the simulation of manipulation tasks in a virtual 3D environment

This work deals with the simulation of complex manipulation tasks in virtual environments. Validating such complex tasks, possibly to be performed under strong geometric constraints, requires considering task and path planning jointly. The contribution of this work focuses on using task-related information at the path planning level. We propose an ontology-based approach to a) model the 3D environment where the simulated task is executed, based on an original multi-level environment model involving higher abstraction level data than the purely geometric models traditionally used, and b) automatically define path planning queries for the primitive ctions of a task plan, together with task-related geometric constraints on these queries. This approach allows the improvement of the state of the art from two points of view. First, our joint task and path planning approach allows the improvement of path planning through better semantic control of the path planning process. Second, if compared to hard-coded geometric constraints, the proposed ontology-based approach introduces a more flexible ay of defining geometric constraints through an inference process, and can be adapted to different applications of manipulation tasks

Internet of robotic things : converging sensing/actuating, hypoconnectivity, artificial intelligence and IoT Platforms

The Internet of Things (IoT) concept is evolving rapidly and influencing newdevelopments in various application domains, such as the Internet of MobileThings (IoMT), Autonomous Internet of Things (A-IoT), Autonomous Systemof Things (ASoT), Internet of Autonomous Things (IoAT), Internetof Things Clouds (IoT-C) and the Internet of Robotic Things (IoRT) etc.that are progressing/advancing by using IoT technology. The IoT influencerepresents new development and deployment challenges in different areassuch as seamless platform integration, context based cognitive network integration,new mobile sensor/actuator network paradigms, things identification(addressing, naming in IoT) and dynamic things discoverability and manyothers. The IoRT represents new convergence challenges and their need to be addressed, in one side the programmability and the communication ofmultiple heterogeneous mobile/autonomous/robotic things for cooperating,their coordination, configuration, exchange of information, security, safetyand protection. Developments in IoT heterogeneous parallel processing/communication and dynamic systems based on parallelism and concurrencyrequire new ideas for integrating the intelligent “devices”, collaborativerobots (COBOTS), into IoT applications. Dynamic maintainability, selfhealing,self-repair of resources, changing resource state, (re-) configurationand context based IoT systems for service implementation and integrationwith IoT network service composition are of paramount importance whennew “cognitive devices” are becoming active participants in IoT applications.This chapter aims to be an overview of the IoRT concept, technologies,architectures and applications and to provide a comprehensive coverage offuture challenges, developments and applications

Cognitive Reasoning for Compliant Robot Manipulation

Physically compliant contact is a major element for many tasks in everyday environments. A universal service robot that is utilized to collect leaves in a park, polish a workpiece, or clean solar panels requires the cognition and manipulation capabilities to facilitate such compliant interaction. Evolution equipped humans with advanced mental abilities to envision physical contact situations and their resulting outcome, dexterous motor skills to perform the actions accordingly, as well as a sense of quality to rate the outcome of the task. In order to achieve human-like performance, a robot must provide the necessary methods to represent, plan, execute, and interpret compliant manipulation tasks. This dissertation covers those four steps of reasoning in the concept of intelligent physical compliance. The contributions advance the capabilities of service robots by combining artificial intelligence reasoning methods and control strategies for compliant manipulation. A classification of manipulation tasks is conducted to identify the central research questions of the addressed topic. Novel representations are derived to describe the properties of physical interaction. Special attention is given to wiping tasks which are predominant in everyday environments. It is investigated how symbolic task descriptions can be translated into meaningful robot commands. A particle distribution model is used to plan goal-oriented wiping actions and predict the quality according to the anticipated result. The planned tool motions are converted into the joint space of the humanoid robot Rollin' Justin to perform the tasks in the real world. In order to execute the motions in a physically compliant fashion, a hierarchical whole-body impedance controller is integrated into the framework. The controller is automatically parameterized with respect to the requirements of the particular task. Haptic feedback is utilized to infer contact and interpret the performance semantically. Finally, the robot is able to compensate for possible disturbances as it plans additional recovery motions while effectively closing the cognitive control loop. Among others, the developed concept is applied in an actual space robotics mission, in which an astronaut aboard the International Space Station (ISS) commands Rollin' Justin to maintain a Martian solar panel farm in a mock-up environment. This application demonstrates the far-reaching impact of the proposed approach and the associated opportunities that emerge with the availability of cognition-enabled service robots