An ontology-based approach towards coupling task and path planning for the simulation of manipulation tasks

This work deals with the simulation and the validation of complex manipulation tasks under strong geometric constraints in virtual environments. The targeted applications relate to the industry 4.0 framework; as up-to-date products are more and more integrated and the economic competition increases, industrial companies express the need to validate, from design stage on, not only the static CAD models of their products but also the tasks (e.g., assembly or maintenance) related to their Product Lifecycle Management (PLM). The scientific community looked at this issue from two points of view: - Task planning decomposes a manipulation task to be realized into a sequence of primitive actions (i.e., a task plan) - Path planning computes collision-free trajectories, notably for the manipulated objects. It traditionally uses purely geometric data, which leads to classical limitations (possible high computational processing times, low relevance of the proposed trajectory concerning the task to be performed, or failure); recent works have shown the interest of using higher abstraction level data. Joint task and path planning approaches found in the literature usually perform a classical task planning step, and then check out the feasibility of path planning requests associated with the primitive actions of this task plan. The link between task and path planning has to be improved, notably because of the lack of loopback between the path planning level and the task planning level: - The path planning information used to question the task plan is usually limited to the motion feasibility where richer information such as the relevance or the complexity of the proposed path would be needed; - path planning queries traditionally use purely geometric data and/or “blind” path planning methods (e.g., RRT), and no task-related information is used at the path planning level Our work focuses on using task level information at the path planning level. The path planning algorithm considered is RRT; we chose such a probabilistic algorithm because we consider path planning for the simulation and the validation of complex tasks under strong geometric constraints. We propose an ontology-based approach to use task level information to specify path planning queries for the primitive actions of a task plan. First, we propose an ontology to conceptualize the knowledge about the 3D environment in which the simulated task takes place. The environment where the simulated task takes place is considered as a closed part of 3D Cartesian space cluttered with mobile/fixed obstacles (considered as rigid bodies). It is represented by a digital model relying on a multilayer architecture involving semantic, topologic and geometric data. The originality of the proposed ontology lies in the fact that it conceptualizes heterogeneous knowledge about both the obstacles and the free space models. Second, we exploit this ontology to automatically generate a path planning query associated to each given primitive action of a task plan. Through a reasoning process involving the primitive actions instantiated in the ontology, we are able to infer the start and the goal configurations, as well as task-related geometric constraints. Finally, a multi-level path planner is called to generate the corresponding trajectory. The contributions of this work have been validated by full simulation of several manipulation tasks under strong geometric constraints. The results obtained demonstrate that using task-related information allows better control on the RRT path planning algorithm involved to check the motion feasibility for the primitive actions of a task plan, leading to lower computational time and more relevant trajectories for primitive actions

Planning with Action Abstraction and Plan Decomposition Hierarchies

Useful and suitable action representations, with accompanying planning algorithms are crucial for the task performance of many agent systems, and thus a core issue of research on intelligent agents. An efficient and expressive representation of actions and plans can allow planning systems to retrieve relevant knowledge faster and to access and use suitable actions more effectively [18]. Two general approaches have been pursued in the past; STRIPS-based planners, which construct plans from scratch, based on primitive action descriptions and planners using pre-defined Plan Decompositions Hierarchies, also known as Hierarchical Task Networks. In our research, we integrated both an inheritance hierarchy of actions, using STRIPS-like action descriptions, with a plan decomposition hierarchy, which consists of pre-defined plan schemata. This combination is suitable for a richer action and plan representation, and thus an improved planning algorithm. We implemented and tested this approach for a prototypical example application: the travel planning domain. 1

Semantic based task planning for domestic service robots

Task Planning is developed for an autonomous mobile robot in order to support the robot to accomplish tasks in various degrees of environmental complexity. This environment can be fixed or deterministic (as in a factory), dynamic (as in the human domestic household), or non-deterministic (as in the space exploration). The robot should be provided with a reliable planning system in order to face its major challenge of being certain that its plan to accomplish a task is generated correctly, regardless of the dynamic or uncertain elements of its environment. This thesis is focused on providing the robot task planner with the ability to generate its plans reliably and detect the failures in generating correct plans. Previous approaches for generating plans depended mainly on action effects (explicit effects) that are encoded in the action model. This means that the action effects should cover most of the characteristics of the newly generated world state. However, this extra information can complicate the action model, especially in the real world. In this thesis, a semantic knowledge base is proposed to derive and check implicit information about the effects of actions during plan generation. For example, this approach would inform the robot, that it had entered a bedroom because it has recorded at least one bed and zero ovens. When a robot enters a room, the implicit expectations are derived from a semantic knowledge base about that type of room. These expectations should be verified in order to make sure the robot is in the correct room. The main contributions of this thesis are as follows: The concept of using the Semantic Knowledge Base (SKB) to support the robot task planner under deterministic conditions has been defined. A new model of high-level robot actions has been developed, and this model represents the details of robot action as ontology. This model is thus known as the Semantic Action Model (SAM). An algorithm that integrates SKB and SAMs has also been developed. This algorithm creates the “planning domain” in the Planning Domain Definition Language (PDDL) style. This is used as input to the planner to generate the plan for robot tasks. Then, a general purpose planning algorithm has also been defined, which can support planning under deterministic conditions, and is based on using ontology to represent SKB. ii A further contribution relates to the development of a probabilistic approach to deal with uncertainty in semantic knowledge based task planning. This approach shows how uncertainties in action effects and world states are taken into account by the planning system. This contribution also served to resolve situations of confusion in finding an object relevant to the successful generation of an action during task planning. The accuracy related to this type of planning in navigation scenario, on average, is (90.10%). An additional contribution is using the planning system to respond to unexpected situations which are caused by lack of information. This contribution is formalised as a general approach that models cases of incomplete information as a planning problem. This approach includes a sequence of steps for modelling and generating a plan of actions to collect the necessary information from the knowledge base to support the robot planner in generating its plan. This results in developing a new type of action which is known as a Semantic Action Model for Information Gathering (SAM_IG). These actions have the ability to access the knowledge base to retrieve the necessary information to support the planning system when it is faced with incomplete information. The information gathering approach is also used to gather the necessary information in order to check the implicit expectations of the generated actions. The correct classification related to this type of planning in navigation scenario, on average, is (92.83 %). Another contribution is concerned with solving the problem of missing information, which is using the methods for measuring concept similarity in order to extend the robot world state with new similar objects to the original one in the action model. This results in developing Semantic Realisation and Refreshment Module (SRRM) which has the ability to estimate the similarities between objects and the quality of the alternative plans. The quality of the alternative plans could be similar to the original plan, in average, 92.1%. The results reported in this thesis have been tested and verified in simulation experiments under the Robot Operating System (ROS) middleware. The performance of the planning system has been evaluated by using the planning time and other known metrics. These results show that using semantic knowledge can lead to high performance and reliability in generating robot plans during its operatio