Task planning using physics-based heuristics on manipulation actions

© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.In order to solve mobile manipulation problems, the efficient combination of task and motion planning is usually required. Moreover, the incorporation of physics-based information has recently been taken into account in order to plan the tasks in a more realistic way. In the present paper, a task and motion planning framework is proposed based on a modified version of the Fast-Forward task planner that is guided by physics-based knowledge. The proposal uses manipulation knowledge for reasoning on symbolic literals (both in offline and online modes) taking into account geometric information in order to evaluate the applicability as well as feasibility of actions while evaluating the heuristic cost. It results in an efficient search of the state space and in the obtention of low-cost physically-feasible plans. The proposal has been implemented and is illustrated with a manipulation problem consisting of a mobile robot and some fixed and manipulatable objects.Peer ReviewedPostprint (author's final draft

Physics-based Motion Planning with Temporal Logic Specifications

One of the main foci of robotics is nowadays centered in providing a great degree of autonomy to robots. A fundamental step in this direction is to give them the ability to plan in discrete and continuous spaces to find the required motions to complete a complex task. In this line, some recent approaches describe tasks with Linear Temporal Logic (LTL) and reason on discrete actions to guide sampling-based motion planning, with the aim of finding dynamically-feasible motions that satisfy the temporal-logic task specifications. The present paper proposes an LTL planning approach enhanced with the use of ontologies to describe and reason about the task, on the one hand, and that includes physics-based motion planning to allow the purposeful manipulation of objects, on the other hand. The proposal has been implemented and is illustrated with didactic examples with a mobile robot in simple scenarios where some of the goals are occupied with objects that must be removed in order to fulfill the task.Comment: The 20th World Congress of the International Federation of Automatic Control, 9-14 July 201

PMK : a knowledge processing framework for autonomous robotics perception and manipulation

Autonomous indoor service robots are supposed to accomplish tasks, like serve a cup, which involve manipulation actions. Particularly, for complex manipulation tasks which are subject to geometric constraints, spatial information and a rich semantic knowledge about objects, types, and functionality are required, together with the way in which these objects can be manipulated. In this line, this paper presents an ontological-based reasoning framework called Perception and Manipulation Knowledge (PMK) that includes: (1) the modeling of the environment in a standardized way to provide common vocabularies for information exchange in human-robot or robot-robot collaboration, (2) a sensory module to perceive the objects in the environment and assert the ontological knowledge, (3) an evaluation-based analysis of the situation of the objects in the environment, in order to enhance the planning of manipulation tasks. The paper describes the concepts and the implementation of PMK, and presents an example demonstrating the range of information the framework can provide for autonomous robots.Peer ReviewedPostprint (published version

Towards general spatial intelligence

The goal of General Spatial Intelligence is to present a unified theory to support the various aspects of spatial experience, whether physical or cognitive. We acknowledge the fact that GIScience has to assume a particular worldview, resulting from specific positions regarding metaphysics, ontology, epistemology, mind, language, cognition and representation. Implicit positions regarding these domains may allow solutions to isolated problems but often hamper a more encompassing approach. We argue that explicitly defining a worldview allows the grounding and derivation of multi-modal models, establishing precise problems, allowing falsifiability. We present an example of such a theory founded on process metaphysics, where the ontological elements are called differences. We show that a worldview has implications regarding the nature of space and, in the case of the chosen metaphysical layer, favours a model of space as true spacetime, i.e. four-dimensionality. Finally we illustrate the approach using a scenario from psychology and AI based planning

Combining task and motion planning for mobile manipulators

Aplicat embargament des de la data de defensa fins el dia 31/12/2019Premi Extraordinari de Doctorat, promoció 2018-2019. Àmbit d’Enginyeria IndustrialThis thesis addresses the combination of task and motion planning which deals with different types of robotic manipulation problems. Manipulation problems are referred to as mobile manipulation, collaborative multiple mobile robots tasks, and even higher dimensional tasks (like bi-manual robots or mobile manipulators). Task and motion planning problems needs to obtain a geometrically feasible manipulation plan through symbolic and geometric search space. The combination of task and motion planning levels has emerged as a challenging issue as the failure leads robots to dead-end tasks due to geometric constraints. In addition, task planning is combined with physics-based motion planning and information to cope with manipulation tasks in which interactions between robots and objects are required, or also a low-cost feasible plan in terms of power is looked for. Moreover, combining task and motion planning frameworks is enriched by introducing manipulation knowledge. It facilitates the planning process and aids to provide the way of executing symbolic actions. Combining task and motion planning can be considered under uncertain information and with human-interaction. Uncertainty can be viewed in the initial state of the robot world or the result of symbolic actions. To deal with such issues, contingent-based task and motion planning is proposed using a perception system and human knowledge. Also, robots can ask human for those tasks which are difficult or infeasible for the purpose of collaboration. An implementation framework to combine different types of task and motion planning is presented. All the required modules and tools are also illustrated. As some task planning algorithms are implemented in Prolog or C++ languages and our geometric reasoner is developed in C++, the flow of information between different languages is explained.Aquesta tesis es centra en les eines de planificació combinada a nivell de tasca i a nivell de moviments per abordar diferents problemes de manipulació robòtica. Els problemes considerats són de navegació de robots mòbil enmig de obstacles no fixes, tasques de manipulació cooperativa entre varis robots mòbils, i tasques de manipulació de dimensió més elevada com les portades a terme amb robots bi-braç o manipuladors mòbils. La planificació combinada de tasques i de moviments ha de cercar un pla de manipulació que sigui geomètricament realitzable, a través de d'un espai de cerca simbòlic i geomètric. La combinació dels nivells de planificació de tasca i de moviments ha sorgit com un repte ja que les fallades degudes a les restriccions geomètriques poden portar a tasques sense solució. Addicionalment, la planificació a nivell de tasca es combina amb informació de la física de l'entorn i amb mètodes de planificació basats en la física, per abordar tasques de manipulació en les que la interacció entre el robot i els objectes és necessària, o també si es busca un pla realitzable i amb un baix cost en termes de potència. A més, el marc proposat per al combinació de la planificació a nivell de tasca i a nivell de moviments es millora mitjançant l'ús de coneixement, que facilita el procés de planificació i ajuda a trobar la forma d'executar accions simbòliques. La combinació de nivells de planificació també es pot considerar en casos d'informació incompleta i en la interacció humà-robot. La incertesa es considera en l'estat inicial i en el resultat de les accions simbòliques. Per abordar aquest problema, es proposa la planificació basada en contingències usant un sistema de percepció i el coneixement de l'operari humà. Igualment, els robots poden demanar col·laboració a l'operari humà per a que realitzi aquelles accions que són difícils o no realitzables pel robot. Es presenta també un marc d'implementació per a la combinació de nivells de planificació usant diferents mètodes, incloent tots els mòduls i eines necessàries. Com que alguns algorismes estan implementats en Prolog i d'altres en C++, i el mòdul de raonament geomètric proposat està desenvolupat en C++, es detalla el flux d'informació entre diferents llenguatges.Award-winningPostprint (published version

A novel framework to improve motion planning of robotic systems through semantic knowledge-based reasoning

The need to improve motion planning techniques for manipulator robots, and new effective strategies to manipulate different objects to perform more complex tasks, is crucial for various real-world applications where robots cooperate with humans. This paper proposes a novel framework that aims to improve the motion planning of a robotic agent (a manipulator robot) through semantic knowledge-based reasoning. The Semantic Web Rule Language (SWRL) was used to infer new knowledge based on the known environment and the robotic system. Ontological knowledge, e.g., semantic maps, were generated through a deep neural network, trained to detect and classify objects in the environment where the robotic agent performs. Manipulation constraints were deduced, and the environment corresponding to the agent’s manipulation workspace was created so the planner could interpret it to generate a collision-free path. For reasoning with the ontology, different SPARQL queries were used. The proposed framework was implemented and validated in a real experimental setup, using the planning framework ROSPlan to perform the planning tasks. The proposed framework proved to be a promising strategy to improve motion planning of robotics systems, showing the benefits of artificial intelligence, for knowledge representation and reasoning in robotics.info:eu-repo/semantics/publishedVersio

Ontological framework to improve motion planning of manipulative agents through semantic knowledge-based reasoning

This paper describes the actions taken in developing a framework that aims to improve the motion planning of a manipulative robotic agent through reasoning based on semantic knowledge. The Semantic Web Rule Language (SWRL) was employed to draw new insights from the existing information about the robotic system and its environment. Recent ontology-based standards have been developed (IEEE 1872-2015; IEEE 1872.2-2021; IEEE 7007-2021), and others are currently under development (IEEE P1872.1; IEEE P1872.3) to improve robot performance in task execution. Ontological knowledge “semantic map" was generated using a deep neural network trained to detect and classify objects in the environment where the manipulator agent acts. Manipulation constraints were deduced, and the environment corresponding to the agent’s manipulation workspace was created so the planner could interpret it to generate a collision-free path. Several SPARQL queries were used to explore the semantic map and allow ontological reasoning. The proposed framework was implemented and validated in a real experimental setting, using the ROSPlan planning framework to perform the planning tasks. This ontology-based framework proved to be a promising strategy. E.g., it allows the robotic manipulative agent to interact with objects, e.g., to choose a mobile phone or a water bottle, using semantic information from the environment to solve the requested tasks.This work is financed by national funds through FCT - Foundation for Science and Technology, I.P., through IDMEC, under LAETA, project UIDB/50022/2020. The work of Rodrigo Bernardo was supported by the PhD Scholarship BD/6841/2020 from FCT. This work has received funding from: the project 0770_EUROAGE2_4_E (POCTEP Programa Interreg V-A Spain-Portugal), and the European Union’s Horizon 2020 programme under StandICT.eu 2023 (under Grant Agreement No.: 951972).info:eu-repo/semantics/publishedVersio