Semantic information for robot navigation: a survey

There is a growing trend in robotics for implementing behavioural mechanisms based on human psychology, such as the processes associated with thinking. Semantic knowledge has opened new paths in robot navigation, allowing a higher level of abstraction in the representation of information. In contrast with the early years, when navigation relied on geometric navigators that interpreted the environment as a series of accessible areas or later developments that led to the use of graph theory, semantic information has moved robot navigation one step further. This work presents a survey on the concepts, methodologies and techniques that allow including semantic information in robot navigation systems. The techniques involved have to deal with a range of tasks from modelling the environment and building a semantic map, to including methods to learn new concepts and the representation of the knowledge acquired, in many cases through interaction with users. As understanding the environment is essential to achieve high-level navigation, this paper reviews techniques for acquisition of semantic information, paying attention to the two main groups: human-assisted and autonomous techniques. Some state-of-the-art semantic knowledge representations are also studied, including ontologies, cognitive maps and semantic maps. All of this leads to a recent concept, semantic navigation, which integrates the previous topics to generate high-level navigation systems able to deal with real-world complex situationsThe research leading to these results has received funding from HEROITEA: Heterogeneous 480 Intelligent Multi-Robot Team for Assistance of Elderly People (RTI2018-095599-B-C21), funded by Spanish 481 Ministerio de Economía y Competitividad. The research leading to this work was also supported project "Robots sociales para estimulacón física, cognitiva y afectiva de mayores"; funded by the Spanish State Research Agency under grant 2019/00428/001. It is also funded by WASP-AI Sweden; and by Spanish project Robotic-Based Well-Being Monitoring and Coaching for Elderly People during Daily Life Activities (RTI2018-095599-A-C22)

Housekeeping with multiple autonomous robots: representation, reasoning, and execution

We consider a housekeeping domain with static or movable objects, where the goal is for multiple autonomous robots to tidy a house collaboratively in a given amount of time. This domain is challenging in the following ways: commonsense knowledge (e.g., expected locations of objects in the house) is required for intelligent behavior of robots; geometric constraints are required to find feasible plans (e.g., to avoid collisions); in case of plan failure while execution (e.g., due to a collision with movable objects whose presence and location are not known in advance or due to heavy objects that cannot be lifted by a single robot), recovery is required depending on the cause of failure; and collaboration of robots is required to complete some tasks (e.g., carrying heavy objects). We introduce a formal planning, execution and monitoring framework to address the challenges of this domain, by embedding knowledge representation and automated reasoning in each level of decision-making (that consists of discrete task planning, continuous motion planning, and plan execution), in such a way as to tightly integrate these levels. At the high-level, we represent not only actions and change but also commonsense knowledge in a logicbased formalism. Geometric reasoning is lifted to the high-level by embedding motion planning in the domain description. Then a discrete plan is computed for each robot using an automated reasoner. At the mid-level, if a continuous trajectory cannot be computed by a motion planner because the discrete plan is not feasible at the continuous-level, then a different plan is computed by the automated reasoner subject to some (temporal) conditions represented as formulas. At the low-level, if the plan execution fails, then a new continuous trajectory is computed by a motion planner at the mid-level or a new discrete plan is computed using an automated reasoner at the high-level. We illustrate the applicability of this formal framework with a simulation of a housekeeping domain

Semantic based task planning for domestic service robots

Task Planning is developed for an autonomous mobile robot in order to support the robot to accomplish tasks in various degrees of environmental complexity. This environment can be fixed or deterministic (as in a factory), dynamic (as in the human domestic household), or non-deterministic (as in the space exploration). The robot should be provided with a reliable planning system in order to face its major challenge of being certain that its plan to accomplish a task is generated correctly, regardless of the dynamic or uncertain elements of its environment. This thesis is focused on providing the robot task planner with the ability to generate its plans reliably and detect the failures in generating correct plans. Previous approaches for generating plans depended mainly on action effects (explicit effects) that are encoded in the action model. This means that the action effects should cover most of the characteristics of the newly generated world state. However, this extra information can complicate the action model, especially in the real world. In this thesis, a semantic knowledge base is proposed to derive and check implicit information about the effects of actions during plan generation. For example, this approach would inform the robot, that it had entered a bedroom because it has recorded at least one bed and zero ovens. When a robot enters a room, the implicit expectations are derived from a semantic knowledge base about that type of room. These expectations should be verified in order to make sure the robot is in the correct room. The main contributions of this thesis are as follows: The concept of using the Semantic Knowledge Base (SKB) to support the robot task planner under deterministic conditions has been defined. A new model of high-level robot actions has been developed, and this model represents the details of robot action as ontology. This model is thus known as the Semantic Action Model (SAM). An algorithm that integrates SKB and SAMs has also been developed. This algorithm creates the “planning domain” in the Planning Domain Definition Language (PDDL) style. This is used as input to the planner to generate the plan for robot tasks. Then, a general purpose planning algorithm has also been defined, which can support planning under deterministic conditions, and is based on using ontology to represent SKB. ii A further contribution relates to the development of a probabilistic approach to deal with uncertainty in semantic knowledge based task planning. This approach shows how uncertainties in action effects and world states are taken into account by the planning system. This contribution also served to resolve situations of confusion in finding an object relevant to the successful generation of an action during task planning. The accuracy related to this type of planning in navigation scenario, on average, is (90.10%). An additional contribution is using the planning system to respond to unexpected situations which are caused by lack of information. This contribution is formalised as a general approach that models cases of incomplete information as a planning problem. This approach includes a sequence of steps for modelling and generating a plan of actions to collect the necessary information from the knowledge base to support the robot planner in generating its plan. This results in developing a new type of action which is known as a Semantic Action Model for Information Gathering (SAM_IG). These actions have the ability to access the knowledge base to retrieve the necessary information to support the planning system when it is faced with incomplete information. The information gathering approach is also used to gather the necessary information in order to check the implicit expectations of the generated actions. The correct classification related to this type of planning in navigation scenario, on average, is (92.83 %). Another contribution is concerned with solving the problem of missing information, which is using the methods for measuring concept similarity in order to extend the robot world state with new similar objects to the original one in the action model. This results in developing Semantic Realisation and Refreshment Module (SRRM) which has the ability to estimate the similarities between objects and the quality of the alternative plans. The quality of the alternative plans could be similar to the original plan, in average, 92.1%. The results reported in this thesis have been tested and verified in simulation experiments under the Robot Operating System (ROS) middleware. The performance of the planning system has been evaluated by using the planning time and other known metrics. These results show that using semantic knowledge can lead to high performance and reliability in generating robot plans during its operatio