On the Convergence of Techniques that Improve Value Iteration

Prioritisation of Bellman backups or updating only a small subset of actions represent important techniques for speeding up planning in MDPs. The recent literature showed new efficient approaches which exploit these directions. Backward value iteration and backing up only the best actions were shown to lead to a significant reduction of the planning time. This paper conducts a theoretical and empirical analysis of these techniques and shows new important proofs. In particular, (1) it identifies weaker requirements for the convergence of backups based on best actions only, (2) a new method for evaluation of the Bellman error is shown for the update that updates one best action once, (3) it presents the theoretical proof of backward value iteration and establishes required initialisation, (4) and shows that the default state ordering of backups in standard value iteration can significantly influence its performance. Additionally, (5) the existing literature did not compare these methods, either empirically or analytically, against policy iteration. The rigorous empirical and novel theoretical parts of the paper reveal important associations and allow drawing guidelines on which type of value or policy iteration is suitable for a given domain. Finally, our chief message is that standard value iteration can be made far more efficient by simple modifications shown in the paper

High level task planning with inference for the TIAGo robot

The need to combine task planning and motion planning in robotics is well understood. The task planner generates a plan to solve the problem while the motion planner executes the actions of the problem. The previous framework is applied in many state machines that solve complex problems. But in this project we want to present an interface that communicates the task planner layer and the motion planner layer, and updates the geometric information of the environment to inform the task planner. This framework allows to solve complex tasks with basic information of the goal, and replan whenever the motion could not be executed. All the information of the problems is modelled as logical predicates. The objective of this project is to generate a generic model of the environment, with a set of feasible motions of the robot, and use this interface to solve many different planning problems involving those actions, by just giving simple goals. The result is to make the robot more autonomous and allow that any user could use it by giving simple orders. Moreover this project presents the different frameworks and algorithms used to simulate those actions in the robot such as: Sequential Quadratic Programming optimization, Rapidly Random Exploring Tree (RRT) or SBPL global planning. It also shows an introduction to PDDL language used to model the problem and the actions, and the Fast-Froward (FF) solver that is the responsible to translate the problem as a graph and solve it. Finally we test it on different experiments in simulation, by using the TIAGo platform of PAL robotics. The results are promising and allow to dream in service robots solving complex tasks simply computing and modelling basic actions

Use-cases on evolution

This report presents a set of use cases for evolution and reactivity for data in the Web and Semantic Web. This set is organized around three different case study scenarios, each of them is related to one of the three different areas of application within Rewerse. Namely, the scenarios are: “The Rewerse Information System and Portal”, closely related to the work of A3 – Personalised Information Systems; “Organizing Travels”, that may be related to the work of A1 – Events, Time, and Locations; “Updates and evolution in bioinformatics data sources” related to the work of A2 – Towards a Bioinformatics Web