Learning object relationships which determine the outcome of actions

Peer reviewedPublisher PD

Zero-shot sim-to-real transfer of tactile control policies for aggressive swing-up manipulation

This paper aims to show that robots equipped with a vision-based tactile sensor can perform dynamic manipulation tasks without prior knowledge of all the physical attributes of the objects to be manipulated. For this purpose, a robotic system is presented that is able to swing up poles of different masses, radii and lengths, to an angle of 180 degrees, while relying solely on the feedback provided by the tactile sensor. This is achieved by developing a novel simulator that accurately models the interaction of a pole with the soft sensor. A feedback policy that is conditioned on a sensory observation history, and which has no prior knowledge of the physical features of the pole, is then learned in the aforementioned simulation. When evaluated on the physical system, the policy is able to swing up a wide range of poles that differ significantly in their physical attributes without further adaptation. To the authors' knowledge, this is the first work where a feedback policy from high-dimensional tactile observations is used to control the swing-up manipulation of poles in closed-loop.Comment: Accompanying video: https://youtu.be/4rG-o2Cz3-

Model of tactile sensors using soft contacts and its application in robot grasping simulation

In the context of robot grasping and manipulation, realistic dynamic simulation requires accurate modeling of contacts between bodies and, in a practical level, accurate simulation of touch sensors. This paper addresses the problem of creating a simulation of a tactile sensor as well as its implementation in a simulation environment. The simulated tactile sensor model utilizes collision detection and response methods using soft contacts as well as a full friction description. The tactile element is created based on a geometry enabling the creation of a variety of different shape tactile sensors. The tactile sensor element can be used to detect touch against triangularized geometries. This independence in shape enables the use of the sensor model for various applications, ranging from regular tactile sensors to more complex geometries as the human hand which makes it possible to explore human-like touch. The developed tactile sensor model is implemented within OpenGRASP and is available in the open-source plugin. The model has been validated through several experiments ranging from physical properties verification to testing on robot grasping applications. This simulated sensor can provide researchers with a valuable tool for robotic grasping research, especially in cases where the real sensors are not accurate enough yet

Hierarchical reinforcement learning for adaptive and autonomous decision-making in robotics

In recent years, Reinforcement Learning has been able to solve extremely complex games in simulation, but with limited success in deployment to real-world scenarios. The goal of this work is create an ecosystem in which Reinforcement Learning algorithms can be deployed onto real robots in complex games. The ecosystem begins with the creation of a development pipeline which can be used to progressively train Reinforcement Learning Algorithms in increasingly realistic scenarios, culminating with the deployment of these algorithm onto a real system. The pipeline is paired with the novel Reinforcement Learning algorithms that are better able to adapt to new scenarios than traditional methods for autonomy and robotic planning.We implement two techniques to enable this adaptation. First, we implement a hierarchical Reinforcement Learning architecture that uses differentiated sub-policies governed by a hierarchical controller to enable fast adaptation. Second we introduce a confidence-based training process for the hierarchical controller which improves training stability and convergence times. These algorithmic contributions were evaluated using our development pipeline

Bootstrapping Relational Affordances of Object Pairs using Transfer

This work was supported in part by the U.K. EPSRC DTG EP/J5000343/1 at Aberdeen, and in part by the EU Cognitive Systems Project XPERIENCE at SDU under Grant FP7-ICT-270273.Peer reviewedPostprin

Network traffic characterisation, analysis, modelling and simulation for networked virtual environments

Networked virtual environment (NVE) refers to a distributed software system where a simulation, also known as virtual world, is shared over a data network between several users that can interact with each other and the simulation in real-time. NVE systems are omnipresent in the present globally interconnected world, from entertainment industry, where they are one of the foundations for many video games, to pervasive games that focus on e-learning, e-training or social studies. From this relevance derives the interest in better understanding the nature and internal dynamics of the network tra c that vertebrates these systems, useful in elds such as network infrastructure optimisation or the study of Quality of Service and Quality of Experience related to NVE-based services. The goal of the present work is to deepen into this understanding of NVE network tra c by helping to build network tra c models that accurately describe it and can be used as foundations for tools to assist in some of the research elds enumerated before. First contribution of the present work is a formal characterisation for NVE systems, which provides a tool to determine which systems can be considered as NVE. Based on this characterisation it has been possible to identify numerous systems, such as several video games, that qualify as NVE and have an important associated literature focused on network tra c analysis. The next contribution has been the study of this existing literature from a NVE perspective and the proposal of an analysis pipeline, a structured collection of processes and techniques to de ne microscale network models for NVE tra c. This analysis pipeline has been tested and validated against a study case focused on Open Wonderland (OWL), a framework to build NVE systems of di erent purpose. The analysis pipeline helped to de ned network models from experimental OWL tra c and assessed on their accuracy from a statistical perspective. The last contribution has been the design and implementation of simulation tools based on the above OWL models and the network simulation framework ns-3. The purpose of these simulations was to con rm the validity of the OWL models and the analysis pipeline, as well as providing potential tools to support studies related to NVE network tra c. As a result of this nal contribution, it has been proposed to exploit the parallelisation potential of these simulations through High Throughput Computing techniques and tools, aimed to coordinate massively parallel computing workloads over distributed resources