StructDiffusion: Object-Centric Diffusion for Semantic Rearrangement of Novel Objects

Robots operating in human environments must be able to rearrange objects into semantically-meaningful configurations, even if these objects are previously unseen. In this work, we focus on the problem of building physically-valid structures without step-by-step instructions. We propose StructDiffusion, which combines a diffusion model and an object-centric transformer to construct structures out of a single RGB-D image based on high-level language goals, such as "set the table." Our method shows how diffusion models can be used for complex multi-step 3D planning tasks. StructDiffusion improves success rate on assembling physically-valid structures out of unseen objects by on average 16% over an existing multi-modal transformer model, while allowing us to use one multi-task model to produce a wider range of different structures. We show experiments on held-out objects in both simulation and on real-world rearrangement tasks. For videos and additional results, check out our website: http://weiyuliu.com/StructDiffusion/

Tactile Convolutional Networks for Online Slip and Rotation Detection

Meier M, Patzelt F, Haschke R, Ritter H. Tactile Convolutional Networks for Online Slip and Rotation Detection. In: Proceedings of the 25th International Conference on Artificial Neural Networks (ICANN). 2016

Social learning and creativity in children in informal learning environments

Social learning, together with innovation, form the pillars of human culture. The vast majority of research regarding innovation and social learning uses artificially created tasks (e. g. puzzle boxes) with clear goals. Studies with children are performed in nurseries, psychology laboratories and in separate rooms in science centres. This enables studies to have high experimental/internal validity. However, it is not known whether these findings also explain behaviours outside of controlled environments. In this thesis I explored how social learning and creativity could be studied in the context of an informal learning environment (Life Science Centre, Newcastle) using an open-ended task, representing a context of increased ecological validity. In Chapter 3 I explored how direct instructions, scaffolding (open questions) and no instructions impacted children’s exploratory behaviour and their creativity when building with shapeshifting wooden blocks that constituted an existing exhibit in the science centre. In Chapter 4 I used an exhibit, the Interactive Research Pod (IRP), which was developed through cooperation between Durham University academics and Life Science Centre practitioners to study social learning and creativity “in the wild” whereby no experimenter is present and instructions, cameras and ethical assent is automated. I studied children, who were using building blocks, in social (transparent partitions), asocial (opaque partitions, building at or around the same time) and asocial control (opaque partitions, different day). In Chapter 5 I used the IRP to enable children to freely interact whilst building and investigated social learning and cooperation as well as the originality of the final structures. In all three studies I used a newly developed web application to evaluate the creativity of the structures children built. I recruited adult raters to acquire a relatively objective measure of the subjective value of the originality of the wooden structures. I used Bayesian statistical methods to analyse the data. Overall children built diverse structures and were not strongly impacted by the conditions (direct instruction, scaffolding, no instruction, social, asocial learning and cooperation) they were in. The findings of the thesis complement existing data regarding social learning and creativity in children in more controlled environments and demonstrate the utility of conducting such studies in ecologically valid contexts, despite the inherent issues of internal validity