Modeling High-Dimensional Humans for Activity Anticipation using Gaussian Process Latent CRFs

Abstract—For robots, the ability to model human configura-tions and temporal dynamics is crucial for the task of anticipating future human activities, yet requires conflicting properties: On one hand, we need a detailed high-dimensional description of human configurations to reason about the physical plausibility of the prediction; on the other hand, we need a compact representation to be able to parsimoniously model the relations between the human and the environment. We therefore propose a new model, GP-LCRF, which admits both the high-dimensional and low-dimensional representation of humans. It assumes that the high-dimensional representation is generated from a latent variable corresponding to its low-dimensional representation using a Gaussian process. The gener-ative process not only defines the mapping function between the high- and low-dimensional spaces, but also models a distribution of humans embedded as a potential function in GP-LCRF along with other potentials to jointly model the rich context among humans, objects and the activity. Through extensive experiments on activity anticipation, we show that our GP-LCRF consistently outperforms the state-of-the-art results and reduces the predicted human trajectory error by 11.6%. I

Placement generation and hybrid planning for robotic rearrangement on cluttered surfaces

Rearranging multiple moving objects across surfaces, e.g. from a table to kitchen shelves as it arises in the context of service robotics, is a challenging problem. The rearrangement problem consists of two subproblems: placement generation and rearrangement planning. Firstly, the collision-free goal poses of the objects to be moved need to be determined subject to the arbitrary geometries of the objects and the state of the surface that already includes movable objects (clutter) and immovable obstacles on it. Secondly, after the goal poses of all objects have been determined, a plan of physical actions must be computed to achieve these goal poses. Computation of such a rearrangement plan is difficult in that it necessitates not only high-level task planning, but also low-level feasibility checks to be integrated with this task plan to ensure that each step of the plan is collision-free. In this thesis, we propose a general solution to the rearrangement of multiple arbitrarily-shaped objects on a cluttered flat surface with multiple movable objects and obstacles. In particular, we introduce a novel method to solve the object placement problem, utilizing nested local searches guided by intelligent heuristics to efficiently perform multi-objective optimizations. The solutions computed by our method satisfy the collision-freeness constraint, and involves minimal movements of the clutter. Based on such a solution, we introduce a hybrid method to generate an optimal feasible rearrangement plan, by integrating ASP-based high-level task planning with low-level feasibility checks. Our hybrid planner is capable of solving challenging non-monotone rearrangement planning instances that cannot be solved by the existing geometric rearrangement approaches. The proposed algorithms have been systematically evaluated in terms of computational efficiency, solution quality, success rate, and scalability. Furthermore, several challenging benchmark instances have been introduced that demonstrate the capabilities of these methods. The real-life applicability of the proposed approaches have also been verified through physical implementation using a Baxter robot

Relationship descriptors for interactive motion adaptation

In this thesis we present an interactive motion adaptation scheme for close interactions between skeletal characters and mesh structures, such as navigating restricted environments and manipulating tools. We propose a new spatial-relationship based representation to encode character-object interactions describing the kinematics of the body parts by the weighted sum of vectors relative to descriptor points selectively sampled over the scene. In contrast to previous discrete representations that either only handle static spatial relationships, or require offline, costly optimization processes, our continuous framework smoothly adapts the motion of a character to deformations in the objects and character morphologies in real-time whilst preserving the original context and style of the scene. We demonstrate the strength of working in our relationship-descriptor space in tackling the issue of motion editing under large environment deformations by integrating procedural animation techniques such as repositioning contacts in an interaction whilst preserving the context and style of the original animation. Furthermore we propose a method that can be used to adapt animations from template objects to novel ones by solving for mappings between the two in our relationship-descriptor space effectively transferring an entire motion from one object to a new one of different geometry whilst ensuring continuity across all frames of the animation, as opposed to mapping static poses only as is traditionally achieved. The experimental results show that our method can be used for a wide range of applications, including motion retargeting for dynamically changing scenes, multi-character interactions, and interactive character control and deformation transfer for scenes that involve close interactions. We further demonstrate a key use case in retargeting locomotion to uneven terrains and curving paths convincingly for bipeds and quadrupeds. Our framework is useful for artists who need to design animated scenes interactively, and modern computer games that allow users to design their own virtual characters, objects and environments, such that they can recycle existing motion data for a large variety of different configurations without the need to manually reconfigure motion from scratch or store expensive combinations of animation in memory. Most importantly it’s achieved in real-time

Designing Playful Systems

Play is a common, yet elusive phenomenon. Many definitions of play and explanations for its existence have been brought forward in various disciplines such as psychology, anthropology, ethology and in the humanities. As an activity apparently serving no other purpose than itself, play can be simply considered a pleasant pastime. Yet its equation with fun has been challenged by artists and scholars alike. Being in a playful state does not warrant extrinsic motivation or being conscious of an external purpose. However, play creates meaning, and scientists are pursuing functional explanations for it. These conflicting observations are contributing to the ambiguity of play and they raise questions about the limits of complexity that present discourses are able to reflect. This thesis presents a comprehensive, transdisciplinary approach to describe and understand play, based on systems-theory, constructivism, cybernetics and practical exploration. Observing play in this way involves theoretical analysis, reflection and critique as well as the practice of design, development and artistic exposition. By constructing, re-contextualising and discussing eight of my own projects, I explore the distinction between theory and practice through which playful systems emerge. Central to my methodology is the concept of distinctions as a fundamental method of observation. It is introduced itself as a distinction and then applied throughout, in order to describe and discuss phenomena of play from a wide range of different perspectives. This includes paradoxical, first-person and conflicting accounts and it enables discourses that cross disciplinary boundaries. In summary, the three interrelated contributions to knowledge in my research project are: I contribute to the emerging field of game studies through a comprehensive systems-theoretical description on play. I also provide a methodology in which theory and practice inform each other through mutual observation, construction, reflection and critical evaluation. Finally, I present eight projects, including a playful system developed in a speculative approach that I call anthroponeutral design. These results represent a novel transdisciplinary perspective on play that offers new opportunities for further research

Hallucinating Humans for Learning Robotic Placement of Objects

Abstract. While a significant body of work has been done on grasping objects, there is little prior work on placing and arranging objects in the environment. In this work, we consider placing multiple objects in complex placing areas, where neither the object nor the placing area may have been seen by the robot before. Specifically, the placements should not only be stable, but should also follow human usage preferences. We present learning and inference algorithms that consider these aspects in placing. In detail, given a set of 3D scenes containing objects, our method, based on Dirichlet process mixture models, samples human poses in each scene and learns how objects relate to those human poses. Then given a new room, our algorithm is able to select meaningful human poses and use them to determine where to place new objects. We evaluate our approach on a variety of scenes in simulation, as well as on robotic experiments.