Modeling Spatial Relations of Human Body Parts for Indexing and Retrieving Close Character Interactions

Retrieving pre-captured human motion for analyzing and synthesizing virtual character movement have been widely used in Virtual Reality (VR) and interactive computer graphics applications. In this paper, we propose a new human pose representation, called Spatial Relations of Human Body Parts (SRBP), to represent spatial relations between body parts of the subject(s), which intuitively describes how much the body parts are interacting with each other. Since SRBP is computed from the local structure (i.e. multiple body parts in proximity) of the pose instead of the information from individual or pairwise joints as in previous approaches, the new representation is robust to minor variations of individual joint location. Experimental results show that SRBP outperforms the existing skeleton-based motion retrieval and classification approaches on benchmark databases

Digital Image Access & Retrieval

The 33th Annual Clinic on Library Applications of Data Processing, held at the University of Illinois at Urbana-Champaign in March of 1996, addressed the theme of "Digital Image Access & Retrieval." The papers from this conference cover a wide range of topics concerning digital imaging technology for visual resource collections. Papers covered three general areas: (1) systems, planning, and implementation; (2) automatic and semi-automatic indexing; and (3) preservation with the bulk of the conference focusing on indexing and retrieval.published or submitted for publicatio

Topology-based character motion synthesis

This thesis tackles the problem of automatically synthesizing motions of close-character interactions which appear in animations of wrestling and dancing. Designing such motions is a daunting task even for experienced animators as the close contacts between the characters can easily result in collisions or penetrations of the body segments. The main problem lies in the conventional representation of the character states that is based on the joint angles or the joint positions. As the relationships between the body segments are not encoded in such a representation, the path-planning for valid motions to switch from one posture to another requires intense random sampling and collision detection in the state-space. In order to tackle this problem, we consider to represent the status of the characters using the spatial relationship of the characters. Describing the scene using the spatial relationships can ease users and animators to analyze the scene and synthesize close interactions of characters. We first propose a method to encode the relationship of the body segments by using the Gauss Linking Integral (GLI), which is a value that specifies how much the body segments are winded around each other. We present how it can be applied for content-based retrieval of motion data of close interactions, and also for synthesis of close character interactions. Next, we propose a representation called Interaction Mesh, which is a volumetric mesh composed of points located at the joint position of the characters and vertices of the environment. This raw representation is more general compared to the tangle-based representation as it can describe interactions that do not involve any tangling nor contacts. We describe how it can be applied for motion editing and retargeting of close character interaction while avoiding penetration and pass-throughs of the body segments. The application of our research is not limited to computer animation but also to robotics, where making robots conduct complex tasks such as tangling, wrapping, holding and knotting are essential to let them assist humans for the daily life

A Two-Stream Recurrent Network for Skeleton-based Human Interaction Recognition

This paper addresses the problem of recognizing human-human interaction from skeletal sequences. Existing methods are mainly designed to classify single human action. Many of them simply stack the movement features of two characters to deal with human interaction, while neglecting the abundant relationships between characters. In this paper, we propose a novel two-stream recurrent neural network by adopting the geometric features from both single actions and interactions to describe the spatial correlations with different discriminative abilities. The first stream is constructed under pairwise joint distance (PJD) in a fully-connected mesh to categorize the interactions with explicit distance patterns. To better distinguish similar interactions, in the second stream, we combine PJD with the spatial features from individual joint positions using graph convolutions to detect the implicit correlations among joints, where the joint connections in the graph are adaptive for flexible correlations. After spatial modeling, each stream is fed to a bi-directional LSTM to encode two-way temporal properties. To take advantage of the diverse discriminative power of the two streams, we come up with a late fusion algorithm to combine their output predictions concerning information entropy. Experimental results show that the proposed framework achieves state-of-the art performance on 3D and comparable performance on 2D interaction datasets. Moreover, the late fusion results demonstrate the effectiveness of improving the recognition accuracy compared with single streams

Interaction-based Human Activity Comparison

Traditional methods for motion comparison consider features from individual characters. However, the semantic meaning of many human activities is usually defined by the interaction between them, such as a high-five interaction of two characters. There is little success in adapting interaction-based features in activity comparison, as they either do not have a fixed topology or are in high dimensional. In this paper, we propose a unified framework for activity comparison from the interaction point of view. Our new metric evaluates the similarity of interaction by adapting the Earth Mover’s Distance onto a customized geometric mesh structure that represents spatial-temporal interactions. This allows us to compare different classes of interactions and discover their intrinsic semantic similarity. We created five interaction databases of different natures, covering both two characters (synthetic and real-people) and character-object interactions, which are open for public uses. We demonstrate how the proposed metric aligns well with the semantic meaning of the interaction. We also apply the metric in interaction retrieval and show how it outperforms existing ones. The proposed method can be used for unsupervised activity detection in monitoring systems and activity retrieval in smart animation systems

An energy-driven motion planning method for two distant postures

In this paper, we present a local motion planning algorithm for character animation. We focus on motion planning between two distant postures where linear interpolation leads to penetrations. Our framework has two stages. The motion planning problem is first solved as a Boundary Value Problem (BVP) on an energy graph which encodes penetrations, motion smoothness and user control. Having established a mapping from the configuration space to the energy graph, a fast and robust local motion planning algorithm is introduced to solve the BVP to generate motions that could only previously be computed by global planning methods. In the second stage, a projection of the solution motion onto a constraint manifold is proposed for more user control. Our method can be integrated into current keyframing techniques. It also has potential applications in motion planning problems in robotics

Opinion Piece: How People Structure Representations of Discourse

Mental models or situation models include representations of people, but much of the literature about such models focuses on the representation of eventualities (events, states, and processes) or (small-scale) situations. In the well-known event-indexing model of Zwaan, Langston, and Graesser (1995), for example, protagonists are just one of five dimensions on which situation models are indexed. They are not given any additional special status. Consideration of longer narratives, and the ways in which readers or listeners relate to them, suggest that people have a more central status in the way we think about texts, and hence in discourse representations, Indeed, such considerations suggest that discourse representations are organised around (the representations of) central characters. The paper develops the idea of the centrality of main characters in representations of longer texts, by considering, among other things, the way information is presented in novels, with L’Éducation Sentimentale by Gustav Flaubert as a case study. Conclusions are also drawn about the role of representations of people in the representation of other types of text