Data Driven Crowd Motion Control with Multi-touch Gestures

Controlling a crowd using multi‐touch devices appeals to the computer games and animation industries, as such devices provide a high‐dimensional control signal that can effectively define the crowd formation and movement. However, existing works relying on pre‐defined control schemes require the users to learn a scheme that may not be intuitive. We propose a data‐driven gesture‐based crowd control system, in which the control scheme is learned from example gestures provided by different users. In particular, we build a database with pairwise samples of gestures and crowd motions. To effectively generalize the gesture style of different users, such as the use of different numbers of fingers, we propose a set of gesture features for representing a set of hand gesture trajectories. Similarly, to represent crowd motion trajectories of different numbers of characters over time, we propose a set of crowd motion features that are extracted from a Gaussian mixture model. Given a run‐time gesture, our system extracts the K nearest gestures from the database and interpolates the corresponding crowd motions in order to generate the run‐time control. Our system is accurate and efficient, making it suitable for real‐time applications such as real‐time strategy games and interactive animation controls

Enter the Circle: Blending Spherical Displays and Playful Embedded Interaction in Public Spaces

Public displays are used a variety of contexts, from utility driven information displays to playful entertainment displays. Spherical displays offer new opportunities for interaction in public spaces, allowing users to face each other during interaction and explore content from a variety of angles and perspectives. This paper presents a playful installation that places a spherical display at the centre of a playful environment embedded with interactive elements. The installation, called Enter the Circle, involves eight chair-sized boxes filled with interactive lights that can be controlled by touching the spherical display. The boxes are placed in a ring around the display, and passers-by must “enter the circle” to explore and play with the installation. We evaluated this installation in a pedestrianized walkway for three hours over an evening, collecting on-screen logs and video data. This paper presents a novel evaluation of a spherical display in a public space, discusses an experimental design concept that blends displays with embedded interaction, and analyses real world interaction with the installation

COMPASS: A Formal Framework and Aggregate Dataset for Generalized Surgical Procedure Modeling

Purpose: We propose a formal framework for the modeling and segmentation of minimally-invasive surgical tasks using a unified set of motion primitives (MPs) to enable more objective labeling and the aggregation of different datasets. Methods: We model dry-lab surgical tasks as finite state machines, representing how the execution of MPs as the basic surgical actions results in the change of surgical context, which characterizes the physical interactions among tools and objects in the surgical environment. We develop methods for labeling surgical context based on video data and for automatic translation of context to MP labels. We then use our framework to create the COntext and Motion Primitive Aggregate Surgical Set (COMPASS), including six dry-lab surgical tasks from three publicly-available datasets (JIGSAWS, DESK, and ROSMA), with kinematic and video data and context and MP labels. Results: Our context labeling method achieves near-perfect agreement between consensus labels from crowd-sourcing and expert surgeons. Segmentation of tasks to MPs results in the creation of the COMPASS dataset that nearly triples the amount of data for modeling and analysis and enables the generation of separate transcripts for the left and right tools. Conclusion: The proposed framework results in high quality labeling of surgical data based on context and fine-grained MPs. Modeling surgical tasks with MPs enables the aggregation of different datasets and the separate analysis of left and right hands for bimanual coordination assessment. Our formal framework and aggregate dataset can support the development of explainable and multi-granularity models for improved surgical process analysis, skill assessment, error detection, and autonomy.Comment: 22 pages, 6 figures, 12 table

Interactive control of multi-agent motion in virtual environments

With the increased use of crowd simulation in animation, specification of crowd motion can be very time consuming, requiring a lot of user input. To alleviate this cost, we wish to allow a user to interactively manipulate the many degrees of freedom in a crowd, whilst accounting for the limitation of low-dimensional signals from standard input devices. In this thesis we present two approaches for achieving this: 1) Combining shape deformation methods with a multitouch input device, allowing a user to control the motion of the crowd in dynamic environments, and 2) applying a data-driven approach to learn the mapping between a crowd’s motion and the corresponding user input to enable intuitive control of a crowd. In our first approach, we represent the crowd as a deformable mesh, allowing a user to manipulate it using a multitouch device. The user controls the shape and motion of the crowd by altering the mesh, and the mesh in turn deforms according to the environment. We handle congestion and perturbation by having agents dynamically reassign their goals in the formation using a mass transport solver. Our method allows control of a crowd in a single pass, improving on the time taken by previous, multistage, approaches. We validate our method with a user study, comparing our control algorithm against a common mouse-based controller. We develop a simplified version of motion data patches to model character-environment interactions that are largely ignored in previous crowd research. We design an environment-aware cost metric for the mass transport solver that considers how these interactions affect a character’s ability to track the user’s commands. Experimental results show that our system can produce realistic crowd scenes with minimal, high-level, input signals from the user. In our second approach, we propose that crowd simulation control algorithms inherently impose restrictions on how user input affects the motion of the crowd. To bypass this, we investigate a data-driven approach for creating a direct mapping between low-dimensional user input and the resulting high-dimensional crowd motion. Results show that the crowd motion can be inferred directly from variations in a user’s input signals, providing a user with greater freedom to define the animation

Practical, appropriate, empirically-validated guidelines for designing educational games

There has recently been a great deal of interest in the potential of computer games to function as innovative educational tools. However, there is very little evidence of games fulfilling that potential. Indeed, the process of merging the disparate goals of education and games design appears problematic, and there are currently no practical guidelines for how to do so in a coherent manner. In this paper, we describe the successful, empirically validated teaching methods developed by behavioural psychologists and point out how they are uniquely suited to take advantage of the benefits that games offer to education. We conclude by proposing some practical steps for designing educational games, based on the techniques of Applied Behaviour Analysis. It is intended that this paper can both focus educational games designers on the features of games that are genuinely useful for education, and also introduce a successful form of teaching that this audience may not yet be familiar with

Pinching sweaters on your phone – iShoogle : multi-gesture touchscreen fabric simulator using natural on-fabric gestures to communicate textile qualities

The inability to touch fabrics online frustrates consumers, who are used to evaluating physical textiles by engaging in complex, natural gestural interactions. When customers interact with physical fabrics, they combine cross-modal information about the fabric's look, sound and handle to build an impression of its physical qualities. But whenever an interaction with a fabric is limited (i.e. when watching clothes online) there is a perceptual gap between the fabric qualities perceived digitally and the actual fabric qualities that a person would perceive when interacting with the physical fabric. The goal of this thesis was to create a fabric simulator that minimized this perceptual gap, enabling accurate perception of the qualities of fabrics presented digitally. We designed iShoogle, a multi-gesture touch-screen sound-enabled fabric simulator that aimed to create an accurate representation of fabric qualities without the need for touching the physical fabric swatch. iShoogle uses on-screen gestures (inspired by natural on-fabric movements e.g. Crunching) to control pre-recorded videos and audio of fabrics being deformed (e.g. being Crunched). iShoogle creates an illusion of direct video manipulation and also direct manipulation of the displayed fabric. This thesis describes the results of nine studies leading towards the development and evaluation of iShoogle. In the first three studies, we combined expert and non-expert textile-descriptive words and grouped them into eight dimensions labelled with terms Crisp, Hard, Soft, Textured, Flexible, Furry, Rough and Smooth. These terms were used to rate fabric qualities throughout the thesis. We observed natural on-fabric gestures during a fabric handling study (Study 4) and used the results to design iShoogle's on-screen gestures. In Study 5 we examined iShoogle's performance and speed in a fabric handling task and in Study 6 we investigated users' preferences for sound playback interactivity. iShoogle's accuracy was then evaluated in the last three studies by comparing participants’ ratings of textile qualities when using iShoogle with ratings produced when handling physical swatches. We also described the recording and processing techniques for the video and audio content that iShoogle used. Finally, we described the iShoogle iPhone app that was released to the general public. Our evaluation studies showed that iShoogle significantly improved the accuracy of fabric perception in at least some cases. Further research could investigate which fabric qualities and which fabrics are particularly suited to be represented with iShoogle

Human Motion Analysis and Synthesis in Computer Graphics

This thesis focuses on solving a challenging problem in the field of computer graphics, namely to model and understand 3D human motion efficiently and meaningfully. This is vital to achieve the analysis (health & sports science), synthesis (character animation) and control (video game) of human movements. Though numerous studies have focused on improving the results of motion analysis, motion synthesis and motion control, only a few of these studies solved the problems from the fundamental part owing to the lack of information encoded in motion data. In my works, the motion of human was divided into the three types, namely single human motion, multi-people interactions and crowd movement. Subsequently, I solved the problems from motion analysis to motion control in different types of motion. In the single human motion, two types of motion graphs on the motion sequence were proposed using Markov Process. The human motion is represented as the directed graphs, which suggests the number of action patterns and transitions among them. By analyzing the graphs topologies, the richness, transitions flexibility and unpredictability among different action patterns inside the human motion sequence can be easily verified. The framework here is capable of visualizing and analyzing the human motion on the high level of action preference, intention and diversity. For the two people interaction motion, the use of 3D volumetric meshes on the interacting people was proposed to model their movement and spatial relationship among them. The semantic meanings of the motions were defined by such relationship. A customized Earth Movers Distance was proposed to assess the topological and geometric difference between two groups of meshes. The above assessment captured the semantic similarities among different two-people interactions, which is consistent with what humans perceive. With this interaction motion representation, the multi-people interactions in semantic level can be retrieved and analyzed, and such complex movements can be easily adapted and synthesized with low computational costs. In the crowd movement, a data-driven gesture-based crowd control system was proposed, in which the control scheme was learned from example gestures provided by different users. The users gestures and corresponding crowd motions, representable to the crowd motions properties and irrelevant to style variations of gestures and crowd motions, were modelled into a compact low dimensional space. With this representation, the proposed framework can take an arbitrary users input gesture and generate appropriate crowd motion in real time. This thesis shows the advantages of higher-level human motion modelling in different scenarios and solves different challenging tasks of computer graphics. The unified framework summarizes the knowledge to analyze, synthesize and control the movement of human