Markerless Motion Capture in the Crowd

This work uses crowdsourcing to obtain motion capture data from video recordings. The data is obtained by information workers who click repeatedly to indicate body configurations in the frames of a video, resulting in a model of 2D structure over time. We discuss techniques to optimize the tracking task and strategies for maximizing accuracy and efficiency. We show visualizations of a variety of motions captured with our pipeline then apply reconstruction techniques to derive 3D structure.Comment: Presented at Collective Intelligence conference, 2012 (arXiv:1204.2991

Running up Blueberry Hill: Prototyping whole body interaction in harmony space

Musical harmony is considered to be one of the most abstract and technically difficult parts of music. It is generally taught formally via abstract, domain-specific concepts, principles, rules and heuristics. By contrast, when harmony is represented using an existing interactive desktop tool, Harmony Space, a new, parsimonious, but equivalently expressive, unified level of description emerges. This focuses not on abstract concepts, but on concrete locations, objects, areas and trajectories. This paper presents a design study of a prototype version of Harmony Space driven by whole body navigation, and characterizes the new opportunities presented for the principled manipulation of chord sequences and bass lines. These include: deeper engagement and directness; rich physical cues for memory and reflection, embodied engagement with rhythmic time constraints; hands which are free for other simultaneous activities (such as playing a traditional instrument); and qualitatively new possibilities for collaborative use

From presence to consciousness through virtual reality

Immersive virtual environments can break the deep, everyday connection between where our senses tell us we are and where we are actually located and whom we are with. The concept of 'presence' refers to the phenomenon of behaving and feeling as if we are in the virtual world created by computer displays. In this article, we argue that presence is worthy of study by neuroscientists, and that it might aid the study of perception and consciousness