Representation and recognition of action in interactive spaces

Thesis (Ph.D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 1999.Includes bibliographical references (p. 246-258).This thesis presents new theory and technology for the representation and recognition of complex, context-sensitive human actions in interactive spaces. To represent action and interaction a symbolic framework has been developed based on Roger Schank's conceptualizations, augmented by a mechanism to represent the temporal structure of the sub-actions based on Allen's interval algebra networks. To overcome the exponential nature of temporal constraint propagation in such networks, we have developed the PNF propagation algorithm based on the projection of IA-networks into simplified, 3-valued (past, now, future) constraint networks called PNF-networks. The PNF propagation algorithm has been applied to an action recognition vision system that handles actions composed of multiple, parallel threads of sub-actions, in situations that can not be efficiently dealt by the commonly used temporal representation schemes such as finite-state machines and HMMs. The PNF propagation algorithm is also the basis of interval scripts, a scripting paradigm for interactive systems that represents interaction as a set of temporal constraints between the individual components of the interaction. Unlike previously proposed non-procedural scripting methods, we use a strong temporal representation (allowing, for example, mutually exclusive actions) and perform control by propagating the temporal constraints in real-time. These concepts have been tested in the context of four projects involving story-driven interactive spaces. The action representation framework has been used in the Intelligent Studio project to enhance the control of automatic cameras in a TV studio. Interval scripts have been extensively employed in the development of "SingSong ", a short interactive performance that introduced the idea of live interaction with computer graphics characters; in "It/I", a full-length computer theater play; and in "It", an interactive art installation based on the play "It /I" that realizes our concept of immersive stages, that is, interactive spaces that can be used both by performers and public.by Claudio Santos Pinhanez.Ph.D

The SCD Architecture and its Use in the Design of Story-Driven Interactive Spaces

Abstract. In this paper we examine story-driven interactive spaces and argue that story development requires centralized control of the actions of the characters that inhabit such environments. This argument leads us to propose the story-character-device (SCD) architecture that, unlike previous approaches, separates the locus of story control in a distinct software module. To exemplify the use of the architecture we examine an art installation (called “It”) that we built according to the SCD model. The development process of “It ” is then compared to the design of another story-driven interactive space that used similar technology and thematic, the computer theater play “It / I”. By looking into the design process of the two environments, we conclude that the SCD architecture allows much more flexibility in the design process of story lines besides bringing an overall simplification of the control system.

Ubiquitous Computing in the Living Room: Concept Sketches and an Implementation of a Persistent User Interface

This video shows some concept sketches of applications that might be created for a living room with ubiquitous display and laser pointer interaction technology. A fully-functioning prototype of a persistent interface is also described: a language-learning tool