A Web-Oriented Multi-layer Model to Interact with Theatrical Performances

This paper presents an innovative approach to online fruition of theater performances. Web applications like traditional viewers are already available for the wide audience of Internet users. Our proposal aims at adding both interactivity and multi-layer fruition, and a way to manipulate and create new media. The premise to reach these goals is digitizing a number of heterogeneous materials in order to describe a single performance comprehensively, e.g. different video and audio-takes from different perspectives, and a number of related materials such as scripts, fashion plates, playbills, etc. The format we adopt to encode such information is based on the XML international standard known as IEEE 1599. Finally, an advanced Web player supporting search and play functions for synchronized materials must be designed. This work describes the whole process, from the acquisition of materials directly on the stage to their publishing on a Web portal

Managing Multiple Media Streams in HTML5: The IEEE 1599-2008 Case Study

This paper deals with the problem of managing multiple multimedia streams in a Web environment. Multimedia types to support are pure audio, video with no sound, and audio/video. Data streams refer to the same event or performance, consequently they both have and should maintain mutual synchronization. Besides, a Web player should be able to play different multimedia streams simultaneously, as well as to switch from one to another in real time. The clarifying example of a music piece encoded in IEEE 1599 format will be presented as a case study

Music-Related Media-Contents Synchronization over theWeb: the IEEE 1599 Initiative

IEEE 1599 is an international standard originally conceived for music, which aims at providing a comprehensive description of the media contents related to a music piece within a multi-layer and synchronized environment. A number of o_- line and stand-alone software prototypes has been realized after its standardization, occurred in 2008. Recently, thanks to some technological advances (e.g. the release of HTML5), the engine of the IEEE 1599 parser has been ported on the Web. Some non-trivial problems have been solved, e.g. the management of multiple simultaneous media streams in a client-server architecture. After providing an overview of the IEEE 1599 standard, this article presents a survey of the recent initiatives regarding audio-driven synchronization over the Web

OperA/ALIVE/OperettA

Comprehensive models for organizations must, on the one hand, be able to specify global goals and requirements but, on the other hand, cannot assume that particular actors will always act according to the needs and expectations of the system design. Concepts as organizational rules (Zambonelli 2002), norms and institutions (Dignum and Dignum 2001; Esteva et al. 2002), and social structures (Parunak and Odell 2002) arise from the idea that the effective engineering of organizations needs high-level, actor-independent concepts and abstractions that explicitly define the organization in which agents live (Zambonelli 2002).Peer ReviewedPostprint (author's final draft

Narrative approaches to design multi-screen augmented reality experiences

This paper explores how traditional narrative language used in film and theatre can be adapted to create interactivity and a greater sense of presence in the virtual heritage environment. It focuses on the fundamental principles of narrative required to create immersion and presence and investigates methods of embedding intangible social histories into these environments. These issues are explored in a case study of Greens Mill in the 1830’s, interweaving the story of the reform bill riots in Nottingham with the life of George Green, mathematician and proprietor of the Mill

The development of an audio-visual language for digital music performance

This practice-based PhD consists of a portfolio of creative work and a supporting commentary. The portfolio illustrates the design decisions relating to my digital music performance system, and focuses upon the visibility and the fluidity of digital music performance. The goal of the design is to enhance the visibility without violating the audience’s auditory imagination unnecessarily, and to enhance the fluidity without relinquishing the unique fixed nature of digital music. The performance system consists of an audio engine, control mapping engine and visual engine. The audio engine and the control mapping engine were programmed with Cycling ‘74 Max. They let the performer deconstruct and reconstruct pre-recorded audio files with her/his hands via MIDI controllers during performance. The visual engine was programmed with Derivative TouchDesigner. In various ways, it visualises and exaggerates the performer’s actions which cause sonic changes, and filters out the rest. The works are presented as videos and the supporting commentary, deals with the contexts and thinking processes which determined the current performance system. By exploring theories of electronic music performance, audiovisual, visual music, acousmatic and medium specificity, I aim to explain the reasoning behind the performance system

Computer-Assisted Interactive Documentary and Performance Arts in Illimitable Space

This major component of the research described in this thesis is 3D computer graphics, specifically the realistic physics-based softbody simulation and haptic responsive environments. Minor components include advanced human-computer interaction environments, non-linear documentary storytelling, and theatre performance. The journey of this research has been unusual because it requires a researcher with solid knowledge and background in multiple disciplines; who also has to be creative and sensitive in order to combine the possible areas into a new research direction. [...] It focuses on the advanced computer graphics and emerges from experimental cinematic works and theatrical artistic practices. Some development content and installations are completed to prove and evaluate the described concepts and to be convincing. [...] To summarize, the resulting work involves not only artistic creativity, but solving or combining technological hurdles in motion tracking, pattern recognition, force feedback control, etc., with the available documentary footage on film, video, or images, and text via a variety of devices [....] and programming, and installing all the needed interfaces such that it all works in real-time. Thus, the contribution to the knowledge advancement is in solving these interfacing problems and the real-time aspects of the interaction that have uses in film industry, fashion industry, new age interactive theatre, computer games, and web-based technologies and services for entertainment and education. It also includes building up on this experience to integrate Kinect- and haptic-based interaction, artistic scenery rendering, and other forms of control. This research work connects all the research disciplines, seemingly disjoint fields of research, such as computer graphics, documentary film, interactive media, and theatre performance together.Comment: PhD thesis copy; 272 pages, 83 figures, 6 algorithm

What do Collaborations with the Arts Have to Say About Human-Robot Interaction?

This is a collection of papers presented at the workshop What Do Collaborations with the Arts Have to Say About HRI , held at the 2010 Human-Robot Interaction Conference, in Osaka, Japan