3D (embodied) projection mapping and sensing bodies : a study in interactive dance performance

This dissertation identifies the synergies between physical and virtual environments when designing for immersive experiences in interactive dance performances. The integration of virtual information in physical space is transforming our interactions and experiences with the world. By using the body and creative expression as the interface between real and virtual worlds, dance performance creates a privileged framework to research and design interactive mixed reality environments and immersive augmented architectures. The research is primarily situated in the fields of visual art and interaction design. It combines performance with transdisciplinary fields and intertwines practice with theory. The theoretical and conceptual implications involved in designing and experiencing immersive hybrid environments are analyzed using the reality–virtuality continuum. These theories helped frame the ways augmented reality architectures are achieved through the integration of dance performance with digital software and reception displays. They also helped identify the main artistic affordances and restrictions in the design of augmented reality and augmented virtuality environments for live performance. These pervasive media architectures were materialized in three field experiments, the live dance performances. Each performance was created in three different stages of conception, design and production. The first stage was to “digitize” the performer’s movement and brain activity to the virtual environment and our system. This was accomplished through the use of depth sensor cameras, 3D motion capture, and brain computer interfaces. The second stage was the creation of the computational architecture and software that aggregates the connections and mapping between the physical body and the spatial dynamics of the virtual environment. This process created real-time interactions between the performer’s behavior and motion and the real-time generative computer 3D graphics. Finally, the third stage consisted of the output modality: 3D projector based augmentation techniques were adopted in order to overlay the virtual environment onto physical space. This thesis proposes and lays out theoretical, technical, and artistic frameworks between 3D digital environments and moving bodies in dance performance. By sensing the body and the brain with the 3D virtual environments, new layers of augmentation and interactions are established, and ultimately this generates mixed reality environments for embodied improvisational self-expression.Radio-Television-Fil

Generative theatre of totality

Generative art can be used for creating complex multisensory and multimedia experiences within predetermined aesthetic parameters, characteristic of the performing arts and remarkably suitable to address Moholy-Nagy's Theatre of Totality vision. In generative artworks the artist will usually take on the role of an experience framework designer, and the system evolves freely within that framework and its defined aesthetic boundaries. Most generative art impacts visual arts, music and literature, but there does not seem to be any relevant work exploring the cross-medium potential, and one could confidently state that most generative art outcomes are abstract and visual, or audio. It is the goal of this article to propose a model for the creation of generative performances within the Theatre of Totality's scope, derived from stochastic Lindenmayer systems, where mapping techniques are proposed to address the seven variables addressed by Moholy-Nagy: light, space, plane, form, motion, sound and man ("man" is replaced in this article with "human", except where quoting from the author), with all the inherent complexities

Aesthetic potential of human-computer interaction in performing arts

Human-computer interaction (HCI) is a multidisciplinary area that studies the communication between users and computers. In this thesis, we want to examine if and how HCI when incorporated into staged performances can generate new possibilities for artistic expression on stage. We define and study four areas of technology-enhanced performance that were strongly influenced by HCI techniques: multimedia expression, body representation, body augmentation and interactive environments. We trace relevant artistic practices that contributed to the exploration of these topics and then present new forms of creative expression that emerged after the incorporation of HCI techniques. We present and discuss novel practices like: performer and the media as one responsive entity, real-time control of virtual characters, on-body projections, body augmentation through humanmachine systems and interactive stage design. The thesis concludes by showing some concrete examples of these novel practices implemented in performance pieces. We present and discuss technologyaugmented dance pieces developed during this master’s degree. We also present a software tool for aesthetic visualisation of movement data and discuss its application in video creation, staged performances and interactive installations

Augmented reality or augmeted space?

Architectural mapping, fa\ue7ade projection, 3D projection videomapping, display surfaces, architectural Vj set are some of the definitions used for a new artistic format and a new technique that consists in projecting video images on buildings, fa\ue7ades, and other structures in public spaces (but also in theatres and museums) or on nearly any kind of complex surface or 3D object to shatter the viewer\u2019s perception of perspective. The projector allows bending and highlighting of any shape, line or space. It creates astonishing optical illusions\u2013a suggestive play of light that turns a physical object into something else by changing its perceived form. The context is that of so-called \u201caugmented reality.\u201

A dramaturgy of intermediality: composing with integrative design

The thesis investigates and develops a compositional system on intermediality in theatre and performance as a dramaturgical practice through integrative design. The position of the visual/sonic media in theatre and performance has been altered by the digitalisation and networking of media technologies, which enables enhanced dynamic variables in the intermedial processes. The emergent intermediality sites are made accessible by developments in media technologies and form part of broader changes towards a mediatised society: a simultaneous shift in cultural contexts, theatre practice and audience perception. The practice-led research is situated within a postdramatic context and develops a system of compositional perspectives and procedures to enhance the knowledge of a dramaturgy on intermediality. The intermediality forms seem to re-situate the actual/virtual relations in theatre and re-construct the processes of theatricalisation in the composition of the stage narrative. The integration of media and performers produces a compositional environment of semiosis, where the theatre becomes a site of narration, and the designed integration in-between medialities emerges as intermediality sites in the performance event. A selection of performances and theatre directors is identified, who each in distinct ways integrate mediating technologies as a core element in their compositional design. These directors and performances constitute a source of reflection on compositional strategies from the perspective of practice, and enable comparative discussions on dramaturgical design and the consistency of intermediality sites. The practice-led research realised a series of prototyping processes situated in performance laboratories in 2004-5. The laboratories staged investigations into the relation between integrative design procedures and parameters for composition of intermediality sites, particularly the relative presence in-between the actual and the virtual, and the relative duration and distance in-between timeness and placeness. The integration of performer activities and media operations into dramaturgical structures were developed as a design process of identifying the mapping and experiencing the landscape through iterative prototyping. The developed compositional concepts and strategies were realised in the prototype performance Still I Know Who I Am, performed October 2006. This final research performance was a full-scale professional production, which explored the developed dramaturgical designs through creative practice. The performance was realised as a public event, and composed of a series of scenes, each presenting a specific composite of the developed integrative design strategies, and generating a particular intermediality site. The research processes in the performance laboratories and the prototype performance developed on characteristics, parameters and procedures of compositional strategies, investigating the viability of a dramaturgy of intermediality through integrative design. The practice undertaken constitutes raw material from which the concepts are drawn and underpins the premises for the theoretical reflections

An aesthetics of touch: investigating the language of design relating to form

How well can designers communicate qualities of touch? This paper presents evidence that they have some capability to do so, much of which appears to have been learned, but at present make limited use of such language. Interviews with graduate designer-makers suggest that they are aware of and value the importance of touch and materiality in their work, but lack a vocabulary to fully relate to their detailed explanations of other aspects such as their intent or selection of materials. We believe that more attention should be paid to the verbal dialogue that happens in the design process, particularly as other researchers show that even making-based learning also has a strong verbal element to it. However, verbal language alone does not appear to be adequate for a comprehensive language of touch. Graduate designers-makers’ descriptive practices combined non-verbal manipulation within verbal accounts. We thus argue that haptic vocabularies do not simply describe material qualities, but rather are situated competences that physically demonstrate the presence of haptic qualities. Such competencies are more important than groups of verbal vocabularies in isolation. Design support for developing and extending haptic competences must take this wide range of considerations into account to comprehensively improve designers’ capabilities