Negotiating Reality

Our understanding of research through design is demonstrated by a close examination of the methods used in the project lifeClipper2. This design research project investigates the applicability of immersive outdoor Augmented Reality (AR). lifeClipper2 offers an audiovisual walking experience in a virtually extended public space and focuses on audiovisual perception as well as on the development of the appropriate technology. The project involves contributions of partners from different fields of research. Thus, lifeClipper2 is able to test the potential of AR for visualizing architecture and archaeological information and to challenge our understanding of perception and interaction. Using examples from our research, the paper reflects on how scenario design contributes to the production of design knowledge and explores the possibilities and variations of AR. Finally, the paper drafts our approach to design research. The three tenets of our work are: the use of scenarios as a tool of interdisciplinary research, the experimental exploration of media and the intention to make design knowledge explicit. Keywords: augmented reality; locative media; hybrid environment; immersion; perception; experience design; research through design; scenario design</p

From Candle Light to Contemporary Lighting Systems: How Lighting Technology Shapes Scenographic Practices

In this article, I discuss the influence of stage lighting on the processes of scenic design and the functioning of performance space. There has been a huge advance in lighting technology with regard to their accessibility, usability, luminosity and costs during the past decades. Light can no longer be thought of as a necessity that can just be added to the performance. It has become one of its basic visual elements, directing and focusing the spectators gaze. The rhythm of changing lighting cues create a visual dramaturgy, which has turned visual design from solid constructrions to a score of temporal events. Today you seldom see a performance without any use of projections or digital videos. I begin with a quick historical survey on the adaptation of electric light in order to exemplify the artistic significance of technological innovations. I move on to a more philosophical conversation about the metaphorical connotations of light as a basic component of the visual mise-en-scène. Then I return to the practices of contemporary theatre making and examine the contributions of the latest projection technology. I suggest that stage lighting has developed from being a technical tool making the scenes visible into a sovereign artistic agency creating images on its own terms. Today’s intermedial scenography can be seen as a parallel to the contemporary experience of our spatio-visual environment in everyday life, echoing the changes that happen in our ways of perceiving and conceptualizing the world

Wearable performance

This is the post-print version of the article. The official published version can be accessed from the link below - Copyright @ 2009 Taylor & FrancisWearable computing devices worn on the body provide the potential for digital interaction in the world. A new stage of computing technology at the beginning of the 21st Century links the personal and the pervasive through mobile wearables. The convergence between the miniaturisation of microchips (nanotechnology), intelligent textile or interfacial materials production, advances in biotechnology and the growth of wireless, ubiquitous computing emphasises not only mobility but integration into clothing or the human body. In artistic contexts one expects such integrated wearable devices to have the two-way function of interface instruments (e.g. sensor data acquisition and exchange) worn for particular purposes, either for communication with the environment or various aesthetic and compositional expressions. 'Wearable performance' briefly surveys the context for wearables in the performance arts and distinguishes display and performative/interfacial garments. It then focuses on the authors' experiments with 'design in motion' and digital performance, examining prototyping at the DAP-Lab which involves transdisciplinary convergences between fashion and dance, interactive system architecture, electronic textiles, wearable technologies and digital animation. The concept of an 'evolving' garment design that is materialised (mobilised) in live performance between partners originates from DAP Lab's work with telepresence and distributed media addressing the 'connective tissues' and 'wearabilities' of projected bodies through a study of shared embodiment and perception/proprioception in the wearer (tactile sensory processing). Such notions of wearability are applied both to the immediate sensory processing on the performer's body and to the processing of the responsive, animate environment. Wearable computing devices worn on the body provide the potential for digital interaction in the world. A new stage of computing technology at the beginning of the 21st Century links the personal and the pervasive through mobile wearables. The convergence between the miniaturisation of microchips (nanotechnology), intelligent textile or interfacial materials production, advances in biotechnology and the growth of wireless, ubiquitous computing emphasises not only mobility but integration into clothing or the human body. In artistic contexts one expects such integrated wearable devices to have the two-way function of interface instruments (e.g. sensor data acquisition and exchange) worn for particular purposes, either for communication with the environment or various aesthetic and compositional expressions. 'Wearable performance' briefly surveys the context for wearables in the performance arts and distinguishes display and performative/interfacial garments. It then focuses on the authors' experiments with 'design in motion' and digital performance, examining prototyping at the DAP-Lab which involves transdisciplinary convergences between fashion and dance, interactive system architecture, electronic textiles, wearable technologies and digital animation. The concept of an 'evolving' garment design that is materialised (mobilised) in live performance between partners originates from DAP Lab's work with telepresence and distributed media addressing the 'connective tissues' and 'wearabilities' of projected bodies through a study of shared embodiment and perception/proprioception in the wearer (tactile sensory processing). Such notions of wearability are applied both to the immediate sensory processing on the performer's body and to the processing of the responsive, animate environment

Windows of memory: perspective panels to communicate archeological heritage

Virtual Archaeology assumed, since the second half of 1900, an educational role: digital reconstructions of monuments, cities and territories aim to communicate the ancient cultural heritage using tools, immediately suitable for the large public, which can thus make a proper, yet intuitive reading of the past, thanks to the likelihood of the images represented. However, this "virtuality" in no way can be compared with the excitement of the visit to an archaeological site, which inevitably leads to a strong impact on the cognitive processes of the visitors. We propose an innovative exhibition in situ of some Greek and Roman ruins, based on a correct hypothesis of reconstruction with a virtual 3D model: the aim of our project is to help the visitor to interpreter the lost spatiality of the monument during the visit archaeological tour. In order to enrich the emotional impact of the visitor and make easier to communicate the historical and architectural value of the archaeological heritage, our project provides a display based on a brand new visual effect of anamorphic perspectives that, overlapping a drawing to the ruins still visible, reconstructs in the eyes of the observer, the original shape of the building. It is a recomposition of signs in a single perspective view, transcribed on a glass panel that we have named "Windows of memory". The transparent panels on which are printed some perspective views taken from the 3D virtual model, collaborate actively in the unveiling the lost architecture, in a cognitive process of strong impact, comparable only to the more expensive "augmented reality", that, however, interposes an electronic device between the object and the observer, where anamorphosis allows you to enjoy the architectonic space only through the direct vision. The project we propose in the article is a flexible solution to be adapted to several archeological sites. It will provide the observer with an emotional experience, so communicating, through a multisensorial effect, the inner value of the good visited

Scenography and new media technologies: history, educational applications and visualization techniques

The endemic presence of digital technology is responsible for numerous changes in contemporary Western societies. This study examines the role of multimedia within the field of theatre studies, with particular focus on the theory and practice of theatre design and education. In the cross-disciplinary literature review, I investigate such primary elements of contemporary media as interactivity, immersion, integration and hyper-textuality, and explore their characteristics in the performing arts before and during the digital epoch. I also discuss various IT applications that transformed the way we experience, learn and co-create our cultural heritage. In order to illustrate how computer-generated environments could change the way we perceive and deliver cultural values, I explore a suite of rapidly-developing communication and computer-visualization techniques, which enable reciprocal exchange between viewers, theatre performances and artefacts. I analyze novel technology-mediated teaching techniques that attempt to provide a new media platform for visually-enhanced information transfer. My findings indicate that the recent changes towards the personalization of knowledge delivery and also towards student-centered study and e-learning necessitated the transformation of the learners from passive consumers of digital products to active and creative participants in the learning experience. The analysis of questionnaires and two case studies (the THEATRON and the VA projects) demonstrate the need for further development of digital-visualization techniques, especially for studying and researching scenographic artefacts. As a practical component of this thesis, I have designed and developed the Set-SPECTRUM educational project, which aims to strengthen the visual skills of the students, ultimately enabling them to use imagery as a creative tool, and as a means to analyze theatrical performances and artefacts. The 3D reconstruction of Norman Bel Geddes' set for The Divine Comedy, first of all, enables academic research of the artefact, exposing some hitherto unknown design-limitations in the original set-model, and revealing some construction inconsistencies; secondly, it contributes to educational and creative practices, offering an innovative way to learn about scenography. And, thirdly, it fills a gap in the history of the Western theatre design. This study attempts to show that when translated into digital language, scenographic artefacts become easily retrievable and highly accessible for learning and research purposes. Therefore, the development of such digital products should be encouraged, but care should also be taken to provide the necessary training for users, in order to realize the applications' full potential

Curating Multisensory Experiences: The Possibilities of Immersive Exhibitions

The immersive exhibition is a specialized exhibition genre flourishing in the 21st century. It creates illusions of time and space by applying multisensory display methods to envelop visitors in a three-dimensional virtual world. The enhanced sense of presence facilitates public engagement. Not only have immersive exhibitions transformed audience experience, they have triggered discussions on the evolution of the museum and how curators can productively incorporate new technology. Based on interdisciplinary theory and the case studies of Grande Experiences (Melbourne) and teamLab (Tokyo), this thesis addresses issues around defining, developing, and curating immersive exhibitions. I discuss the genre’s operation from three perspectives: managing the physical space by using nonlinear thematic frameworks and dynamic displays; curating audiences’ subjective immersive experience by applying natural sensory interactions, mediator-based interactions, and collaborative participation; and identifying multisensory stimulation as immersive exhibitions’ critical attribute. These aspects hold possibilities for implementation in all museum contexts. The purpose of this thesis is to offer curators and exhibition creators practical guidelines on reshaping existing displays and collections with immersive design and technology. Immersive exhibitions encourage museums to rethink the possibilities for enabling and diversifying connections between art, technology, and the public in the digital era

Shadows, touch and digital puppeteering: a media archaeological approach

Aims The practical aim of this research project is to create a multi-touch digital puppetry system that simulates shadow theatre environments and translates gestural acts of touch into live and expressive control of virtual shadow figures. The research is focussed on the qualities of movement achievable through the haptics of single and multi-touch control of the digital puppets in the simulation. An associated aim is to create a collaborative environment where multiple performers can control dynamic animation and scenography, and create novel visualisations and narratives. The conceptual aim is to link traditional and new forms of puppetry seeking cultural significance in the ‘remediation’ of old forms that avail themselves of new haptic resources and collaborative interfaces. The thesis evaluates related prior art where traditional worlds of shadow performance meet new media, digital projection and 3D simulation, in order to investigate how changing technical contexts transform the potential of shadows as an expressive medium. Methodology The thesis uses cultural analysis of relevant documentary material to contextualise the practical work by relating the media archaeology of 2D puppetry—shadows, shadowgraphs and silhouettes—to landmark work in real-time computer graphics and performance animation. The survey considers the work of puppeteers, animators, computer graphics specialists and media artists. Through practice and an experimental approach to critical digital creativity, the study provides practical evidence of multiple iterations of controllable physics-based animation delivering expressive puppet motion through touch and multiuser interaction. Video sequences of puppet movement and written observational analysis document the intangible aspects of animation in performance. Through re-animation of archival shadow puppets, the study presents an emerging artistic media archaeological method. The major element of this method has been the restoration of a collection of Turkish Karagöz Shadow puppets from the Institut International de la Marionnette (Charleville, France) into a playable digital form. Results The thesis presents a developing creative and analytical framework for digital shadow puppetry. It proposes a media archaeological method for working creatively with puppet archives that unlock the kinetic and expressive potential of restored figures. The interaction design introduces novel approaches to puppetry control systems—using spring networks—with objects under physics-simulation that demonstrate emergent expressive qualities. The system facilitates a dance of agency¹ between puppeteer and digital instrument. The practical elements have produced several software iterations and a tool-kit for generating elegant, nuanced multi-touch shadow puppetry. The study presents accidental discoveries—serendipitous benefits of open-ended practical exploration. For instance: the extensible nature of the control system means novel input—other than touch—can provide exciting potential for accessible user interaction, e.g. with gaze duration and eye direction. The study also identifies limitations including the rate of software change and obsolescence, the scope of physics-based animation and failures of simulation. Originality/value The work has historical value in that it documents and begins a media archaeology of digital puppetry, an animated phenomenon of increasing academic and commercial interest. The work is of artistic value providing an interactive approach to making digital performance from archival material in the domain of shadow theatre. The work contributes to the electronic heritage of existing puppetry collections. The study establishes a survey of digital puppetry, setting a research agenda for future studies. Work may proceed to digitise, rig and create collaborative and web-mediated touch-based motion control systems for 2D and 3D puppets. The present study thus provides a solid platform to restore past performances and create new work from old, near forgotten-forms. ¹ Following Andrew Pickering, puppetry is ‘a temporally extended back-and-forth dance of human and non-human agency in which activity and passivity on both sides are reciprocally intertwined’ PICKERING, A. 2010. Material Culture and the Dance of Agency. In: BEAUDRY, M. C. & HICKS, D. (eds.) Oxford Handbook of Material Culture Studies. Oxford University Press.

Virtual Reality and Spatial Augmented Reality for Social Inclusion: The “Includiamoci” Project

Extended Reality (XR) technology represents an innovative tool to address the challenges of the present, as it allows for experimentation with new solutions in terms of content creation and its fruition by different types of users. The potential to modulate the experience based on the target audience’s needs and the project’s objectives makes XR suitable for creating new accessibility solutions. The “Includiamoci” project was carried out with the aim of creating workshops on social inclusion through the combination of art and technology. Specifically, the experimentation involved ten young people between the ages of 28 and 50, with cognitive disabilities, who participated in Extended Reality workshops and Art Therapy workshops. In the course of these activities, the outputs obtained were two: a virtual museum, populated by the participants’ works, and a digital set design for a theatrical performance. Through two tests, one on user experience (UX) and one on the degree of well-being, the effectiveness of the entire project was evaluated. In conclusion, the project demonstrated how the adopted solutions were appropriate to the objectives, increasing our knowledge of UX for a target audience with specific user needs and using XR in the context of social inclusion