Interaction design for live performance

PhD Thesis Multimedia item accompanying this thesis to be consulted at Robinson LibraryThe role of interactive technology in live performance has increased substantially in recent years. Practices and experiences of existing forms of live performance have been transformed and new genres of technology-­‐mediated live performance have emerged in response to novel technological opportunities. Consequently, designing for live performance is set to become an increasingly important concern for interaction design researchers and practitioners. However, designing interactive technology for live performance is a challenging activity, as the experiences of both performers and their audiences are shaped and influenced by a number of delicate and interconnected issues, which relate to different forms and individual practices of live performance in varied and often conflicting ways. The research presented in this thesis explores how interaction designers might be better supported in engaging with this intricate and multifaceted design space. This is achieved using a practice-­‐led methodology, which involves the researcher’s participation in both the investigation of, and design response to, issues of live performance as they are embodied in the lived and felt experiences of individual live performers’ practices during three interaction design case studies. This research contributes to the field of interaction design for live performance in three core areas. Understandings of the relationships between key issues of live performance and individual performers’ lived and felt experiences are developed, approaches to support interaction designers in engaging individual live performers’ lived and felt experiences in design are proposed and innovative interfaces and interaction techniques for live performance are designed. It is anticipated that these research outcomes will prove directly applicable or inspiring to the practices of interaction designers wishing to address live performance and will contribute to the ongoing academic discourse around the experience of, and design for, live performance.Engineering and Physical Sciences Research Council

Design fiction for mixed-reality performances

Designing for mixed-reality performances is challenging both in terms of technology design, and in terms of understanding the interplay between technology, narration, and (the outcomes of) audience interactions. This complexity also stems from the variety of roles in the creative team often entailing technology designers, artists, directors, producers, set-designers and performers. In this multidisciplinary, one-day workshop, we seek to bring together HCI scholars, designers, artists, and curators to explore the potential provided by Design Fiction as a method to generate ideas for Mixed-Reality Performance (MRP) through various archetypes including scripts, programs, and posters. By drawing attention to novel interactive technologies, such as bio-sensors and environmental IoT, we seek to generate design fiction scenarios capturing the aesthetic and interactive potential for mixed-reality performances, as well as the challenges to gain access to audience members’ data – i.e. physiological states, daily routines, conversations, etc

Design process of AirLoop-pedal: Experimenting with standard guitar pedals

My master’s thesis, consisting of both a written documentation and an experimental un- dertaking, is a report of the design process of AirLoop-pedal as well as a brief overview of today’s creative solutions in music. Within the scope of my thesis I investigate new creative strategies for guitar signal manipulation and standard guitar pedal controlling. AirLoop-pedal is a non-contact signal processing device for guitar that is designed to control standard guitar pedal loops through motion detection. This project is an experimental design process supported by the examples of experimental music and interface design in- troduced in subsequent chapters. The main outcome of the project is a functional prototype of AirLoop-pedal that corresponds with the requirements laid out during the design process. This work is comprised of two overarching ideas, both of which provide complementary supports to the understanding of this process. These concepts were an integral part of my thinking during the design process of AirLoop-pedal. Firstly: new technology creates new aesthetics of music, and secondly: coincidences have the ability to feed creativity. During this work I will examine creative strategies for experimental music, focusing on guitar signal processing, coincidental discoveries, experimenting with coincidences, and motion mapping and tracking with musical instruments. In the design of AirLoop-pedal I aim to utilize these main creative concepts, realizations and information gained throughout the duration of the design and building process, as well as my own experience as a musician

Supporting improvised games for young people in public spaces

PhD ThesisResearchers looking at technologically mediated play and games have explored how games can be taken away from the computer screen and played in outdoor spaces. This has resulted in new pervasive games that benefit from the opportunities for rich social and physical interaction in new mobile contexts. However, we have only just begun to explore these opportunities; game designs should bring young people together in these new contexts in play that is appropriate, meaningful, and can be enjoyed on their own mobile devices. The research in this thesis explores how game designers and interaction designers can design more playful mobile games for young people that can be played together in public spaces. This work draws upon a research through design approach that has been informed by the researcher’s own practice of game design and working co-creatively with custodians of public spaces. The contributions are based on the analysis of empirical data collected from two case studies in a community library and a country house, while additionally drawing upon three further game designs made in collaborations with other partners. This work contributes a game design framework that provides an approach, a step by step method, guidelines and a software library for making mobile games with more open, spontaneous, and improvised styles of play. The mobile games are designed with and based on a simplistic game system that presents digital playing cards to provide the game structure and bound play, while the mobile device is also used to configure the play space and sustain play. The intention is to provide designers with a practical and evidence-based approach to designing digital games for new mobile contexts. This work will appeal to game designers who are motivated by an interest in play and playfulness that will resonate with our childhood memories of play.UK AHRC KE Hub for the Creative Economy (ref: AH/J005150/1 Creative Exchange

Lights, camera, interaction! Interactive Film and its Transformative Potential.

This thesis concentrates on the emerging field of interactive film. Digital interactive and networked media offer so many possibilities to create stories that it is necessary to define what an interactive film is and whether there is any continuity with the linear film form. This thesis explores whether interactive narratives in the form of interactive film have the potential to offer a transformative learning experience regarding societal and political topics. Butterfly is an interactive short film that uses a second-screen technique to raise awareness of the dangers of cyberbullying. As a case study for a potential transformative experience, the film is described and evaluated by means of interactive screenings and a user experience study. Findings show there is definite potential for interactive films to create strong emotions in users and to possibly produce a transformative experience with educational implications. Keywords: Transformative design, Interactive Narrative Design, Interactive film, User Experience Evaluation

Embodied Imagination: An Exploration of Participatory Performance and Interactive Technology to Support Stroke Recovery

Life after a stroke leads to the challenge of adjusting to new possibilities and fosters an increased risk of social isolation and depression. Re-building personal narratives and creating new community networks are key to conceiving an identity beyond the stroke. In this context, participatory performance provides methods for exploring physical and social identities, imagining new ways of being. Meanwhile, digital technology offers tools to help envision these possibilities. A participatory performance workshop supported by real-time motion capture technology has been redesigned in collaboration with the performance company Split Britches. The Green Screening workshop’s objective is to help stroke survivors imagine new physical and social possibilities by enacting fantasies of things they have always wanted to do. Participants construct storylines supported by a custom-built interactive scenography. Movement data is collected and transformed into real-time visualisations to progressively build fantasy narratives enacted with and for other participants. Three research studies conducted with stroke support groups around England analysed progressive iterations of the Green Screening workshop. The first study focused on the project's feasibility in aiding social support. The second study explored embodied imagination and social collaboration in enacted storylines. Finally, the third study analysed communication as a means of recovery and further potential to foster social collaboration. Findings are based on qualitative analysis of the participants' experience. Results reveal that the narrative process and visualisations encouraged a rich repertoire of improvised movements, and the communal aspect of the process was found especially important in achieving these results. This work argues that this framework can simultaneously bring a rich, prospective and political understanding of people’s lived experience to the design space in HCI and provide community stroke support

THE PHILOSOPHY OF ACTION IN LIVE PERFORMANCE INTERACTION DESIGN: ALIGNING FLOWS OF INTENTIONALITY

In recent years, ubiquitous computing has altered traditional performance spaces. Arts organizations have notably tested various strategies to either accommodate or eliminate the persistent and disruptive “glowing screen” of smartphones. While theatre and performance artists and scholars correctly identify many problems created by this influx of new technology, this dissertation argues that the rise of ubiquitous computing presents immense potential for theatre and performance studies to begin solving the design problems faced by computer scientists and user experience designers. Theatre and performance scholars hold a crucial role in ubiquitous technology design for live performance, and we have key knowledge of action that user experience designers seek now more than ever. I propose that human action is the basis for a common nomenclature and theoretical bridge between user experience design and theatre and performance studies. I extend Aristotle’s intentionalist mimetic theory using current philosophy of action and cognitive science, and argue that performance artists and designers select and align flows of intentionality in action that immerse spectators in the intentional presentation of an action. Furthermore, I follow Elizabeth Anscombe’s theory of action to argue for the incommensurability of propositionally articulated theoretical knowledge and non-propositional practical knowledge. Audiences experience the flow of a performance as they ascertain the interweaving of these incommensurable yet complimentary articulations of intentionality through a reciprocal feedback loop of active perception. Both performers and audiences derive the meaning of a performance from an “expanded description” of the teleological structure of actions that comprise it. This action-centric analysis of performance provides the basis for dialogue with human experience designers through an ecologically balanced mapping of the four Aristotelian causes of a performance onto the design of new technology. As a practical application of this theoretical framework, the dissertation also proposes a new platform for smartphone-based audience interactivity at live Jazz concerts. Applying the theoretical argument to the intentional flows of action in live jazz, the Nymbus system seeks to align the material, formal, and efficient causality of smartphones at concerts with the intentional flows in jazz performance in order to heighten and compliment audience immersion in jazz performance flow