DigitalBeing – Using the Environment as an Expressive Medium for Dance

Dancers express their feelings and moods through gestures and body movements. We seek to extend this mode of expression by dynamically and automatically adjusting music and lighting in the dance environment to reflect the dancer’s arousal state. Our intention is to offer a space that performance artists can use as a creative tool that extends the grammar of dance. To enable the dynamic manipulation of lighting and music, the performance space will be augmented with several sensors: physiological sensors worn by a dancer to measure her arousal state, as well as pressure sensors installed in a floor mat to track the dancers’ locations and movements. Data from these sensors will be passed to a three layered architecture. Layer 1 is composed of a sensor analysis system that analyzes and synthesizes physiological and pressure sensor signals. Layer 2 is composed of intelligent systems that adapt lighting and music to portray the dancer’s arousal state. The intelligent on-stage lighting system dynamically adjusts on-stage lighting direction and color. The intelligent virtual lighting system dynamically adapts virtual lighting in the projected imagery. The intelligent music system dynamically and unobtrusively adjusts the music. Layer 3 translates the high-level adjustments made by the intelligent systems in layer 2 to appropriate lighting board, image rendering, and audio box commands. In this paper, we will describe this architecture in detail as well as the equipment and control systems used

DigitalBeing: an Ambient Intelligent Dance Space.

DigitalBeing is an ambient intelligent system that aims to use stage lighting and lighting in projected imagery within a dance performance to portray dancer’s arousal state. The dance space will be augmented with pressure sensors to track dancers’ movements; dancers will also wear physiological sensors. Sensor data will be passed to a three layered architecture. Layer 1 is composed of a system that analyzes sensor data. Layer 2 is composed of two intelligent lighting systems that use the analyzed sensor information to adapt onstage and virtual lighting to show dancer’s arousal level. Layer 3 translates lighting changes to appropriate lighting board commands as well as rendering commands to render the projected imagery

Interactive dance choreography assistance

Creative support for the performing arts is prevalent in many fields, however, for the art of dance, automated tools supporting creativity have been scarce. In this research, we describe ongoing research into (semi)automatic automated creative choreography support. Based on state-of-the-art and a survey among 54 choreographers we establish functionalities and requirements for a choreography assistance tool, including the semantic levels at which it should operate and communicate with the end-users. We describe a user study with a prototype tool which presents choreography alternatives using various simple strategies in three dance styles. The results show that the needs for such a tool vary based on the dance discipline. In a second user study, we investigate various methods of presenting choreography variations. Here, we evaluate four presentation methods: textual descriptions, 2D animations, 3D animations and auditory instructions in two different dance styles. The outcome of the expert survey shows that the tool is effective in communicating the variations to the experts and that they express a preference for 3D animations. Based on these results, we propose a design for an interactive dance choreography assistant tool

Moving sounds and sonic moves : exploring interaction quality of embodied music mediation technologies through a user-centered perspective

This research project deals with the user-experience related to embodied music mediation technologies. More specifically, adoption and policy problems surrounding new media (art) are considered, which arise from the usability issues that to date pervade new interfaces for musical expression. Since the emergence of new wireless mediators and control devices for musical expression, there is an explicit aspiration of the creative industries and various research centers to embed such technologies into different areas of the cultural industries. The number of applications and their uses have exponentially increased over the last decade. Conversely, many of the applications to date still suffer from severe usability problems, which not only hinder the adoption by the cultural sector, but also make culture participants take a rather cautious, hesitant, or even downright negative stance towards these technologies. Therefore, this thesis takes a vantage point that is in part sociological in nature, yet has a link to cultural studies as well. It combines this with a musicological frame of reference to which it introduces empirical user-oriented approaches, predominantly taken from the field of human-computer-interaction studies. This interdisciplinary strategy is adopted to cope with the complex nature of digital embodied music controlling technologies. Within the Flanders cultural (and creative) industries, opportunities of systems affiliated with embodied interaction are created and examined. This constitutes an epistemological jigsaw that looks into 1) “which stakeholders require what various levels of involvement, what interactive means and what artistic possibilities?”, 2) “the way in which artistic aspirations, cultural prerequisites and operational necessities of (prospective) users can be defined?”, 3) “how functional, artistic and aesthetic requirements can be accommodated?”, and 4) “how quality of use and quality of experience can be achieved, quantified, evaluated and, eventually, improved?”. Within this multi-facetted problem, the eventual aim is to assess the applicability of the foresaid technology, both from a theoretically and empirically sound basis, and to facilitate widening and enhancing the adoption of said technologies. Methodologically, this is achieved by 1) applied experimentation, 2) interview techniques, 3) self-reporting and survey research, 4) usability evaluation of existing devices, and 5) human-computer interaction methods applied – and attuned – to the specific case of embodied music mediation technologies. Within that scope, concepts related to usability, flow, presence, goal assessment and game enjoyment are scrutinized and applied, and both task- and experience-oriented heuristics and metrics are developed and tested. In the first part, covering three chapters, the general context of the thesis is given. In the first chapter, an introduction to the topic is offered and the current problems are enumerated. In the second chapter, a broader theoretical background is presented of the concepts that underpin the project, namely 1) the paradigm of embodiment and its connection to musicology, 2) a state of the arts concerning new interfaces for musical expression, 3) an introduction into HCI-usability and its application domain in systematic musicology, 4) an insight into user-centered digital design procedures, and 5) the challenges brought about by e-culture and digitization for the cultural-creative industries. In the third chapter, the state of the arts concerning the available methodologies related to the thesis’ endeavor is discussed, a set of literature-based design guidelines are enumerated and from this a conceptual model is deduced which is gradually presented throughout the thesis, and fully deployed in the “SoundField”-project (as described in Chapter 9). The following chapters, contained in the second part of the thesis, give a quasi-chronological overview of how methodological concepts have been applied throughout the empirical case studies, aimed specifically at the exploration of the various aspects of the complex status quaestionis. In the fourth chapter, a series of application-based tests, predominantly revolving around interface evaluation, illustrate the complex relation between gestural interfaces and meaningful musical expression, advocating a more user-centered development approach to be adopted. In the fifth chapter, a multi-purpose questionnaire dubbed “What Moves You” is discussed, which aimed at creating a survey of the (prospective) end-users of embodied music mediation technologies. Therefore, it primarily focused on cultural background, musical profile and preferences, views on embodied interaction, literacy of and attitudes towards new technology and participation in digital culture. In the sixth chapter, the ethnographical studies that accompanied the exhibition of two interactive art pieces, entitled "Heart as an Ocean" & "Lament", are discussed. In these studies, the use of interview and questionnaire methodologies together with the presentation and reception of interactive art pieces, are probed. In the seventh chapter, the development of the collaboratively controlled music-game “Sync-In-Team” is presented, in which interface evaluation, presence, game enjoyment and goal assessment are the pivotal topics. In the eighth chapter, two usability studies are considered, that were conducted on prototype systems/interfaces, namely a heuristic evaluation of the “Virtual String” and a usability metrics evaluation on the “Multi-Level Sonification Tool”. The findings of these two studies in conjunction with the exploratory studies performed in association with the interactive art pieces, finally gave rise to the “SoundField”-project, which is recounted in full throughout the ninth chapter. The integrated participatory design and evaluation method, presented in the conceptual model is fully applied over the course of the “SoundField”-project, in which technological opportunities and ecological validity and applicability are investigated through user-informed development of numerous use cases. The third and last part of the thesis renders the final conclusions of this research project. The tenth chapter sets out with an epilogue in which a brief overview is given on how the state of the arts has evolved since the end of the project (as the research ended in 2012, but the research field has obviously moved on), and attempts to consolidate the implications of the research studies with some of the realities of the Flemish cultural-creative industries. Chapter eleven continues by discussing the strengths and weaknesses of the conceptual model throughout the various stages of the project. Also, it comprises the evaluation of the hypotheses, how the assumptions that were made held up, and how the research questions eventually could be assessed. Finally, the twelfth and last chapter concludes with the most important findings of the project. Also, it discusses some of the implications on cultural production, artistic research policy and offers an outlook on future research beyond the scope of the “SoundField” project

THE VARIETIES OF USER EXPERIENCE BRIDGING EMBODIED METHODOLOGIES FROM SOMATICS AND PERFORMANCE TO HUMAN COMPUTER INTERACTION

Embodied Interaction continues to gain significance within the field of Human Computer Interaction (HCI). Its growing recognition and value is evidenced in part by a remarkable increase in systems design and publication focusing on various aspects of Embodiment. The enduring need to interact through experience has spawned a variety of interdisciplinary bridging strategies in the hope of gaining deeper understanding of human experience. Along with phenomenology, cognitive science, psychology and the arts, recent interdisciplinary contributions to HCI include the knowledge-rich domains of Somatics and Performance that carry long-standing traditions of embodied practice. The common ground between HCI and the fields of Somatics and Performance is based on the need to understand and model human experience. Yet, Somatics and Performance differ from normative HCI in their epistemological frameworks of embodiment. This is particularly evident in their histories of knowledge construction and representation. The contributions of Somatics and Performance to the history of embodiment are not yet fully understood within HCI. Differing epistemologies and their resulting approaches to experience identify an under-theorized area of research and an opportunity to develop a richer knowledge and practice base. This is examined by comparing theories and practices of embodied experience between HCI and Somatics (Performance) and analyzing influences, values and assumptions underlying epistemological frameworks. The analysis results in a set of design strategies based in embodied practices within Somatics and Performance. The subsequent application of these strategies is examined through a series of interactive art installations that employ embodied interaction as a central expression of technology. Case Studies provide evidence in the form of rigorously documented design processes that illustrate these strategies. This research exemplifies 'Research through Art' applied in the context of experience design for tangible, wearable and social interaction

Proceedings of the 7th Sound and Music Computing Conference

Proceedings of the SMC2010 - 7th Sound and Music Computing Conference, July 21st - July 24th 2010