Analysis of Noisy Biosignals for Musical Performance

Proceeding volume: 7619Peer reviewe

Bridging the gap between emotion and joint action

Our daily human life is filled with a myriad of joint action moments, be it children playing, adults working together (i.e., team sports), or strangers navigating through a crowd. Joint action brings individuals (and embodiment of their emotions) together, in space and in time. Yet little is known about how individual emotions propagate through embodied presence in a group, and how joint action changes individual emotion. In fact, the multi-agent component is largely missing from neuroscience-based approaches to emotion, and reversely joint action research has not found a way yet to include emotion as one of the key parameters to model socio-motor interaction. In this review, we first identify the gap and then stockpile evidence showing strong entanglement between emotion and acting together from various branches of sciences. We propose an integrative approach to bridge the gap, highlight five research avenues to do so in behavioral neuroscience and digital sciences, and address some of the key challenges in the area faced by modern societies

Interactive Spaces: Model for Motion-based Music Applications

With the extensive utilization of touch screens, smartphones and various reactive surfaces, reality- based and intuitive interaction styles have now become customary. The employment of larger interactive areas, like floors or peripersonal three-dimensional spaces, further increase the reality- based interaction affordances, allowing full-body involvement and the development of a co- located, shared user experience. Embodied and spatial cognition play a fundamental role for the interaction in this kind of spaces, where users act in the reality with no device in the hands and obtain an audio and graphical output depending on their movements. Starting from the early experiments of Myron Krueger in 1971, responsive floors have been developed through various technologies including sensorized tiles and computer vision systems, to be employed in learn- ing environments, entertainment, games and rehabilitation. Responsive floors allow the spatial representation of concepts and for this reason are suitable for immediate communication and engagement. As many musical features have meaningful spatial representations, they can easily be reproduced in the physical space through a conceptual blending approach and be made available to a great number of users. This is the key idea for the design of the original music applications presented in this thesis. The applications, devoted to music learning, production and active listening, introduce a novel creative approach to music, which can be further assumed as a general paradigm for the design of motion-based learning environments. Application assessment with upper elementary and high school students has proved that users engagement and bodily inter- action have a high learning power, which can be a valid resource for deeper music knowledge and more creative learning processes. Although further interface tests showed that touch screen interaction performs better than full-body interaction, some important guidelines for the design of reactive floors applications have been obtained on the basis of these test results. Moreover, the conceptual framework developed for the design of music applications can represent a valid paradigm also in the general field of human-computer interaction

Unheard Sounds: A Manual for the Sonification of Emotional States in an Artistic Space

This paper explores the use of sonification technology within an artistic space, specifically a live musical performance setting. The current work should be thought of as a manual for performers, laying out the theoretical background for understanding the use of this technology as well as the technical components for successful incorporation. Most of the theoretical knowledge presented in this paper focuses on emotions and how they are understood to function. The technical and practical information is about different methods of measuring emotional states and the subsequent task of sonifying this information. While this may sound like a relatively new idea, the sonification of bio-signals has a history in music that began around the 1970’s, with the premiere of Alvin Lucier’s Music for Solo Performer. After presenting both historical examples and modern incarnations of this idea, the discussion explores the aesthetics of such works and their place within a postmodern music scene

Abstraction of representation in live theater

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2009.Cataloged from PDF version of thesis.Includes bibliographical references (p. 151-158).Early in Tod Machover's opera Death and the Powers, the main character, Simon Powers, is subsumed into a technological environment of his own creation. The theatrical set comes alive in the form of robotic, visual, and sonic elements that allow the actor to extend his range and influence across the stage in unique and dynamic ways. The environment must compellingly assume the behavior and expression of the absent Simon. This thesis presents a new approach called Disembodied Performance that adapts ideas from affective psychology, cognitive science, and the theatrical tradition to create a framework for thinking about the translation of stage presence. An implementation of a system informed by this methodology is demonstrated. In order to distill the essence of this character, we recover performance parameters in real-time from physiological sensors, voice, and vision systems. This system allows the offstage actor to express emotion and interact with others onstage. The Disembodied Performance approach takes a new direction in augmented performance by employing a nonrepresentational abstraction of a human presence that fully translates a character into an environment. The technique and theory presented also have broad-reaching applications outside of theater for personal expression, telepresence, and storytelling.Peter Alexander Torpey.S.M

Past and Future of Multi-Mind Brain-Computer Interfaces

The great improvements in brain–computer interface (BCI) performance that are brought upon by merging brain activity from multiple users have made this a popular strategy that allows even for human augmentation. These multi-mind BCIs have contributed in changing the role of BCIs from assistive technologies for people with disabilities into tools for human enhancement. This chapter reviews the history of multi-mind BCIs that have their root in the hyperscanning technique; the collaborative and competitive approaches; and the different ways that exist to integrate the brain signals from multiple people and optimally form groups to maximize performance. The main applications of multi-mind BCIs, including control of external devices, entertainment, and decision making, are also surveyed and discussed, in order to help the reader understand what are the most promising avenues and find the gaps that are worthy of future exploration. The chapter also provides a step-by-step tutorial to the design and implementation of a multi-mind BCI, with theoretical guidelines and a sample application

Paradoxes of interactivity: perspectives for media theory, human-computer interaction, and artistic investigations

Current findings from anthropology, genetics, prehistory, cognitive and neuroscience indicate that human nature is grounded in a co-evolution of tool use, symbolic communication, social interaction and cultural transmission. Digital information technology has recently entered as a new tool in this co-evolution, and will probably have the strongest impact on shaping the human mind in the near future. A common effort from the humanities, the sciences, art and technology is necessary to understand this ongoing co- evolutionary process. Interactivity is a key for understanding the new relationships formed by humans with social robots as well as interactive environments and wearables underlying this process. Of special importance for understanding interactivity are human-computer and human-robot interaction, as well as media theory and New Media Art. "Paradoxes of Interactivity" brings together reflections on "interactivity" from different theoretical perspectives, the interplay of science and art, and recent technological developments for artistic applications, especially in the realm of sound

Paradoxes of Interactivity

Current findings from anthropology, genetics, prehistory, cognitive and neuroscience indicate that human nature is grounded in a co-evolution of tool use, symbolic communication, social interaction and cultural transmission. Digital information technology has recently entered as a new tool in this co-evolution, and will probably have the strongest impact on shaping the human mind in the near future. A common effort from the humanities, the sciences, art and technology is necessary to understand this ongoing co- evolutionary process. Interactivity is a key for understanding the new relationships formed by humans with social robots as well as interactive environments and wearables underlying this process. Of special importance for understanding interactivity are human-computer and human-robot interaction, as well as media theory and New Media Art. »Paradoxes of Interactivity« brings together reflections on »interactivity« from different theoretical perspectives, the interplay of science and art, and recent technological developments for artistic applications, especially in the realm of sound

Enheduanna – A Manifesto of Falling: first demonstration of a live brain-computer cinema performance with multi-brain BCI interaction for one performer and two audience members

The new commercial-grade Electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs) have led to a phenomenal development of applications across health, entertainment and the arts, while an increasing interest in multi-brain interaction has emerged. In the arts, there is already a number of works that involve the interaction of more than one participants with the use of EEG-based BCIs. However, the field of live brain-computer cinema and mixed-media performances is rather new, compared to installations and music performances that involve multi-brain BCIs. In this context, we present the particular challenges involved. We discuss Enheduanna – A Manifesto of Falling, the first demonstration of a live brain-computer cinema performance that enables the real-time brain-activity interaction of one performer and two audience members; and we take a cognitive perspective on the implementation of a new passive multi-brain EEG-based BCI system to realise our creative concept. This article also presents the preliminary results and future work