The Body as Musical Instrument

This chapter explores the possibility of thinking of the human body as musical instrument. It builds on the philosophy of phenomenology to discuss body schemata that might be considered “instrumental” and discusses the diversity of bodies proposed by body theory to consider the incorporation of digital technology. Concepts of embodied interaction from the scientific field of human–computer interaction are discussed with an eye toward musical application. The history of gestural musical instruments is presented, from the Theremin to instruments from the STEIM studio. The text then focuses on the use of physiological signals to create music, from historical works of Lucier and Rosenboom to recent performances by the authors. The body as musical instrument is discussed in a dynamic of coadaptation between performer and instrument in different configurations of body and technology

Biosignal-driven Art: Beyond biofeedback

Biosignal monitoring in interactive arts, although present for over forty years, remains a relatively little known field of research within the artistic community as compared to other sensing technologies. Since the early 1960s, an ever-increasing number of artists have collaborated with neuroscientists, physicians and electrical engineers, in order to devise means that allow for the acquisition of the minuscule electrical potentials generated by the human body. This has enabled direct manifestations of human physiology to be incorporated into interactive artworks. This paper presents an introduction to this field of artistic practice and scientific research that uses human physiology as its main element. A brief introduction to the main concepts and history of biosignal-driven art is followed by a review of various artworks and scientific enquiry developed by the authors. This aims at giving a complete overview of the various strategies developed for biosignal-driven interactive art

BioMuse to Bondage: Corporeal Interaction in Performance and Exhibition

This chapter looks at technologies of biosensing and techniques of sonification that invoke physicalization of sound to create a sense of intimacy in musical performance and gallery exhibition. By comparing the dynamic across stage and gallery settings, we will see how the corporeal activation of sound can become a key not just to decode experimental performance, but as an entry point to possible intimate spaces created by digital interaction. We will draw upon notions of physicalization, enaction, and embodiment to describe how a visceral, nearly carnal situation can be created in technology mediated work

Inside the conductor's jacket : analysis, interpretation and musical synthesis of expressive gesture

Thesis (Ph.D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2000.Includes bibliographical references (leaves 154-167).We present the design and implementation of the Conductor's Jacket, a unique wearable device that measures physiological and gestural signals, together with the Gesture Construction, a musical software system that interprets these signals and applies them expressively in a musical context. Sixteen sensors have been incorporated into the Conductor's Jacket in such a way as to not encumber or interfere with the gestures of a working orchestra conductor. The Conductor's Jacket system gathers up to sixteen data channels reliably at rates of 3 kHz per channel, and also provides mcal-time graphical feedback. Unlike many gesture-sensing systems it not only gathers positional and accelerational data but also senses muscle tension from several locations on each arm. The Conductor's Jacket was used to gather conducting data from six subjects, three professional conductors and three students, during twelve hours of rehearsals and performances. Analyses of the data yielded thirty-five significant features that seem to reflect intuitive and natural gestural tendencies, including context-based hand switching, anticipatory 'flatlining' effects, and correlations between respiration and phrasing. The results indicate that muscle tension and respiration signals reflect several significant and expressive characteristics of a conductor's gestures. From these results we present nine hypotheses about human musical expression, including ideas about efficiency, intentionality, polyphony, signal-to-noise ratios, and musical flow state. Finally, this thesis describes the Gesture Construction, a musical software system that analyzes and performs music in real-time based on the performer's gestures and breathing signals. A bank of software filters extracts several of the features that were found in the conductor study, including beat intensities and the alternation between arms. These features are then used to generate real-time expressive effects by shaping the beats, tempos, articulations, dynamics, and note lengths in a musical score.by Teresa Marrin Nakra.Ph.D

Sonified freaks and sounding prostheses: sonic representation of bodies in performance art

This study is concerned with the role of sound in the presentation and representation of bodies in performance art that incorporates digital technologies. It consists of a written thesis accompanied by a portfolio with documentation of original artwork. Since the 1960s, performance artists have explored the use of sensor technologies to register signals generated by the body and synthesize or control sound. However, both practical and theoretical approaches to biosignal sonification in this field have almost entirely focused on musical (formalist) perspectives, technological innovation, or heightening the performer’s and spectator’s awareness of their body’s physiology. Little attention has been paid to the usually conspicuous interaction between body and technological equipment and the role of the generated sound in the context of cultural critical debates regarding the performing body. The present study responds to this observation in two ways: Firstly, the written part of the study examines existing biosignal performance practices. It seeks to demonstrate that artists’ decisions on the design of sensor technology and sound synthesis or manipulation methods are often complicit in the representation of normative body types and behaviour. Drawing from a concept of the sonified body as a transgressive or ‘freak’ body, three critical perspectives on biosignal sonification in digital performance are proposed: A reading of body sonification methods from a gender-critical perspective, an inquiry in the context of Mikhail Bakhtin’s concepts of the grotesque and the classical body, and a conceptualization of the sonified body as a posthuman prosthetisized body. This part of the study serves as a framework for its second objective: the development of practical performance strategies to address and challenge cultural conventions concerning ‘the’ body’s form and role in society. This aspect of the thesis is developed in conjunction with, and further explored in, the artwork documented in the portfolio. The practical part of the study consists of three digital performance works. ELECTRODE (2011) involves an anal electrode that registers the activity of my sphincter muscle and uses this data to synthesize sound. For this work, I modified a commercially available muscle tension sensor device designed for people with faecal incontinence problems. Feedback (2010) encompasses components of a commercially available fetal Doppler sensor intended to listen to the heartbeat of unborn babies. SUIT (2009-2010) encompasses several performances that feature a PVC overall equipped with a loudspeaker, sensor interface and Doppler and humidity sensors

Configuring Corporeality: Performing bodies, vibrations and new musical instruments.

How to define the relationship of human bodies, sound and technological instruments in musical performance? This enquiry investigates the issue through an iterative mode of research. Aesthetic and technical insights on sound and body art performance with new musical instruments combine with analytical views on technological embodiment in philosophy and cultural studies. The focus is on corporeality: the physiological, phenomenological and cultural basis of embodied practices. The thesis proposes configuration as an analytical device and a blueprint for artistic creation. Configuration defines the relationship of the human being and technology as one where they affect each other's properties through a continuous, situated negotiation. In musical performance, this involves a performer's intuition, cognition, and sensorimotor skills, an instrument's material, musical and computational properties, and sound's vibrational and auditive qualities. Two particular kinds of configuration feature in this enquiry. One arises from an experiment on the effect of vibration on the sensorimotor system and is fully developed through a subsequent installation for one visitor at a time. The other emerges from a scientific study of gesture expressivity through muscle physiological sensing and is consolidated into an ensuing body art performance for sound and light. Both artworks rely upon intensely intimate sensorial and physical experiences, uses and abuses of the performer's body and bioacoustic sound feedback as a material force. This work contends that particular configurations in musical performance reinforce, alter or disrupt societal criteria against which human bodies and technologies are assessed. Its contributions are: the notion of configuration, which affords an understanding of human-machine co-dependence and its politics; two sound-based artworks, joining and expanding musical performance and body art; two experiments, and their hardware and software tools, providing insights on physiological computing methods for corporeal human-computer interaction

Robot Control Using Electromyography (EMG) Signals of the Wrist

The aim of this paper is to design a human–interface system, using EMG signals elicited by various wrist movements, to control a robot. EMG signals are normalized and based on joint torque. A three-layer neural network is used to estimate posture of the wrist and forearm from EMG signals. After training the neural network and obtaining appropriate weights, the subject was able to control the robot in real time using wrist and forearm movements

Embodied Musical Interaction

Music is a natural partner to human-computer interaction, offering tasks and use cases for novel forms of interaction. The richness of the relationship between a performer and their instrument in expressive musical performance can provide valuable insight to human-computer interaction (HCI) researchers interested in applying these forms of deep interaction to other fields. Despite the longstanding connection between music and HCI, it is not an automatic one, and its history arguably points to as many differences as it does overlaps. Music research and HCI research both encompass broad issues, and utilize a wide range of methods. In this chapter I discuss how the concept of embodied interaction can be one way to think about music interaction. I propose how the three “paradigms” of HCI and three design accounts from the interaction design literature can serve as a lens through which to consider types of music HCI. I use this conceptual framework to discuss three different musical projects—Haptic Wave, Form Follows Sound, and BioMuse

Cinema Fabriqué : a gestural environment for realtime video performance

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2003.Includes bibliographical references (p. [107]-[108]).This thesis presents an environment that enables a single person to improvise video and audio programming in real time through gesture control. The goal of this system is to provide the means to compose and edit video stories for a live audience with an interface that is exposed and engaging to watch. Many of the software packages used today for realtime audio-visual performance were not built with this use in mind, and have been repurposed or modified with plug-ins to meet the performer's needs. Also, these applications are typically controlled by standard keyboard, mouse, or MIDI inputs, which were not designed for precise video control or live spectacle. As an alternative I built a system called Cinema Fabriqué which integrates video editing and effects software and hand gesture tracking methods into a single system for audio-visual performance.by Justin Manor.S.M

Unheard Sounds: A Manual for the Sonification of Emotional States in an Artistic Space

This paper explores the use of sonification technology within an artistic space, specifically a live musical performance setting. The current work should be thought of as a manual for performers, laying out the theoretical background for understanding the use of this technology as well as the technical components for successful incorporation. Most of the theoretical knowledge presented in this paper focuses on emotions and how they are understood to function. The technical and practical information is about different methods of measuring emotional states and the subsequent task of sonifying this information. While this may sound like a relatively new idea, the sonification of bio-signals has a history in music that began around the 1970’s, with the premiere of Alvin Lucier’s Music for Solo Performer. After presenting both historical examples and modern incarnations of this idea, the discussion explores the aesthetics of such works and their place within a postmodern music scene