On Mapping EEG Information into Music

With the rise of ever-more affordable EEG equipment available to musicians, artists and researchers, designing and building a Brain-Computer Music Interface (BCMI) system has recently become a realistic achievement. This chapter discusses previous research in the fields of mapping, sonification and musification in the context of designing a BCMI system and will be of particular interest to those who seek to develop their own. Design of a BCMI requires unique consider-ations due to the characteristics of the EEG as a human interface device (HID). This chapter analyses traditional strategies for mapping control from brain waves alongside previous research in bio-feedback musical systems. Advances in music technology have helped provide more complex approaches with regards to how music can be affected and controlled by brainwaves. This, paralleled with devel-opments in our understanding of brainwave activity has helped push brain-computer music interfacing into innovative realms of real-time musical perfor-mance, composition and applications for music therapy

BCI for Music Making: Then, Now, and Next

Brain–computer music interfacing (BCMI) is a growing field with a history of experimental applications derived from the cutting edge of BCI research as adapted to music making and performance. BCMI offers some unique possibilities over traditional music making, including applications for emotional music selection and emotionally driven music creation for individuals as communicative aids (either in cases where users might have physical or mental disabilities that otherwise preclude them from taking part in music making or in music therapy cases where emotional communication between a therapist and a patient by means of traditional music making might otherwise be impossible). This chapter presents an overview of BCMI and its uses in such contexts, including existing techniques as they are adapted to musical control, from P300 and SSVEP (steady-state visually evoked potential) in EEG (electroencephalogram) to asymmetry, hybrid systems, and joint fMRI (functional magnetic resonance imaging) studies correlating affective induction (by means of music) with neurophysiological cues. Some suggestions for further work are also volunteered, including the development of collaborative platforms for music performance by means of BCMI

La tecnología como extensión de las facultades sensoriales en las artes escénicas

Programa de Doctorado en Humanidades por la Universidad Carlos III de MadridPresidente: Ricardo Iglesias García.- Secretario: Eduardo Blázquez Mateos.- Vocal: Victoria Pérez Arroy

NON-VERBAL COMMUNICATION WITH PHYSIOLOGICAL SENSORS. THE AESTHETIC DOMAIN OF WEARABLES AND NEURAL NETWORKS

Historically, communication implies the transfer of information between bodies, yet this phenomenon is constantly adapting to new technological and cultural standards. In a digital context, it’s commonplace to envision systems that revolve around verbal modalities. However, behavioural analysis grounded in psychology research calls attention to the emotional information disclosed by non-verbal social cues, in particular, actions that are involuntary. This notion has circulated heavily into various interdisciplinary computing research fields, from which multiple studies have arisen, correlating non-verbal activity to socio-affective inferences. These are often derived from some form of motion capture and other wearable sensors, measuring the ‘invisible’ bioelectrical changes that occur from inside the body. This thesis proposes a motivation and methodology for using physiological sensory data as an expressive resource for technology-mediated interactions. Initialised from a thorough discussion on state-of-the-art technologies and established design principles regarding this topic, then applied to a novel approach alongside a selection of practice works to compliment this. We advocate for aesthetic experience, experimenting with abstract representations. Atypically from prevailing Affective Computing systems, the intention is not to infer or classify emotion but rather to create new opportunities for rich gestural exchange, unconfined to the verbal domain. Given the preliminary proposition of non-representation, we justify a correspondence with modern Machine Learning and multimedia interaction strategies, applying an iterative, human-centred approach to improve personalisation without the compromising emotional potential of bodily gesture. Where related studies in the past have successfully provoked strong design concepts through innovative fabrications, these are typically limited to simple linear, one-to-one mappings and often neglect multi-user environments; we foresee a vast potential. In our use cases, we adopt neural network architectures to generate highly granular biofeedback from low-dimensional input data. We present the following proof-of-concepts: Breathing Correspondence, a wearable biofeedback system inspired by Somaesthetic design principles; Latent Steps, a real-time auto-encoder to represent bodily experiences from sensor data, designed for dance performance; and Anti-Social Distancing Ensemble, an installation for public space interventions, analysing physical distance to generate a collective soundscape. Key findings are extracted from the individual reports to formulate an extensive technical and theoretical framework around this topic. The projects first aim to embrace some alternative perspectives already established within Affective Computing research. From here, these concepts evolve deeper, bridging theories from contemporary creative and technical practices with the advancement of biomedical technologies.Historicamente, os processos de comunicação implicam a transferência de informação entre organismos, mas este fenómeno está constantemente a adaptar-se a novos padrões tecnológicos e culturais. Num contexto digital, é comum encontrar sistemas que giram em torno de modalidades verbais. Contudo, a análise comportamental fundamentada na investigação psicológica chama a atenção para a informação emocional revelada por sinais sociais não verbais, em particular, acções que são involuntárias. Esta noção circulou fortemente em vários campos interdisciplinares de investigação na área das ciências da computação, dos quais surgiram múltiplos estudos, correlacionando a actividade nãoverbal com inferências sócio-afectivas. Estes são frequentemente derivados de alguma forma de captura de movimento e sensores “wearable”, medindo as alterações bioeléctricas “invisíveis” que ocorrem no interior do corpo. Nesta tese, propomos uma motivação e metodologia para a utilização de dados sensoriais fisiológicos como um recurso expressivo para interacções mediadas pela tecnologia. Iniciada a partir de uma discussão aprofundada sobre tecnologias de ponta e princípios de concepção estabelecidos relativamente a este tópico, depois aplicada a uma nova abordagem, juntamente com uma selecção de trabalhos práticos, para complementar esta. Defendemos a experiência estética, experimentando com representações abstractas. Contrariamente aos sistemas de Computação Afectiva predominantes, a intenção não é inferir ou classificar a emoção, mas sim criar novas oportunidades para uma rica troca gestual, não confinada ao domínio verbal. Dada a proposta preliminar de não representação, justificamos uma correspondência com estratégias modernas de Machine Learning e interacção multimédia, aplicando uma abordagem iterativa e centrada no ser humano para melhorar a personalização sem o potencial emocional comprometedor do gesto corporal. Nos casos em que estudos anteriores demonstraram com sucesso conceitos de design fortes através de fabricações inovadoras, estes limitam-se tipicamente a simples mapeamentos lineares, um-para-um, e muitas vezes negligenciam ambientes multi-utilizadores; com este trabalho, prevemos um potencial alargado. Nos nossos casos de utilização, adoptamos arquitecturas de redes neurais para gerar biofeedback altamente granular a partir de dados de entrada de baixa dimensão. Apresentamos as seguintes provas de conceitos: Breathing Correspondence, um sistema de biofeedback wearable inspirado nos princípios de design somaestético; Latent Steps, um modelo autoencoder em tempo real para representar experiências corporais a partir de dados de sensores, concebido para desempenho de dança; e Anti-Social Distancing Ensemble, uma instalação para intervenções no espaço público, analisando a distância física para gerar uma paisagem sonora colectiva. Os principais resultados são extraídos dos relatórios individuais, para formular um quadro técnico e teórico alargado para expandir sobre este tópico. Os projectos têm como primeiro objectivo abraçar algumas perspectivas alternativas às que já estão estabelecidas no âmbito da investigação da Computação Afectiva. A partir daqui, estes conceitos evoluem mais profundamente, fazendo a ponte entre as teorias das práticas criativas e técnicas contemporâneas com o avanço das tecnologias biomédicas

Enmeshed 3

Enmeshed 3, for cello and live electronics, is the third in a series of works in which a solo instrument becomes ‘enmeshed’ in multiple layers of transformations derived from the live performance. The works are shaped and structured in terms of the varying relationships between these layers and the ‘distances’ between the original acoustic performance and the various transformations, in terms of pitch, time delay, timbre, texture and space. At certain points in the work these almost converge whilst at other times large distances open up, with the different layers in a wild counterpoint. All the sounds in the work derive from live transformation of the soloist's performance. The composer’s own granular synthesis algorithms play a significant role in these transformations. Multichannel spatialisation also plays an important part in terms of spatial positioning and movement, the creation of different virtual spatial environments and in the definition of different layers. It can be performed variously with between 8 and 24 channels. Enmeshed 3 is in five contrasting but inter-related sections centering around a long slow meditative central passage. It was written for Madeleine Shapiro who premiered it at the New York City Electroacoustic Music Festival in April 2013

Music and Digital Media: A planetary anthropology

Anthropology has neglected the study of music. Music and Digital Media shows how and why this should be redressed. It does so by enabling music to expand the horizons of digital anthropology, demonstrating how the field can build interdisciplinary links to music and sound studies, digital/media studies, and science and technology studies. Music and Digital Media is the first comparative ethnographic study of the impact of digital media on music worldwide. It offers a radical and lucid new theoretical framework for understanding digital media through music, showing that music is today where the promises and problems of the digital assume clamouring audibility. The book contains ten chapters, eight of which present comprehensive original ethnographies; they are bookended by an authoritative introduction and a comparative postlude. Five chapters address popular, folk, art and crossover musics in the global South and North, including Kenya, Argentina, India, Canada and the UK. Three chapters bring the digital experimentally to the fore, presenting pioneering ethnographies of an extra-legal peer-to-peer site and the streaming platform Spotify, a series of prominent internet-mediated music genres, and the first ethnography of a global software package, the interactive music platform Max. The book is unique in bringing ethnographic research on popular, folk, art and crossover musics from the global North and South into a comparative framework on a large scale, and creates an innovative new paradigm for comparative anthropology. It shows how music enlarges anthropology while demanding to be understood with reference to classic themes of anthropological theory

Music and Digital Media

Anthropology has neglected the study of music. Music and Digital Media shows how and why this should be redressed. It does so by enabling music to expand the horizons of digital anthropology, demonstrating how the field can build interdisciplinary links to music and sound studies, digital/media studies, and science and technology studies. Music and Digital Media is the first comparative ethnographic study of the impact of digital media on music worldwide. It offers a radical and lucid new theoretical framework for understanding digital media through music, showing that music is today where the promises and problems of the digital assume clamouring audibility. The book contains ten chapters, eight of which present comprehensive original ethnographies; they are bookended by an authoritative introduction and a comparative postlude. Five chapters address popular, folk, art and crossover musics in the global South and North, including Kenya, Argentina, India, Canada and the UK. Three chapters bring the digital experimentally to the fore, presenting pioneering ethnographies of anextra-legal peer-to-peer site and the streaming platform Spotify, a series of prominent internet-mediated music genres, and the first ethnography of a global software package, the interactive music platform Max. The book is unique in bringing ethnographic research on popular, folk, art and crossover musics from the global North and South into a comparative framework on a large scale, and creates an innovative new paradigm for comparative anthropology. It shows how music enlarges anthropology while demanding to be understood with reference to classic themes of anthropological theory. Praise for Music and Digital Media ‘Music and Digital Media is a groundbreaking update to our understandings of sound, media, digitization, and music. Truly transdisciplinary and transnational in scope, it innovates methodologically through new models for collaboration, multi-sited ethnography, and comparative work. It also offers an important defense of—and advancement of—theories of mediation.’ Jonathan Sterne, Communication Studies and Art History, McGill University 'Music and Digital Media is a nuanced exploration of the burgeoning digital music scene across both the global North and the global South. Ethnographically rich and theoretically sophisticated, this collection will become the new standard for this field.' Anna Tsing, Anthropology, University of California at Santa Cruz 'The global drama of music's digitisation elicits extreme responses – from catastrophe to piratical opportunism – but between them lie more nuanced perspectives. This timely, absolutely necessary collection applies anthropological understanding to a deliriously immersive field, bringing welcome clarity to complex processes whose impact is felt far beyond what we call music.' David Toop, London College of Communication, musician and writer ‘Spanning continents and academic disciplines, the rich ethnographies contained in Music and Digital Media makes it obligatory reading for anyone wishing to understand the complex, contradictory, and momentous effects that digitization is having on musical cultures.’ Eric Drott, Music, University of Texas, Austin ‘This superb collection, with an authoritative overview as its introduction, represents the state of the art in studies of the digitalisation of music. It is also a testament to what anthropology at its reflexive best can offer the rest of the social sciences and humanities.’ David Hesmondhalgh, Media and Communication, University of Leeds ‘This exciting volume forges new ground in the study of local conditions, institutions, and sounds of digital music in the Global South and North. The book’s planetary scope and its commitment to the “messiness” of ethnographic sites and concepts amplifies emergent configurations and meanings of music, the digital, and the aesthetic.’ Marina Peterson, Anthropology, University of Texas, Austi

Music and Digital Media

Anthropology has neglected the study of music. Music and Digital Media shows how and why this should be redressed. It does so by enabling music to expand the horizons of digital anthropology, demonstrating how the field can build interdisciplinary links to music and sound studies, digital/media studies, and science and technology studies. Music and Digital Media is the first comparative ethnographic study of the impact of digital media on music worldwide. It offers a radical and lucid new theoretical framework for understanding digital media through music, showing that music is today where the promises and problems of the digital assume clamouring audibility. The book contains ten chapters, eight of which present comprehensive original ethnographies; they are bookended by an authoritative introduction and a comparative postlude. Five chapters address popular, folk, art and crossover musics in the global South and North, including Kenya, Argentina, India, Canada and the UK. Three chapters bring the digital experimentally to the fore, presenting pioneering ethnographies of anextra-legal peer-to-peer site and the streaming platform Spotify, a series of prominent internet-mediated music genres, and the first ethnography of a global software package, the interactive music platform Max. The book is unique in bringing ethnographic research on popular, folk, art and crossover musics from the global North and South into a comparative framework on a large scale, and creates an innovative new paradigm for comparative anthropology. It shows how music enlarges anthropology while demanding to be understood with reference to classic themes of anthropological theory. Praise for Music and Digital Media ‘Music and Digital Media is a groundbreaking update to our understandings of sound, media, digitization, and music. Truly transdisciplinary and transnational in scope, it innovates methodologically through new models for collaboration, multi-sited ethnography, and comparative work. It also offers an important defense of—and advancement of—theories of mediation.’ Jonathan Sterne, Communication Studies and Art History, McGill University 'Music and Digital Media is a nuanced exploration of the burgeoning digital music scene across both the global North and the global South. Ethnographically rich and theoretically sophisticated, this collection will become the new standard for this field.' Anna Tsing, Anthropology, University of California at Santa Cruz 'The global drama of music's digitisation elicits extreme responses – from catastrophe to piratical opportunism – but between them lie more nuanced perspectives. This timely, absolutely necessary collection applies anthropological understanding to a deliriously immersive field, bringing welcome clarity to complex processes whose impact is felt far beyond what we call music.' David Toop, London College of Communication, musician and writer ‘Spanning continents and academic disciplines, the rich ethnographies contained in Music and Digital Media makes it obligatory reading for anyone wishing to understand the complex, contradictory, and momentous effects that digitization is having on musical cultures.’ Eric Drott, Music, University of Texas, Austin ‘This superb collection, with an authoritative overview as its introduction, represents the state of the art in studies of the digitalisation of music. It is also a testament to what anthropology at its reflexive best can offer the rest of the social sciences and humanities.’ David Hesmondhalgh, Media and Communication, University of Leeds ‘This exciting volume forges new ground in the study of local conditions, institutions, and sounds of digital music in the Global South and North. The book’s planetary scope and its commitment to the “messiness” of ethnographic sites and concepts amplifies emergent configurations and meanings of music, the digital, and the aesthetic.’ Marina Peterson, Anthropology, University of Texas, Austi