QuBits, an Interactive Virtual Reality Project and Compositional Space for Sound and Image

AbstractQuBits, an Interactive Virtual Reality Project and Compositional Spacefor Sound and ImagebyJonathan KulpaDoctor of Philosophy in Music CompositionUniversity of California, BerkeleyProfessor Edmund Campion, ChairThis paper describes the QuBits project, a virtual reality (VR) environment created by the composer, offering an expanded medium for musical experience with integrated space and visuals. The QuBits VR environment is the essential supporting material for the dissertation and this paper provides a full description of the project.The environment was designed to explore a musical aesthetic valuing sound mass, spatial sound, evolution, and algorithmically generated sonic structures. Additionally, the user of the VR system plays a key role in shaping these musical elements. The user first must discover what behaviors are possible by exploring and through chance encounters. They can then shape each discovered behavior with nuance if they choose. In the VR environment, each sound has a corresponding visual component. To achieve this, a system was built with two software platforms, one for digital sound processing and another for 3D graphics and algorithmic event generation. These platforms communicate via Open Sound Control (OSC). The sounds are a mix of real world sampled sound, granular synthesis, and real-time generated synthetic sound. The QuBits VR environment represents the results of this methodology. Pros and cons of the methodology are discussed, as well as implications for future projects

Gesture-Controlled Interaction with Aesthetic Information Sonification

Information representation in augmented and virtual reality systems, and social physical (building) spaces can enhance the efficacy of interacting with and assimilating abstract, non-visual data. Sanification is the process of automatically generated real time information representation. There is a gap in our implementation and knowledge of auditory display systems used to enhance interaction in virtual and augmented reality. This paper addresses that gap by examining methodologies for mapping socio-spatial data to spatialised sanification manipulated with gestural controllers. This is a system of interactive knowledge representation that completes the human integration loop, enabling the user to interact with and manipulate data using 3D spatial gesture and 3D auditory display. Benefits include 1) added immersion in an augmented or virtual reality interface; 2) auditory display avoids visual overload in visually-saturated processes such as designing, evacuation in emergencies, flying aircraft; computer gaming; and 3) bi-modal or auditory representation, due to its time-based character, facilitates cognition of complex information

Music and Digital Media

Anthropology has neglected the study of music. Music and Digital Media shows how and why this should be redressed. It does so by enabling music to expand the horizons of digital anthropology, demonstrating how the field can build interdisciplinary links to music and sound studies, digital/media studies, and science and technology studies. Music and Digital Media is the first comparative ethnographic study of the impact of digital media on music worldwide. It offers a radical and lucid new theoretical framework for understanding digital media through music, showing that music is today where the promises and problems of the digital assume clamouring audibility. The book contains ten chapters, eight of which present comprehensive original ethnographies; they are bookended by an authoritative introduction and a comparative postlude. Five chapters address popular, folk, art and crossover musics in the global South and North, including Kenya, Argentina, India, Canada and the UK. Three chapters bring the digital experimentally to the fore, presenting pioneering ethnographies of anextra-legal peer-to-peer site and the streaming platform Spotify, a series of prominent internet-mediated music genres, and the first ethnography of a global software package, the interactive music platform Max. The book is unique in bringing ethnographic research on popular, folk, art and crossover musics from the global North and South into a comparative framework on a large scale, and creates an innovative new paradigm for comparative anthropology. It shows how music enlarges anthropology while demanding to be understood with reference to classic themes of anthropological theory. Praise for Music and Digital Media ‘Music and Digital Media is a groundbreaking update to our understandings of sound, media, digitization, and music. Truly transdisciplinary and transnational in scope, it innovates methodologically through new models for collaboration, multi-sited ethnography, and comparative work. It also offers an important defense of—and advancement of—theories of mediation.’ Jonathan Sterne, Communication Studies and Art History, McGill University 'Music and Digital Media is a nuanced exploration of the burgeoning digital music scene across both the global North and the global South. Ethnographically rich and theoretically sophisticated, this collection will become the new standard for this field.' Anna Tsing, Anthropology, University of California at Santa Cruz 'The global drama of music's digitisation elicits extreme responses – from catastrophe to piratical opportunism – but between them lie more nuanced perspectives. This timely, absolutely necessary collection applies anthropological understanding to a deliriously immersive field, bringing welcome clarity to complex processes whose impact is felt far beyond what we call music.' David Toop, London College of Communication, musician and writer ‘Spanning continents and academic disciplines, the rich ethnographies contained in Music and Digital Media makes it obligatory reading for anyone wishing to understand the complex, contradictory, and momentous effects that digitization is having on musical cultures.’ Eric Drott, Music, University of Texas, Austin ‘This superb collection, with an authoritative overview as its introduction, represents the state of the art in studies of the digitalisation of music. It is also a testament to what anthropology at its reflexive best can offer the rest of the social sciences and humanities.’ David Hesmondhalgh, Media and Communication, University of Leeds ‘This exciting volume forges new ground in the study of local conditions, institutions, and sounds of digital music in the Global South and North. The book’s planetary scope and its commitment to the “messiness” of ethnographic sites and concepts amplifies emergent configurations and meanings of music, the digital, and the aesthetic.’ Marina Peterson, Anthropology, University of Texas, Austi

Music and Digital Media: A planetary anthropology

Anthropology has neglected the study of music. Music and Digital Media shows how and why this should be redressed. It does so by enabling music to expand the horizons of digital anthropology, demonstrating how the field can build interdisciplinary links to music and sound studies, digital/media studies, and science and technology studies. Music and Digital Media is the first comparative ethnographic study of the impact of digital media on music worldwide. It offers a radical and lucid new theoretical framework for understanding digital media through music, showing that music is today where the promises and problems of the digital assume clamouring audibility. The book contains ten chapters, eight of which present comprehensive original ethnographies; they are bookended by an authoritative introduction and a comparative postlude. Five chapters address popular, folk, art and crossover musics in the global South and North, including Kenya, Argentina, India, Canada and the UK. Three chapters bring the digital experimentally to the fore, presenting pioneering ethnographies of an extra-legal peer-to-peer site and the streaming platform Spotify, a series of prominent internet-mediated music genres, and the first ethnography of a global software package, the interactive music platform Max. The book is unique in bringing ethnographic research on popular, folk, art and crossover musics from the global North and South into a comparative framework on a large scale, and creates an innovative new paradigm for comparative anthropology. It shows how music enlarges anthropology while demanding to be understood with reference to classic themes of anthropological theory

Music and Digital Media

Anthropology has neglected the study of music. Music and Digital Media shows how and why this should be redressed. It does so by enabling music to expand the horizons of digital anthropology, demonstrating how the field can build interdisciplinary links to music and sound studies, digital/media studies, and science and technology studies. Music and Digital Media is the first comparative ethnographic study of the impact of digital media on music worldwide. It offers a radical and lucid new theoretical framework for understanding digital media through music, showing that music is today where the promises and problems of the digital assume clamouring audibility. The book contains ten chapters, eight of which present comprehensive original ethnographies; they are bookended by an authoritative introduction and a comparative postlude. Five chapters address popular, folk, art and crossover musics in the global South and North, including Kenya, Argentina, India, Canada and the UK. Three chapters bring the digital experimentally to the fore, presenting pioneering ethnographies of anextra-legal peer-to-peer site and the streaming platform Spotify, a series of prominent internet-mediated music genres, and the first ethnography of a global software package, the interactive music platform Max. The book is unique in bringing ethnographic research on popular, folk, art and crossover musics from the global North and South into a comparative framework on a large scale, and creates an innovative new paradigm for comparative anthropology. It shows how music enlarges anthropology while demanding to be understood with reference to classic themes of anthropological theory. Praise for Music and Digital Media ‘Music and Digital Media is a groundbreaking update to our understandings of sound, media, digitization, and music. Truly transdisciplinary and transnational in scope, it innovates methodologically through new models for collaboration, multi-sited ethnography, and comparative work. It also offers an important defense of—and advancement of—theories of mediation.’ Jonathan Sterne, Communication Studies and Art History, McGill University 'Music and Digital Media is a nuanced exploration of the burgeoning digital music scene across both the global North and the global South. Ethnographically rich and theoretically sophisticated, this collection will become the new standard for this field.' Anna Tsing, Anthropology, University of California at Santa Cruz 'The global drama of music's digitisation elicits extreme responses – from catastrophe to piratical opportunism – but between them lie more nuanced perspectives. This timely, absolutely necessary collection applies anthropological understanding to a deliriously immersive field, bringing welcome clarity to complex processes whose impact is felt far beyond what we call music.' David Toop, London College of Communication, musician and writer ‘Spanning continents and academic disciplines, the rich ethnographies contained in Music and Digital Media makes it obligatory reading for anyone wishing to understand the complex, contradictory, and momentous effects that digitization is having on musical cultures.’ Eric Drott, Music, University of Texas, Austin ‘This superb collection, with an authoritative overview as its introduction, represents the state of the art in studies of the digitalisation of music. It is also a testament to what anthropology at its reflexive best can offer the rest of the social sciences and humanities.’ David Hesmondhalgh, Media and Communication, University of Leeds ‘This exciting volume forges new ground in the study of local conditions, institutions, and sounds of digital music in the Global South and North. The book’s planetary scope and its commitment to the “messiness” of ethnographic sites and concepts amplifies emergent configurations and meanings of music, the digital, and the aesthetic.’ Marina Peterson, Anthropology, University of Texas, Austi

Machine perception of natural musical conducting gestures

Thesis (M.S.)--Massachusetts Institute of Technology, Program in Media Arts & Sciences, 1996.Includes bibliographical references (p. 53-54).by Matthew Wayne Krom.M.S

Development of a 3D audio panning and realtime visualisation toolset using emerging technologies

This thesis documents a body of research that links the field of electro-acoustic diffusion and spatialisation with practice in the music and film post-production industries. Three research questions are posed: "How can the physical user-interfaces used for panning by the music and film post-production industries offer creative alternatives to the fader-based hardware approach commonly used for electro-acoustic performance?" "How can a Digital Audio Workstation (DAW) be used as an alternative to dedicated panning hardware? "How can emerging programming technologies offer creative alternatives to the MAX/MSP or hardware-based tools commonly used for sound spatialisation?" This practice-based PhD addresses these questions by designing, developing and testing a set of hardware and software tools, the Requirement Specification for this `Toolset' results from literature review and critical analysis of current systems to determine potential research gaps. This analysis is followed by the selection of a suitable methodology for development and testing that allows the research questions to be explored effectively and results in the following Toolset: OctoPanner: Amulti-featured eight-channel 3D touchscreen panner application for Apple's Mac OS X controlling a DAW hosted customisable VST 3D panning plug-in with C++ source code. ShapePanner: A synchronisable shape-based sequencer application for Mac OS X inspired by Experimentalstudio's Halaphon. The user is ab{e to describe the movement of sounds in a 3D space using shape primitives such as lines and circles and thus extend the capabilities of the Toolset beyond realtime manual manipulation of sounds. 3DMIDIVisualiser: An application to allow the user to work without access to a multi-speaker system by enabling the movement of sounds to be viewed within a virtual room. Foot Puck: Afoot-controlled panning controller enabling a musician to spatialise their instrument using foot movement. Initial prototyping was achieved using Cycling `74's Max/MSP but the final applications are written using Apple's Cocoa environment and Objective C. This thesis gives close analysis and discussion of the various stages of research carried out; including the use of Apple's CoreMlDl and CoreAudio Clock OS X Core Services in a Cocoa application.EThOS - Electronic Theses Online ServiceGBUnited Kingdo