Music Information Retrieval in Live Coding: A Theoretical Framework

The work presented in this article has been partly conducted while the first author was at Georgia Tech from 2015–2017 with the support of the School of Music, the Center for Music Technology and Women in Music Tech at Georgia Tech. Another part of this research has been conducted while the first author was at Queen Mary University of London from 2017–2019 with the support of the AudioCommons project, funded by the European Commission through the Horizon 2020 programme, research and innovation grant 688382. The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link.Music information retrieval (MIR) has a great potential in musical live coding because it can help the musician–programmer to make musical decisions based on audio content analysis and explore new sonorities by means of MIR techniques. The use of real-time MIR techniques can be computationally demanding and thus they have been rarely used in live coding; when they have been used, it has been with a focus on low-level feature extraction. This article surveys and discusses the potential of MIR applied to live coding at a higher musical level. We propose a conceptual framework of three categories: (1) audio repurposing, (2) audio rewiring, and (3) audio remixing. We explored the three categories in live performance through an application programming interface library written in SuperCollider, MIRLC. We found that it is still a technical challenge to use high-level features in real time, yet using rhythmic and tonal properties (midlevel features) in combination with text-based information (e.g., tags) helps to achieve a closer perceptual level centered on pitch and rhythm when using MIR in live coding. We discuss challenges and future directions of utilizing MIR approaches in the computer music field

Real-time audiovisual and interactive applications for desktop and mobile platforms

Tese de mestrado integrado. Engenharia Informática e Computação. Universidade do Porto. Faculdade de Engenharia. 201

LC: A Mostly-strongly-timed Prototype-based Computer Music Programming Language that Integrates Objects and Manipulations for Microsound Synthesis

Ph.DDOCTOR OF PHILOSOPH

Investigating the cognitive foundations of collaborative musical free improvisation: Experimental case studies using a novel application of the subsumption architecture

This thesis investigates the cognitive foundations of collaborative musical free improvisation. To explore the cognitive underpinnings of the collaborative process, a series of experimental case studies was undertaken in which expert improvisors performed with an artificial agent. The research connects ecological musicology and subsumption robotics, and builds upon insights from empirical psychology pertaining to the attribution of intentionality. A distinguishing characteristic of free improvisation is that no over-arching framework of formal musical conventions defines it, and it cannot be positively identified by sound alone, which poses difficulties for traditional musicology. Current musicological research has begun to focus on the social dimension of music, including improvisation. Ecological psychology, which focuses on the relation of cognition to agent–environment dynamics using the notion of affordances, has been shown to be a promising approach to understanding musical improvisation. This ecological approach to musicology makes it possible to address the subjective and social aspects of improvised music, as opposed to the common treatment of music as objective and neutral. The subjective dimension of musical listening has been highlighted in music cognition studies of cue abstraction, whereby listeners perceive emergent structures while listening to certain forms of music when no structures are identified in advance. These considerations informed the design of the artificial agent, Odessa, used for this study. In contrast to traditional artificial intelligence (AI), which tends to view the world as objective and neutral, behaviour-based robotics historically developed around ideas similar to those of ecological psychology, focused on agent–environment dynamics and the ability to deal with potentially rapidly changing environments. Behaviour-based systems that are designed using the subsumption architecture are robust and flexible in virtue of their modular, decentralised design comprised of simple interactions between simple mechanisms. The competence of such agents is demonstrated on the basis of their interaction with the environment and ability to cope with unknown and dynamic conditions, which suggests the concept of improvisation. This thesis documents a parsimonious subsumption design for an agent that performs musical free improvisation with human co-performers, as well as the experimental studies conducted with this agent. The empirical component examines the human experience of collaborating with the agent and, more generally, the cognitive psychology of collaborative improvisation. The design was ultimately successful, and yielded insights about cognition in collaborative improvisation, in particular, concerning the central relationship between perceived intentionality and affordances. As a novel application of the subsumption architecture, this research contributes to AI/robotics and to research on interactive improvisation systems. It also contributes to music psychology and cognition, as well as improvisation studies, through its empirical grounding of an ecological model of musical interaction

Seventh Annual Workshop on Space Operations Applications and Research (SOAR 1993), volume 2

This document contains papers presented at the Space Operations, Applications and Research Symposium (SOAR) Symposium hosted by NASA/Johnson Space Center (JSC) and cosponsored by NASA/JSC and U.S. Air Force Materiel Command. SOAR included NASA and USAF programmatic overviews, plenary session, panel discussions, panel sessions, and exhibits. It invited technical papers in support of U.S. Army, U.S. Navy, Department of Energy, NASA, and USAF programs in the following areas: robotics and telepresence, automation and intelligent systems, human factors, life support, and space maintenance and servicing. SOAR was concerned with Government-sponsored research and development relevant to aerospace operations

Soundscape Generation Using Web Audio Archives

Os grandes e crescentes acervos de áudio na web têm transformado a prática do design de som. Neste contexto, sampling -- uma ferramenta essencial do design de som -- mudou de gravações mecânicas para os domínios da cópia e reprodução no computador. A navegação eficaz nos grandes acervos e a recuperação de conteúdo tornaram-se um problema bem identificado em Music Information Retrieval, nomeadamente através da adoção de metodologias baseadas no conteúdo do áudio.Apesar da sua robustez e eficácia, as soluções tecnológicas atuais assentam principalmente em métodos (estatísticos) de processamento de sinal, cuja terminologia atinge um nível de adequação centrada no utilizador.Esta dissertação avança uma nova estratégia orientada semanticamente para navegação e recuperação de conteúdo de áudio, em particular, sons ambientais, a partir de grandes acervos de áudio na web. Por fim, pretendemos simplificar a extração de pedidos definidos pelo utilizador para promover uma geração fluida de paisagens sonoras. No nosso trabalho, os pedidos aos acervos de áudio na web são feitos por dimensões afetivas que se relacionam com estados emocionais (exemplo: baixa ativação e baixa valência) e descrições semânticas das fontes de áudio (exemplo: chuva). Para tal, mapeamos as anotações humanas das dimensões afetivas para descrições espectrais de áudio extraídas do conteúdo do sinal. A extração de novos sons dos acervos da web é feita estipulando um pedido que combina um ponto num plano afetivo bidimensional e tags semânticas. A aplicação protótipo, MScaper, implementa o método no ambiente Ableton Live. A avaliação da nossa pesquisa avaliou a confiabilidade perceptual dos descritores espectrais de áudio na captura de dimensões afetivas e a usabilidade da MScaper. Os resultados mostram que as características espectrais do áudio capturam significativamente as dimensões afetivas e que o MScaper foi entendido pelos os utilizadores experientes como tendo excelente usabilidade.The large and growing archives of audio content on the web have been transforming the sound design practice. In this context, sampling -- a fundamental sound design tool -- has shifted from mechanical recording to the realms of the copying and cutting on the computer. To effectively browse these large archives and retrieve content became a well-identified problem in Music Information Retrieval, namely through the adoption of audio content-based methodologies. Despite its robustness and effectiveness, current technological solutions rely mostly on (statistical) signal processing methods, whose terminology do attain a level of user-centered explanatory adequacy.This dissertation advances a novel semantically-oriented strategy for browsing and retrieving audio content, in particular, environmental sounds, from large web audio archives. Ultimately, we aim to streamline the retrieval of user-defined queries to foster a fluid generation of soundscapes. In our work, querying web audio archives is done by affective dimensions that relate to emotional states (e.g., low arousal and low valence) and semantic audio source descriptions (e.g., rain). To this end, we map human annotations of affective dimensions to spectral audio-content descriptions extracted from the signal content. Retrieving new sounds from web archives is then made by specifying a query which combines a point in a 2-dimensional affective plane and semantic tags. A prototype application, MScaper, implements the method in the Ableton Live environment. An evaluation of our research assesses the perceptual soundness of the spectral audio-content descriptors in capturing affective dimensions and the usability of MScaper. The results show that spectral audio features significantly capture affective dimensions and that MScaper has been perceived by expert-users as having excellent usability

NASA Tech Briefs, February 2005

Topics discussed include: Instrumentation for Sensitive Gas Measurements; Apparatus for Testing Flat Specimens of Thermal Insulation; Quadrupole Ion Mass Spectrometer for Masses of 2 to 50 Da; Miniature Laser Doppler Velocimeter for Measuring Wall Shear; Coherent Laser Instrument Would Measure Range and Velocity; Printed Microinductors for Flexible Substrates; Digital Receiver for Microwave Radiometry; Printed Antennas Made Reconfigurable by Use of MEMS Switches; Traffic-Light-Preemption Vehicle-Transponder Software Module; Intersection-Controller Software Module; Central-Monitor Software Module; Estimating Effects of Multipath Propagation on GPS Signals; Parallel Adaptive Mesh Refinement Library; Predicting Noise From Aircraft Turbine-Engine Combustors; Generating Animated Displays of Spacecraft Orbits; Diagnosis and Prognosis of Weapon Systems; Training Software in Artificial-Intelligence Computing Techniques; APGEN Version 5.0; Single-Command Approach and Instrument Placement by a Robot on a Target; Three-Dimensional Audio Client Library; Isogrid Membranes for Precise, Singly Curved Reflectors; Nickel-Tin Electrode Materials for Nonaqueous Li-Ion Cells; Photocatalytic Coats in Glass Drinking-Water Bottles; Fast Laser Shutters With Low Vibratory Disturbances; Series-Connected Buck Boost Regulators; Space Physics Data Facility Web Services; Split-Resonator, Integrated-Post Vibratory Microgyroscope; Blended Buffet-Load-Alleviation System for Fighter Airplane; Gifford-McMahon/Joule-Thomson Refrigerator Cools to 2.5 K; High-Temperature, High-Load-Capacity Radial Magnetic Bearing; Fabrication of Spherical Reflectors in Outer Space; Automated Rapid Prototyping of 3D Ceramic Parts; Tissue Engineering Using Transfected Growth-Factor Genes; Automation of Vapor-Diffusion Growth of Protein Crystals; Atom Skimmers and Atom Lasers Utilizing Them; Gears Based on Carbon Nanotubes; Patched Off-Axis Bending/Twisting Actuators for Thin Mirrors; and Improving Control in a Joule-Thomson Refrigerator

NASA Tech Briefs, October 2006

Topics covered include: Protein Sensors Based on Optical Ring Resonators; Phase Sensor for Aligning a Segmented Telescope Mirror; Control Software for Advanced Video Guidance Sensor; Generating Control Commands From Gestures Sensed by EMG; Multiple-Flat-Panel System Displays Multidimensional Data; 3D X-Ray Luggage-Screening System; Probe Station and Near-Field Scanner for Testing Antennas; Photodetector Arrays for Multicolor Visible/Infrared Imaging; Semiconductor Bolometers Give Background-Limited Performance; Multichannel X-Band Dielectric-Resonator Oscillator; Automatic Alignment of Displacement-Measuring Interferometer; Earth Observing System Data Gateway; Power User Interface; Mercury Shopping Cart Interface; Cassini Archive Tracking System; Architecture Adaptive Computing Environment; Computing Fault Displacements from Surface Deformations; Oxygen-Permeable, Hydrophobic Membranes of Silanized alpha-Al2O3; SiC Composite Turbine Vanes; Retaining Device for the Interior Structure of a Spacecraft Payload; Tool for Torquing Circular Electrical-Connector Collars; System for Continuous Deaeration of Hydraulic Oil; Solar-Powered Cooler and Heater for an Automobile Interior; Improved Oxygen-Beam Texturing of Glucose-Monitoring Optics; Tool for Two Types of Friction Stir Welding; Stationary Apparatus Would Apply Forces of Walking to Feet; Instrument Would Detect and Collect Biological Aerosols; Boundary Condition for Modeling Semiconductor Nanostructures; Miniature Distillation Column for Producing LOX From Air; Even Illumination from Fiber-Optic-Coupled Laser Diodes; Optically Driven Deformable Mirrors; Algorithm for Automated Detection of Edges of Clouds; Exploiting Quantum Resonance to Solve Combinatorial Problems; Hybrid Terrain Database; On Release of Microbe-Laden Particles from Mars Landers; A Concept for Run-Time Support of the Chapel Language; Thermoelectric Inhomogeneities in (Ag(sub 1-y)SbTe2)(sub x)(PbTe)(sub 1-x); and Spacecraft Escape Capsule