Designing and Composing for Interdependent Collaborative Performance with Physics-Based Virtual Instruments

Interdependent collaboration is a system of live musical performance in which performers can directly manipulate each other’s musical outcomes. While most collaborative musical systems implement electronic communication channels between players that allow for parameter mappings, remote transmissions of actions and intentions, or exchanges of musical fragments, they interrupt the energy continuum between gesture and sound, breaking our cognitive representation of gesture to sound dynamics. Physics-based virtual instruments allow for acoustically and physically plausible behaviors that are related to (and can be extended beyond) our experience of the physical world. They inherently maintain and respect a representation of the gesture to sound energy continuum. This research explores the design and implementation of custom physics-based virtual instruments for realtime interdependent collaborative performance. It leverages the inherently physically plausible behaviors of physics-based models to create dynamic, nuanced, and expressive interconnections between performers. Design considerations, criteria, and frameworks are distilled from the literature in order to develop three new physics-based virtual instruments and associated compositions intended for dissemination and live performance by the electronic music and instrumental music communities. Conceptual, technical, and artistic details and challenges are described, and reflections and evaluations by the composer-designer and performers are documented

Map, Trigger, Score, Procedure: machine-listening paradigms in live-electronics

Since the advent of real-time computer music environments, composers have increasingly incorporated DSP analysis, synthesis, and processing algorithms in their creative practices. Those processes became part of interactive systems that use real-time computational tools in musical compositions that explore diverse techniques to generate, spatialize, and process instrumental/vocal sounds. Parallel to the development of these tools and the expansion of DSP methods, new techniques focused on sound/musical information extraction became part of the tools available for music composition. In this context, this article discusses the creative use of Machine Listening and Musical Information Retrieval techniques applied in the composition of live-electronics works. By pointing out some practical applications and creative approaches, we aim to circumscribe, in a general way, the strategies for employing Machine Listening and Music Information Retrieval techniques observed in a set of live-electronics pieces, categorizing four compositional approaches, namely: mapping, triggering, scoring, and procedural paradigms of application of machine listening techniques in the context of live-electronics music compositions

{3D} Morphable Face Models -- Past, Present and Future

In this paper, we provide a detailed survey of 3D Morphable Face Models over the 20 years since they were first proposed. The challenges in building and applying these models, namely capture, modeling, image formation, and image analysis, are still active research topics, and we review the state-of-the-art in each of these areas. We also look ahead, identifying unsolved challenges, proposing directions for future research and highlighting the broad range of current and future applications

Sensory Communication

Contains table of contents for Section 2, an introduction and reports on twelve research projects.National Institutes of Health Grant 5 R01 DC00117National Institutes of Health Contract 2 P01 DC00361National Institutes of Health Grant 5 R01 DC00126National Institutes of Health Grant R01-DC00270U.S. Air Force - Office of Scientific Research Contract AFOSR-90-0200National Institutes of Health Grant R29-DC00625U.S. Navy - Office of Naval Research Grant N00014-88-K-0604U.S. Navy - Office of Naval Research Grant N00014-91-J-1454U.S. Navy - Office of Naval Research Grant N00014-92-J-1814U.S. Navy - Naval Training Systems Center Contract N61339-93-M-1213U.S. Navy - Naval Training Systems Center Contract N61339-93-C-0055U.S. Navy - Naval Training Systems Center Contract N61339-93-C-0083U.S. Navy - Office of Naval Research Grant N00014-92-J-4005U.S. Navy - Office of Naval Research Grant N00014-93-1-119

Proceedings of the Linux Audio Conference 2018

These proceedings contain all papers presented at the Linux Audio Conference 2018. The conference took place at c-base, Berlin, from June 7th - 10th, 2018 and was organized in cooperation with the Electronic Music Studio at TU Berlin

Modelling the live-electronics in electroacoustic music using particle systems

Contemporary music is largely influenced by technology. Empowered by the current available tools and resources, composers have the possibility to not only compose with sounds, but also to compose the sounds themselves. Personal computers powered with intuitive and interactive audio applications and development tools allow the creation of a vast range of real-time manipulation of live instrumental input and also real-time generation of sound through synthesis techniques. Consequently, achieving a desired sonority and interaction between the electronic and acoustic sounds in real-time, deeply rely on the choice and technical implementation of the audio processes and logical structures that will perform the electronic part of the composition. Due to the artistic and technical complexity of the development and implementation of such a complex artistic work, a very common strategy historically adopted by composers is to develop the composition in collaboration with a technology expert, which in this context is known as a musical assistant. In this perspective, the work of the musical assistant can be considered as one of translating musical, artistic and aesthetic concepts into mathematical algorithms and audio processes. The work presented in this dissertation addresses the problem of choosing, combining and manipulating the audio processes and logical structures that take place on the liveelectronics (i.e the electronic part of a mixed music composition) of a contemporary electroacoustic music composition, by using particle systems to model and simulate the dynamic behaviors that reflect the conceptual and aesthetic principles envisaged by the composer for a determined musical piece. The presented research work initiates with a thorough identification and analysis of the agents, processes and structures that are present in the live-electronics system of a mixed music composition. From this analysis a logical formalization of a typical live-electronics system is proposed, and then adapted to integrate a particle-based modelling strategy. From the formalization, a theoretical and practical framework for developing and implementing live-electronics systems for mixed music compositions using particle systems is proposed. The framework is experimented and validated in the development of distinct mixed music compositions by distinct composers, in real professional context. From the analysis of the case studies and the logical formalization, and the feedback given by the composers, it is possible to conclude that the proposed particle systems modelling method proves to be effective in the task of assisting the conceptual translation of musical and aesthetic ideas into implementable audio processing software.A música contemporânea é amplamente influenciada pela tecnologia. Os recursos tecnológicos atualmente disponíveis permitem que os compositores criem com sons e ao mesmo tempo criem os sons em si próprios. Os atuais aplicativos e ferramentas de software focados no desenvolvimento, controle e manipulação de processamentos de áudio, permitem a elaboração de diversos tipos de tratamentos e sínteses de som com a capacidade de serem executados e manipulados em tempo real. Consequentemente, a escolha dos algoritmos de processamento de áudio e suas respectivas implementações técnicas em forma de software, são determinantes para que a sonoridade desejada seja atingida, e para que o resultado sonoro satisfaça os objetivos estéticos e conceituais da relação entre as fontes sonoras acústicas e os sons eletrônicos presentes em uma composição eletroacústica de caráter misto. Devido à complexidade artística e técnica do desenvolvimento e implementação do sistema de eletrônica em tempo real de uma composição eletroacústica mista, uma estratégia historicamente adotada por compositores é a de desenvolver a composição em colaboração com um especialista em tecnologia, que neste contexto é usualmente referido como assistente musical. Nesta perspectiva, o trabalho do assistente musical pode ser interpretado como o de traduzir conceitos musicais, artísticos e estéticos em algoritmos matemáticos e processamento de áudio. O trabalho apresentado nesta dissertação aborda a problemática da escolha, combinação e manipulação dos processamentos de áudio e estruturas lógicas presentes no sistema de eletrônica em tempo real de uma composição de música eletroacústica contemporânea, e propõem o uso de sistemas de partículas para modelar e simular os comportamentos dinâmicos e morfológicos que refletem os princípios conceituais e estéticos previstos pelo compositor para uma determinada composição. A parte inicial do trabalho apresentado consiste na identificação e análise detalhada dos agentes, estruturas e processos envolvidos na realização e execução do sistema de eletrônica em tempo real. A partir desta análise é proposta uma formalização lógica e genérica de um sistema de eletrônica em tempo real. Em seguida, esta formalização é modificada e adaptada para integrar uma estratégia de modelagem através de sistemas de partículas. Em sequencia da formalização lógica, um método teórico e prático para o desenvolvimento de sistemas de eletrônica em tempo real para composições de música mista é proposto. O teste e consequente validação do método se dá através de sua utilização na realização da eletrônica em tempo real para obras de diferentes compositores. A análise dos casos de estudo e da formalização lógica, e também o parecer e opinião dos compositores, permitem concluir que o método proposto é de fato eficaz na tarefa de auxiliar o processo de tradução dos conceitos musicais e estéticos propostos pelos compositores em forma de algoritmos e processamentos de som implementados em software