I am Streaming in a Room

Internet Acoustics is the study of sound traveling through the Internet, treating it as an acoustical medium just like air or water. Real-time streaming of sound, something commonplace nowadays, can be exploited for its own “physics” of propagation. In a digitally-connected telecommunication world, rooms of the kind which will be described enclose remotely collaborating musicians in their own reverberated sound. The ambience which results is the product of an acoustical loop which creates room-like resonances created between internet endpoints which recirculate sound echoes on the paths between them. These are synthesized acoustical spaces engineered to resemble actual rooms and distinct from other kinds of online rooms where “room” is used metaphorically for gatherings of users participating in teleconference or chat applications. The present article describes room-like internet reverberation for local area and wide area networking, respectively named LAIR and WAIR. Aspects of the medium, algorithms used and initial musical experiments are detailed. To support these topics, the article also presents a theory of operation for jacktrip, the low-latency internet streaming software which was modified for the project

FAUST2SMARTKEYB: A TOOL TO MAKE MOBILE INSTRUMENTS FOCUSING ON SKILLS TRANSFER IN THE FAUST PROGRAMMING LANGUAGE

International audienceIn this paper, we present faust2smartkeyb, a tool to create musical apps for Android and iOS using the FAUST programming language. The use of musical instrument physical models in this context through the FAUST Physical Modeling Library is emphasized. We also demonstrate how this system allows for the design of interfaces facilitating skills transfer from existing musical instruments

Experiencing Remote Classical Music Performance Over Long Distance: A JackTrip Concert Between Two Continents During the Pandemic

The recent lockdown restrictions imposed by the severe acute respiratory syndrome coronavirus 2 pandemic have heightened the need for new forms of remote collaboration for music schools, conservatories, musician ensembles, and artists, each of which would benefit from being provided with adequate tools to make high-quality, live collaborative music in a distributed fashion. This paper demonstrates the usage of the Networked Music Performance software JackTrip to support a distributed classical concert involving singers and musicians from four different locations in two continents, using readily available hardware/software solutions and internet connections while guaranteeing high-fidelity audio quality. This paper provides a description of the technical setup with a numerical analysis of the achieved mouth-to-ear latency and assessment of the music-making experience as perceived by the performers

A Deep Learning Approach for Low-Latency Packet Loss Concealment of Audio Signals in Networked Music Performance Applications

Networked Music Performance (NMP) is envisioned as a potential game changer among Internet applications: it aims at revolutionizing the traditional concept of musical interaction by enabling remote musicians to interact and perform together through a telecommunication network. Ensuring realistic conditions for music performance, however, constitutes a significant engineering challenge due to extremely strict requirements in terms of audio quality and, most importantly, network delay. To minimize the end-to-end delay experienced by the musicians, typical implementations of NMP applications use un-compressed, bidirectional audio streams and leverage UDP as transport protocol. Being connection less and unreliable,audio packets transmitted via UDP which become lost in transit are not re-transmitted and thus cause glitches in the receiver audio playout. This article describes a technique for predicting lost packet content in real-time using a deep learning approach. The ability of concealing errors in real time can help mitigate audio impairments caused by packet losses, thus improving the quality of audio playout in real-world scenarios.Comment: 8 pages, 2 figure

Web-Based Networked Music Performances via WebRTC: A Low-Latency PCM Audio Solution

Nowadays, widely used videoconferencing software has been diffused even further by the social distancing measures adopted during the SARS-CoV-2 pandemic. However, none of the Web-based solutions currently available support high-fidelity stereo audio streaming, which is a fundamental prerequisite for networked music applications. This is mainly because of the fact that the WebRTC RTCPeerConnection standard or Web-based audio streaming do not handle uncompressed audio formats. To overcome that limitation, an implementation of 16-bit pulse code modulation (PCM) stereo audio transmission on top of the WebRTC RTCDataChannel, leveraging Web Audio and AudioWorklets, is discussed. Results obtained with multiple configurations, browsers, and operating systems showthat the proposed approach outperforms theWebRTC RTCPeerConnection standard in terms of audio quality and latency, which in the authors' best case to date has been reduced to only 40 ms between twoMacBooks on a local area network

THE FAUST PHYSICAL MODELING LIBRARY: A MODULAR PLAYGROUND FOR THE DIGITAL LUTHIER

International audienceThis paper introduces the FAUST Physical Modeling Library, an environment to create physical models of musical instruments in a modular way in the FAUST programming language. Low and high level elements can be combined to implement existing or completely novel instruments. Various examples of physical models are provided. The combined use of mesh2faust, a tool to generate FAUST physical models from 3D drawings, and of the FAUST Physical Modeling Library is also demonstrated through the implementation of a marimba physical model

faust2api: a Comprehensive API Generator for Android and iOS

International audienceWe introduce faust2api, a tool to generate custom DSP engines for Android and iOS using the Faust programming language. Faust DSP ob jects can easily be turned into MIDI-controllable polyphonic synthesizers or audio effects with built-in sensors support, etc. The various elements of the DSP engine can be accessed through a high-level API, made uniform across platforms and languages. This paper provides technical details on the implementation of this system as well as an evaluation of its various features

Nuance: Adding Multi-touch Force Detection to the iPad

(Abstract to follow

Mending Bells and Closing Belfries with Faust

Finite Element Analyses (FEA) was used to predict the resonant modes of the Tsar Kolokol, a 200-ton fractured bell that sits outside the Kremlin in Moscow. Frequency and displacement data informed a physical model implemented in the Faust programming language (Functional Audio Stream). The authors hosted a concert for Tsar bell and carillon with the generous support of Meyer Sound and a University of Michigan bicentennial grant. In the concert, the simulated Tsar bell was triggered by the keyboard and perceptually fused with the bourdon of the Baird Carillon on the University of Michigan campus in Ann Arbor