Points2Sound: From mono to binaural audio using 3D point cloud scenes

For immersive applications, the generation of binaural sound that matches the visual counterpart is crucial to bring meaningful experiences to people in a virtual environment. Recent works have shown the possibility to use neural networks for synthesizing binaural audio from mono audio using 2D visual information as guidance. Extending this approach by guiding the audio using 3D visual information and operating in the waveform domain may allow for a more accurate auralization of a virtual audio scene. In this paper, we present Points2Sound, a multi-modal deep learning model which generates a binaural version from mono audio using 3D point cloud scenes. Specifically, Points2Sound consists of a vision network with 3D sparse convolutions which extracts visual features from the point cloud scene to condition an audio network, which operates in the waveform domain, to synthesize the binaural version. Experimental results indicate that 3D visual information can successfully guide multi-modal deep learning models for the task of binaural synthesis. In addition, we investigate different loss functions and 3D point cloud attributes, showing that directly predicting the full binaural signal and using rgb-depth features increases the performance of our proposed model.Comment: Code, data, and listening examples: https://github.com/francesclluis/points2soun

The influence of the vocal tract on the attack transients in clarinet playing

When playing single-reed woodwind instruments, players can modulate the spectral content of the airflow in their vocal tract, upstream of the vibrating reed. In an empirical study with professional clarinettists (Np=11), blowing pressure and mouthpiece pressure were measured during the performance of Clarinet Concerto excerpts. By comparing mouth pressure and mouthpiece pressure signals in the time domain, a method to detect instances of vocal tract adjustments was established. Results showed that players tuned their vocal tract in both clarion and altissimo registers. Furthermore, the analysis revealed that vocal tract adjustments support shorter attack transients and help to avoid lower bore resonances

Direction Specific Ambisonics Source Separation with End-To-End Deep Learning

Ambisonics is a scene-based spatial audio format that has several useful features compared to object-based formats, such as efficient whole scene rotation and versatility. However, it does not provide direct access to the individual source signals, so that these have to be separated from the mixture when required. Typically, this is done with linear spherical harmonics (SH) beamforming. In this paper, we explore deep-learning-based source separation on static Ambisonics mixtures. In contrast to most source separation approaches, which separate a fixed number of sources of specific sound types, we focus on separating arbitrary sound from specific directions. Specifically, we propose three operating modes that combine a source separation neural network with SH beamforming: refinement, implicit, and mixed mode. We show that a neural network can implicitly associate conditioning directions with the spatial information contained in the Ambisonics scene to extract specific sources. We evaluate the performance of the three proposed approaches and compare them to SH beamforming on musical mixtures generated with the musdb18 dataset, as well as with mixtures generated with the FUSS dataset for universal source separation, under both anechoic and room conditions. Results show that the proposed approaches offer improved separation performance and spatial selectivity compared to conventional SH beamforming.Comment: To be published in Acta Acustica. Code and listening examples: https://github.com/francesclluis/direction-ambisonics-source-separatio

Who is Buckia? Ein neues Nachschlagewerk von Swissbryophytes ermöglicht es, taxonomische und nomenklatorische Änderungen der Moosarten zu verfolgen

Die zahlreichen Veränderungen der Namen und der taxonomischen Umschreibung der Moosarten sind eine grosse Herausforderung für Datenmelder und Datenverwalter. Um diese Veränderungen besser nachvollziehen zu können, wurde auf der Webseite von Swissbryophytes ein neues taxonomisches Nachschlagewerk aufgeschaltet, das ‚Who is Who der Taxa’. Es enthält alle aus der Schweiz bekannten Taxa, die seit 1984, dem Beginn der Datenbank von Swissbryophytes (früher NISM), in die nationale Checkliste aufgenommen wurden und weitere Synonyms dieser Taxa, die in der Literatur verwendet werden. Dieses neue Instrument ermöglicht es nachzuvollziehen, was zu welcher Zeit unter einem Taxonnamen verstanden wurde und in welcher Beziehung die verschiedenen Taxa zueinander stehen (www.swissbryophytes.ch -> ‘Moose’ -> ‘Namen’). Es trägt dazu bei, die Veränderungen transparenter zu machen und damit auch die Qualität gemeldeter Daten zu erhöhen

Read/Write Digital Libraries for Musicology

The Web and other digital technologies have democratised music creation, reception, and analysis, putting music in the hands, ears, and minds of billions of users. Music digital libraries typically focus on an essential subset of this deluge—commercial and academic publications, and historical materials—but neglect to incorporate contributions by scholars, performers, and enthusiasts, such as annotations or performed interpretations of these artifacts, despite their potential utility for many types of users. In this paper we consider means by which digital libraries for musicology may incorporate such contributions into their collections, adhering to principles of FAIR data management and respecting contributor rights as outlined in the EU’s General Data Protection Regulation. We present an overview of centralised and decentralised approaches to this problem, and propose hybrid solutions in which contributions reside in a) user-controlled personal online datastores, b) decentralised file storage, and c) are published and aggregated into digital library collections. We outline the implementation of these ideas using Solid, a Web decentralisation project building on W3C standard technologies to facilitate publication and control over Linked Data. We demonstrate the feasibility of this approach by implementing prototypes supporting two types of contribution: Web Annotations describing or analysing musical elements in score encodings and music recordings; and, music performances and associated metadata supporting performance analyses across many renditions of a given piece. Finally, we situate these ideas within a wider conception of enriched, decentralised, and interconnected online music repositories

Automated Additive Construction (AAC) for Earth and Space Using In-situ Resources

Using Automated Additive Construction (AAC), low-fidelity large-scale compressive structures can be produced out of a wide variety of materials found in the environment. Compressionintensive structures need not utilize materials that have tight specifications for internal force management, meaning that the production of the building materials do not require costly methods for their preparation. Where a certain degree of surface roughness can be tolerated, lower-fidelity numerical control of deposited materials can provide a low-cost means for automating building processes, which can be utilized in remote or extreme environments on Earth or in Space. For space missions where every kilogram of mass must be lifted out of Earth’s gravity well, the promise of using in-situ materials for the construction of outposts, facilities, and installations could prove to be enabling if significant reduction of payload mass can be achieved. In a 2015 workshop sponsored by the Keck nstitute for Space Studies, on the topic of Three Dimensional (3D) Additive Construction For Space Using In-situ Resources, was conducted with additive construction experts from around the globe in attendance. The workshop explored disparate efforts, methods, and technologies and established a proposed framework for the field of Additive Construction Using In-situ Resources. This paper defines the field of Automated Additive Construction Using In-situ Resources, describes the state-of-the-art for various methods, establishes a vision for future efforts, identifies gaps in current technologies, explores investment opportunities, and proposes potential technology demonstration missions for terrestrial, International Space Station (ISS), lunar, deep space zero-gravity, and Mars environments

Seeking the Holy Grail: robust chronologies from archaeology and radiocarbon dating combined

The strengths of formal Bayesian chronological modelling are restated, combining as it does knowledge of the archaeology with the radiocarbon dating of carefully chosen samples of known taphonomy in association with diagnostic material culture. The risks of dating bone samples are reviewed, along with a brief history of the development of approaches to the radiocarbon dating of bone. In reply to Strien (2017), selected topics concerned with the emergence and aftermath of the LBK are discussed, as well as the early Vin≠a, Ra∫i∏te and Hinkelstein sequences. The need for rigour in an approach which combines archaeology and radiocarbon dating is underlined

Insights into the Membrane Interactions of the Saposin-Like Proteins Na-SLP-1 and Ac-SLP-1 from Human and Dog Hookworm

Saposin-like proteins (SAPLIPs) from soil-transmitted helminths play pivotal roles in host-pathogen interactions and have a high potential as targets for vaccination against parasitic diseases. We have identified two non-orthologous SAPLIPs from human and dog hookworm, Na-SLP-1 and Ac-SLP-1, and solved their three-dimensional crystal structures. Both proteins share the property of membrane binding as monitored by liposome co-pelleting assays and monolayer adsorption. Neither SAPLIP displayed any significant haemolytic or bactericidal activity. Based on the structural information, as well as the results from monolayer adsorption, we propose models of membrane interactions for both SAPLIPs. Initial membrane contact of the monomeric Na-SLP-1 is most likely by electrostatic interactions between the membrane surface and a prominent basic surface patch. In case of the dimeric Ac-SLP-1, membrane interactions are most likely initiated by a unique tryptophan residue that has previously been implicated in membrane interactions in other SAPLIPs