Theory and Practice of Modified Frequency Modulation Synthesis

The theory and applications of a variant of the well-known synthesis method of frequency modulation, modified frequency modulation (ModFM), is discussed. The technique addresses some of the shortcomings of classic FM and provides a more smoothly evolving spectrum with respect to variations in the modulation index. A complete description of the method is provided, discussing its characteristics and practical considerations of instrument design. A phase synchronous version of ModFM is presented and its applications on resonant and formant synthesis are explored. Extensions to the technique are introduced, providing means of changing spectral envelope symmetry. Finally its applications as an adaptive effect are discussed. Sound examples for the various applications of the technique are offered online

Nonlinear Distortion Synthesis Using the Split-Sideband Method, with Applications to Adaptive Signal Processing

Split-sideband synthesis (SpSB)—which is related to the well-known principles of waveshaping, single-sideband modulation, and frequency modulation—offers the possibility of creating four independent sideband outputs: upper, lower, odd, and even. Novel spectra and timbres can be created by the way in which these four outputs are combined. As with similar techniques for distortion synthesis, an SpSB process is controlled by the modulator and carrier frequencies as well as the modulation index. The technique can also be used as an adaptive effect applied to arbitrary monophonic signals. A number of sound samples illustrate the technique

Applications of Cross-Adaptive Audio Effects: Automatic Mixing, Live Performance and Everything in Between

This paper provides a systematic review of cross-adaptive audio effects and their applications. These effects extend the boundaries of traditional audio effects by potentially having many inputs and outputs, and deriving their behavior based on analysis of the signals. This mode of control allows the effects to adapt to different material, seemingly “being aware” of what they do to signals. By extension, cross-adaptive processes are designed to take into account features of, and relations between, several simultaneous signals. Thus a more global awareness and responsivity can be achieved in the processing system. When such a system is used in real-time for music performance, we observe cross-adaptive performative effects. When a musician uses the signals of other performers directly to inform the timbral character of her own instrument, it enables a radical expansion of the human-to-human interaction during music making. In order to give the signal interactions a sturdy frame of reference, we engage in a brief history of applications as well as a classification of effects types and clarifications in relation to earlier literature. With this background, the current paper defines the field, lays a formal framework, explores technical aspects and applications, and considers the future of this growing field

Study and design of an interface for remote audio processing

This project focused on the study and design of an interface for remote audio processing, with the objective of acquiring by filtering, biasing, and amplifying an analog signal before digitizing it by means of two MCP3208 ADCs to achieve a 24-bit resolution signal. The resulting digital signal was then transmitted to a Raspberry Pi using SPI protocol, where it was processed by a Flask server that could be accessed from both local and remote networks. The design of the PCB was a critical component of the project, as it had to accommodate various components and ensure accurate signal acquisition and transmission. The PCB design was created using KiCad software, which allowed for the precise placement and routing of all components. A major challenge in the design of the interface was to ensure that the analog signal was not distorted during acquisition and amplification. This was achieved through careful selection of amplifier components and using high-pass and low-pass filters to remove any unwanted noise. Once the analog signal was acquired and digitized, the resulting digital signal was transmitted to the Raspberry Pi using SPI protocol. The Raspberry Pi acted as the host for a Flask server, which could be accessed from local and remote networks using a web browser. The Flask server allowed for the processing of the digital signal and provided a user interface for controlling the gain and filtering parameters of the analog signal. This enabled the user to adjust the signal parameters to suit their specific requirements, making the interface highly flexible and adaptable to a variety of audio processing applications. The final interface was capable of remote audio processing, making it highly useful in scenarios where the audio signal needed to be acquired and processed in a location separate from the user. For example, it could be used in a recording studio, where the audio signal from the microphone could be remotely processed using the interface. The gain and filtering parameters could be adjusted in real-time, allowing the sound engineer to fine-tune the audio signal to produce the desired recording. In conclusion, the project demonstrated the feasibility and potential benefits of using a remote audio processing system for various applications. The design of the PCB, selection of components, and use of the Flask server enabled the creation of an interface that was highly flexible, accurate, and adaptable to a variety of audio processing requirements. Overall, the project represents a significant step forward in the field of remote audio processing, with the potential to benefit many different applications in the future

Adaptive digital audio effects (A-DAFx): a new class of sound transformations

International audienceAfter covering the basics of sound perception and giving an overview of commonly used audio effects (using a perceptual categorization), we propose a new concept called adaptive digital audio effects (A-DAFx). This consists of combining a sound transformation with an adaptive control. To create A-DAFx, low-level and perceptual features are extracted from the input signal, in order to derive the control values according to specific mapping functions. We detail the implementation of various new adaptive effects and give examples of their musical use

Advanced automatic mixing tools for music

PhDThis thesis presents research on several independent systems that when combined together can generate an automatic sound mix out of an unknown set of multi‐channel inputs. The research explores the possibility of reproducing the mixing decisions of a skilled audio engineer with minimal or no human interaction. The research is restricted to non‐time varying mixes for large room acoustics. This research has applications in dynamic sound music concerts, remote mixing, recording and postproduction as well as live mixing for interactive scenes. Currently, automated mixers are capable of saving a set of static mix scenes that can be loaded for later use, but they lack the ability to adapt to a different room or to a different set of inputs. In other words, they lack the ability to automatically make mixing decisions. The automatic mixer research depicted here distinguishes between the engineering mixing and the subjective mixing contributions. This research aims to automate the technical tasks related to audio mixing while freeing the audio engineer to perform the fine‐tuning involved in generating an aesthetically‐pleasing sound mix. Although the system mainly deals with the technical constraints involved in generating an audio mix, the developed system takes advantage of common practices performed by sound engineers whenever possible. The system also makes use of inter‐dependent channel information for controlling signal processing tasks while aiming to maintain system stability at all times. A working implementation of the system is described and subjective evaluation between a human mix and the automatic mix is used to measure the success of the automatic mixing tools

Making music through real-time voice timbre analysis: machine learning and timbral control

PhDPeople can achieve rich musical expression through vocal sound { see for example human beatboxing, which achieves a wide timbral variety through a range of extended techniques. Yet the vocal modality is under-exploited as a controller for music systems. If we can analyse a vocal performance suitably in real time, then this information could be used to create voice-based interfaces with the potential for intuitive and ful lling levels of expressive control. Conversely, many modern techniques for music synthesis do not imply any particular interface. Should a given parameter be controlled via a MIDI keyboard, or a slider/fader, or a rotary dial? Automatic vocal analysis could provide a fruitful basis for expressive interfaces to such electronic musical instruments. The principal questions in applying vocal-based control are how to extract musically meaningful information from the voice signal in real time, and how to convert that information suitably into control data. In this thesis we address these questions, with a focus on timbral control, and in particular we develop approaches that can be used with a wide variety of musical instruments by applying machine learning techniques to automatically derive the mappings between expressive audio input and control output. The vocal audio signal is construed to include a broad range of expression, in particular encompassing the extended techniques used in human beatboxing. The central contribution of this work is the application of supervised and unsupervised machine learning techniques to automatically map vocal timbre to synthesiser timbre and controls. Component contributions include a delayed decision-making strategy for low-latency sound classi cation, a regression-tree method to learn associations between regions of two unlabelled datasets, a fast estimator of multidimensional di erential entropy and a qualitative method for evaluating musical interfaces based on discourse analysis

Deep Learning for Audio Effects Modeling

PhD Thesis.Audio effects modeling is the process of emulating an audio effect unit and seeks to recreate the sound, behaviour and main perceptual features of an analog reference device. Audio effect units are analog or digital signal processing systems that transform certain characteristics of the sound source. These transformations can be linear or nonlinear, time-invariant or time-varying and with short-term and long-term memory. Most typical audio effect transformations are based on dynamics, such as compression; tone such as distortion; frequency such as equalization; and time such as artificial reverberation or modulation based audio effects. The digital simulation of these audio processors is normally done by designing mathematical models of these systems. This is often difficult because it seeks to accurately model all components within the effect unit, which usually contains mechanical elements together with nonlinear and time-varying analog electronics. Most existing methods for audio effects modeling are either simplified or optimized to a very specific circuit or type of audio effect and cannot be efficiently translated to other types of audio effects. This thesis aims to explore deep learning architectures for music signal processing in the context of audio effects modeling. We investigate deep neural networks as black-box modeling strategies to solve this task, i.e. by using only input-output measurements. We propose different DSP-informed deep learning models to emulate each type of audio effect transformations. Through objective perceptual-based metrics and subjective listening tests we explore the performance of these models when modeling various analog audio effects. Also, we analyze how the given tasks are accomplished and what the models are actually learning. We show virtual analog models of nonlinear effects, such as a tube preamplifier; nonlinear effects with memory, such as a transistor-based limiter; and electromechanical nonlinear time-varying effects, such as a Leslie speaker cabinet and plate and spring reverberators. We report that the proposed deep learning architectures represent an improvement of the state-of-the-art in black-box modeling of audio effects and the respective directions of future work are given