Deep Learning for Audio Signal Processing

Given the recent surge in developments of deep learning, this article provides a review of the state-of-the-art deep learning techniques for audio signal processing. Speech, music, and environmental sound processing are considered side-by-side, in order to point out similarities and differences between the domains, highlighting general methods, problems, key references, and potential for cross-fertilization between areas. The dominant feature representations (in particular, log-mel spectra and raw waveform) and deep learning models are reviewed, including convolutional neural networks, variants of the long short-term memory architecture, as well as more audio-specific neural network models. Subsequently, prominent deep learning application areas are covered, i.e. audio recognition (automatic speech recognition, music information retrieval, environmental sound detection, localization and tracking) and synthesis and transformation (source separation, audio enhancement, generative models for speech, sound, and music synthesis). Finally, key issues and future questions regarding deep learning applied to audio signal processing are identified.Comment: 15 pages, 2 pdf figure

Local Periodicity-Based Beat Tracking for Expressive Classical Piano Music

To model the periodicity of beats, state-of-the-art beat tracking systems use "post-processing trackers" (PPTs) that rely on several empirically determined global assumptions for tempo transition, which work well for music with a steady tempo. For expressive classical music, however, these assumptions can be too rigid. With two large datasets of Western classical piano music, namely the Aligned Scores and Performances (ASAP) dataset and a dataset of Chopin's Mazurkas (Maz-5), we report on experiments showing the failure of existing PPTs to cope with local tempo changes, thus calling for new methods. In this paper, we propose a new local periodicity-based PPT, called predominant local pulse-based dynamic programming (PLPDP) tracking, that allows for more flexible tempo transitions. Specifically, the new PPT incorporates a method called "predominant local pulses" (PLP) in combination with a dynamic programming (DP) component to jointly consider the locally detected periodicity and beat activation strength at each time instant. Accordingly, PLPDP accounts for the local periodicity, rather than relying on a global tempo assumption. Compared to existing PPTs, PLPDP particularly enhances the recall values at the cost of a lower precision, resulting in an overall improvement of F1-score for beat tracking in ASAP (from 0.473 to 0.493) and Maz-5 (from 0.595 to 0.838).Comment: Accepted to IEEE/ACM Transactions on Audio, Speech, and Language Processing (July 2023

Tracking beats and microtiming in Afro-Latin American music using conditional random fields and deep learning

Trabajo presentado en ISMIR 2019 : 20th Conference of the International Society for Music Information Retrieval, Delft, Netherlands, 4-8 nov, 2019PostprintEvents in music frequently exhibit small-scale temporal deviations (microtiming), with respect to the underlying regular metrical grid. In some cases, as in music from the Afro-Latin American tradition, such deviations appear systematically, disclosing their structural importance in rhythmic and stylistic configuration. In this work we explore the idea of automatically and jointly tracking beats and microtiming in timekeeper instruments of Afro-Latin American music, in particular Brazilian samba and Uruguayan candombe. To that end, we propose a language model based on conditional random fields that integrates beat and onset likelihoods as observations. We derive those activations using deep neural networks and evaluate its performance on manually annotated data using a scheme adapted to this task. We assess our approach in controlled conditions suitable for these timekeeper instruments, and study the microtiming profiles’ dependency on genre and performer, illustrating promising aspects of this technique towards a more comprehensive understanding of these music traditions

Cubaneo In Latin Piano: A Parametric Approach To Gesture, Texture, And Motivic Variation

ABSTRACT CUBANEO IN LATIN PIANO: A PARAMETRIC APPROACH TO GESTURE, TEXTURE, AND MOTIVIC VARIATION COPYRIGHT Orlando Enrique Fiol 2018 Dr. Carol A. Muller Over the past century of recorded evidence, Cuban popular music has undergone great stylistic changes, especially regarding the piano tumbao. Hybridity in the Cuban/Latin context has taken place on different levels to varying extents involving instruments, genres, melody, harmony, rhythm, and musical structures. This hybridity has involved melding, fusing, borrowing, repurposing, adopting, adapting, and substituting. But quantifying and pinpointing these processes has been difficult because each variable or parameter embodies a history and a walking archive of sonic aesthetics. In an attempt to classify and quantify precise parameters involved in hybridity, this dissertation presents a paradigmatic model, organizing music into vocabularies, repertories, and abstract procedures. Cuba\u27s pianistic vocabularies are used very interactively, depending on genre, composite ensemble texture, vocal timbre, performing venue, and personal taste. These vocabularies include: melodic phrases, harmonic progressions, rhythmic cells and variation schemes to replace repetition with methodical elaboration of the piano tumbao as a main theme. These pianistic vocabularies comprise what we actually hear. Repertories, such as pre-composed songs, ensemble arrangements, and open- ended montuno and solo sections, situate and contextualize what we hear in real life musical performances. Abstract procedures are the thoughts, aesthetics, intentions, and parametric rules governing what Cuban/Latin pianists consider possible. Abstract procedures alter vocabularies by displacing, expanding, contracting, recombining, permuting, and layering them. As Cuba\u27s popular musics find homes in its musical diaspora (the United States, Latin America and Europe), Cuban pianists have sought to differentiate their craft from global salsa and Latin jazz pianists. Expanding the piano\u27s gestural/textural vocabulary beyond pre-Revolutionary traditions and performance practices, the timba piano tumbao is a powerful marker of Cuban identity and musical pride, transcending national borders and cultural boundaries