DEEP-RHYTHM FOR TEMPO ESTIMATION AND RHYTHM PATTERN RECOGNITION

International audienceIt has been shown that the harmonic series at the tempo frequency of the onset-strength-function of an audio signal accurately describes its rhythm pattern and can be used to perform tempo or rhythm pattern estimation. Recently, in the case of multi-pitch estimation, the depth of the input layer of a convolutional network has been used to represent the harmonic series of pitch candidates. We use a similar idea here to represent the harmonic series of tempo candidates. We propose the Harmonic-Constant-Q-Modulation which represents, using a 4D-tensors, the harmonic series of modulation frequencies (considered as tempo frequencies) in several acoustic frequency bands over time. This representation is used as input to a convolutional network which is trained to estimate tempo or rhythm pattern classes. Using a large number of datasets, we evaluate the performance of our approach and compare it with previous approaches. We show that it slightly increases Accuracy-1 for tempo estimation but not the average-mean-Recall for rhythm pattern recognition

Visualization of Musical Instruments through MIDI Interface

We have created a Music Visualization system that controls an LED strip by parsing MIDI signals that are sent by a musical instrument. This is achieved through a program that parses the MIDI note signals into three-byte RGB signals which are then transferred over WIFI to four Arduino boards that control the LEDs. The system provides a musician the ability to add a dynamic light display that responds to the music in real-time. The system utilizes Windows Presentation Foundation because of its event handling and GUI capabilities. The board we are using is the Arduino MKR 1010 WIFI board as it offers WIFI transmission and it can control the LEDs using pulse width modulation (PWM). The algorithms we designed for the RGB parsing are driven by the pitch and velocity of the MIDI note signals

Operant EEG-based BMI: Learning and consolidating device control with brain activity

"Whether you are reading this thesis on paper or screen, it is easy to take for granted all the highly specialized movements you are doing at this very moment just to go through each page. Just to turn a page, you have to reach for and grasp it, turn it and let go at the precise moment not to rip it.(...)

Audio Mastering as a Musical Competency

In this dissertation, I demonstrate that audio mastering is a musical competency by elucidating the most significant, and clearly audible, facets of this competence. In fact, the mastering process impacts traditionally valued musical aspects of records, such as timbre and dynamics. By applying the emerging creative scholarship method used within the field of music production studies, this dissertation will aid scholars seeking to hear and understand audio mastering by elucidating its core practices as musical endeavours. And, in so doing, I hope to enable increased clarity and accuracy in future scholarly discussions on the topic of audio mastering, as well as the end product of the mastering process: records. Audio mastering produces a so-called master of a record, that is, a finished version of a record optimized for duplication and distribution via available formats (i.e, vinyl LP, audio cassette, compact disc, mp3, wav, and so on). This musical process plays a crucial role in determining how records finally sound, and it is not, as is so often inferred in research, the sole concern of a few technicians working in isolated rooms at a record label\u27s corporate headquarters. In fact, as Mark Cousins and Russ Hepworth-Sawyer (2013: 2) explain, nowadays “all musicians and engineers, to a lesser or greater extent, have to actively engage in the mastering process.” Thus, this dissertation clarifies the creative nature of audio mastering through an investigation of how mastering engineers hear records, and how they use technology to achieve the sonic goals they conceptualize

Music Onset Detection Based on Resonator Time Frequency Image

This paper describes a new method for music onset detection. The novelty of the approach consists mainly of two elements: the time–frequency processing and the detection stages. The resonator time frequency image (RTFI) is the basic time–frequency analysis tool. The time–frequency processing part is in charge of transforming the RTFI energy spectrum into more natural energy change and pitch-change cues that are then used as input elements for the detection of music onsets by detection tools. Two detection algorithms have been developed: an energy-based algorithm and a pitch-based one. The energy-based detection algorithm exploits energy-change cues and performs particularly well for the detection of hard onsets. The pitch-based algorithm successfully exploits stable pitch cues for the onset detection in polyphonic music, and achieves much better performances than the energy-based algorithm when applied to the detection of soft onsets. Results for both the energy-based and pitch-based detection algorithms have been obtained on a large music dataset

Multisource and Multitemporal Data Fusion in Remote Sensing

The sharp and recent increase in the availability of data captured by different sensors combined with their considerably heterogeneous natures poses a serious challenge for the effective and efficient processing of remotely sensed data. Such an increase in remote sensing and ancillary datasets, however, opens up the possibility of utilizing multimodal datasets in a joint manner to further improve the performance of the processing approaches with respect to the application at hand. Multisource data fusion has, therefore, received enormous attention from researchers worldwide for a wide variety of applications. Moreover, thanks to the revisit capability of several spaceborne sensors, the integration of the temporal information with the spatial and/or spectral/backscattering information of the remotely sensed data is possible and helps to move from a representation of 2D/3D data to 4D data structures, where the time variable adds new information as well as challenges for the information extraction algorithms. There are a huge number of research works dedicated to multisource and multitemporal data fusion, but the methods for the fusion of different modalities have expanded in different paths according to each research community. This paper brings together the advances of multisource and multitemporal data fusion approaches with respect to different research communities and provides a thorough and discipline-specific starting point for researchers at different levels (i.e., students, researchers, and senior researchers) willing to conduct novel investigations on this challenging topic by supplying sufficient detail and references

Functional Magnetic Resonance Imaging at High Spatiotemporal Resolution using EPI Combined with Different k-Space Undersampling Techniques at 3 Tesla

In functional magnetic resonance imaging (fMRI), major drawbacks of the commonly used echo-planar imaging (EPI) sequence are limited spatial specificity due to blurring and distortions as well as signal cancellation in areas affected by susceptibility gradients, such as the orbitofrontal cortex (OFC). In contrast, segmented EPI techniques facilitate ultra-high spatial but low temporal resolution. In this work, an EPI sequence with optimized slice-dependent echo time was developed avoiding signal drop outs in the OFC in 50 % of all subjects during fMRI (N = 12) compared to a standard EPI sequence. The average number of activated voxels detected in the OFC was thereby increased by a factor of 6.3. It was further shown for the first time that the spatial specificity in EPI fMRI at 3 T can be improved by increasing the matrix size in combination with the parallel imaging factor beyond conventional EPI parameter settings. By using the proposed high-resolution compared to a standard EPI protocol, the multi-subject analysis of a simple fingertapping task (N = 6) and a sophisticated motivation task (N = 15) showed robust and clearly less blurred activation in the sensorimotor cortex (SMC) and in the nucleus accumbens (NAcc), respectively. The number of separable clusters detected in the SMC and in the NAcc was thereby increased by a factor of 2.7 and 1.4, respectively. In order to perform fMRI at ultra-high spatial and high temporal resolution, a segmented EPI sequence was highly accelerated (R = 8) with the so-called UNFOLD technique. Both, the MR sequence and data post-processing were optimized facilitating the robust detection of neuronal activation at 0.7 x 0.7 mm2 resolution and half-brain coverage. Last but not least, a novel filtering strategy is proposed minimizing temporal coherences in UNFOLD datasets and thus improving the detectability of neuronal activation. By using the proposed filter compared to a standard filter, the number of activated voxels detected in the SMC (N = 5) was increased up to a factor of 1.4

Integrating Local and Global Error Statistics for Multi-Scale RBF Network Training: An Assessment on Remote Sensing Data

Background This study discusses the theoretical underpinnings of a novel multi-scale radial basis function (MSRBF) neural network along with its application to classification and regression tasks in remote sensing. The novelty of the proposed MSRBF network relies on the integration of both local and global error statistics in the node selection process. Methodology and Principal Findings The method was tested on a binary classification task, detection of impervious surfaces using a Landsat satellite image, and a regression problem, simulation of waveform LiDAR data. In the classification scenario, results indicate that the MSRBF is superior to existing radial basis function and back propagation neural networks in terms of obtained classification accuracy and training-testing consistency, especially for smaller datasets. The latter is especially important as reference data acquisition is always an issue in remote sensing applications. In the regression case, MSRBF provided improved accuracy and consistency when contrasted with a multi kernel RBF network. Conclusion and Significance Results highlight the potential of a novel training methodology that is not restricted to a specific algorithmic type, therefore significantly advancing machine learning algorithms for classification and regression tasks. The MSRBF is expected to find numerous applications within and outside the remote sensing field

Lunar Gravitational-Wave Antenna

Monitoring of vibrational eigenmodes of an elastic body excited by gravitational waves was one of the first concepts proposed for the detection of gravitational waves. At laboratory scale, these experiments became known as resonant-bar detectors first developed by Joseph Weber in the 1960s. Due to the dimensions of these bars, the targeted signal frequencies were in the kHz range. Weber also pointed out that monitoring of vibrations of Earth or Moon could reveal gravitational waves in the mHz band. His Lunar Surface Gravimeter experiment deployed on the Moon by the Apollo 17 crew had a technical failure rendering the data useless. In this article, we revisit the idea and propose a Lunar Gravitational-Wave Antenna (LGWA). We find that LGWA could become an important partner observatory for joint observations with the space-borne, laser-interferometric detector LISA, and at the same time contribute an independent science case due to LGWA's unique features. Technical challenges need to be overcome for the deployment of the experiment, and development of inertial vibration sensor technology lays out a future path for this exciting detector concept.Comment: 29 pages, 17 figure