High-resolution sinusoidal analysis for resolving harmonic collisions in music audio signal processing

Many music signals can largely be considered an additive combination of multiple sources, such as musical instruments or voice. If the musical sources are pitched instruments, the spectra they produce are predominantly harmonic, and are thus well suited to an additive sinusoidal model. However, due to resolution limits inherent in time-frequency analyses, when the harmonics of multiple sources occupy equivalent time-frequency regions, their individual properties are additively combined in the time-frequency representation of the mixed signal. Any such time-frequency point in a mixture where multiple harmonics overlap produces a single observation from which the contributions owed to each of the individual harmonics cannot be trivially deduced. These overlaps are referred to as overlapping partials or harmonic collisions. If one wishes to infer some information about individual sources in music mixtures, the information carried in regions where collided harmonics exist becomes unreliable due to interference from other sources. This interference has ramifications in a variety of music signal processing applications such as multiple fundamental frequency estimation, source separation, and instrumentation identification. This thesis addresses harmonic collisions in music signal processing applications. As a solution to the harmonic collision problem, a class of signal subspace-based high-resolution sinusoidal parameter estimators is explored. Specifically, the direct matrix pencil method, or equivalently, the Estimation of Signal Parameters via Rotational Invariance Techniques (ESPRIT) method, is used with the goal of producing estimates of the salient parameters of individual harmonics that occupy equivalent time-frequency regions. This estimation method is adapted here to be applicable to time-varying signals such as musical audio. While high-resolution methods have been previously explored in the context of music signal processing, previous work has not addressed whether or not such methods truly produce high-resolution sinusoidal parameter estimates in real-world music audio signals. Therefore, this thesis answers the question of whether high-resolution sinusoidal parameter estimators are really high-resolution for real music signals. This work directly explores the capabilities of this form of sinusoidal parameter estimation to resolve collided harmonics. The capabilities of this analysis method are also explored in the context of music signal processing applications. Potential benefits of high-resolution sinusoidal analysis are examined in experiments involving multiple fundamental frequency estimation and audio source separation. This work shows that there are indeed benefits to high-resolution sinusoidal analysis in music signal processing applications, especially when compared to methods that produce sinusoidal parameter estimates based on more traditional time-frequency representations. The benefits of this form of sinusoidal analysis are made most evident in multiple fundamental frequency estimation applications, where substantial performance gains are seen. High-resolution analysis in the context of computational auditory scene analysis-based source separation shows similar performance to existing comparable methods

The DESAM toolbox: spectral analysis of musical audio

International audienceIn this paper is presented the DESAM Toolbox, a set of Matlab functions dedicated to the estimation of widely used spectral models for music signals. Although those models can be used in Music Information Retrieval (MIR) tasks, the core functions of the toolbox do not focus on any specific application. It is rather aimed at providing a range of state-of-the-art signal processing tools that decompose music files according to different signal models, giving rise to different ``mid-level'' representations. After motivating the need for such a toolbox, this paper offers an overview of the overall organization of the toolbox, and describes all available functionalities

Principled Design and Implementation of Steerable Detectors

We provide a complete pipeline for the detection of patterns of interest in an image. In our approach, the patterns are assumed to be adequately modeled by a known template, and are located at unknown position and orientation. We propose a continuous-domain additive image model, where the analyzed image is the sum of the template and an isotropic background signal with self-similar isotropic power-spectrum. The method is able to learn an optimal steerable filter fulfilling the SNR criterion based on one single template and background pair, that therefore strongly responds to the template, while optimally decoupling from the background model. The proposed filter then allows for a fast detection process, with the unknown orientation estimation through the use of steerability properties. In practice, the implementation requires to discretize the continuous-domain formulation on polar grids, which is performed using radial B-splines. We demonstrate the practical usefulness of our method on a variety of template approximation and pattern detection experiments

Tunable transport with broken space-time symmetries

Transport properties of particles and waves in spatially periodic structures that are driven by external time-dependent forces manifestly depend on the space-time symmetries of the corresponding equations of motion. A systematic analysis of these symmetries uncovers the conditions necessary for obtaining directed transport. In this work we give a unified introduction into the symmetry analysis and demonstrate its action on the motion in one-dimensional periodic, both in time and space, potentials. We further generalize the analysis to quasi-periodic drivings, higher space dimensions, and quantum dynamics. Recent experimental results on the transport of cold and ultracold atomic ensembles in ac-driven optical potentials are reviewed as illustrations of theoretical considerations.Comment: Phys. Rep., in pres

Study on a miniaturized satellite payload for atmospheric temperature measurements

The atmospheric temperature reflects the thermal balance of the atmosphere and is a valuable indicator of climate change. It has been widely recognized that the atmospheric gravity wave activity has a profound effect on the large-scale circulation, thermal and constituent structures in the mesosphere and lower thermosphere (MLT). Temperature distribution in this region is an essential component to identify and quantify gravity waves. Observation from remote sensing instruments on satellite platforms is an effective way to measure the temperature in the MLT region. A miniaturized satellite payload is developed to measure the atmospheric temperature in the MLT region via observing the O2A-band emission. Following a Boltzmann distribution, the relative intensities of the emission lines can be used to derive the temperature profile. Based on the spatial heterodyne spectroscopy, this instrument is capable of resolving individual emission lines in the O2A-band for the spatial and spectral information simultaneously. The monolithic and compact feature of this spectrometer makes it suitable for operating on satellite platforms. In this work, the characterization of the instrument is investigated for the purpose of simultaneously measuring multiple emission lines of the O2A-band. The instrument is explored through a series of experimental methods, providing characteristics of the instrument and evaluation of its performance. In spatial and spectral domain, Level- 0 and Level-1 data processors are developed to convert the raw data to the calibrated spectral radiance for further temperature and gravity wave characterization. Within this framework, the performance of the utilized detector is evaluated along with its radiation tolerance in space environment. In the processor, the detector artifacts are corrected based on the measurements in laboratory or in space. The radiometric response of the instrument is characterized on a pixel-by-pixel basis using a blackbody. An interferogram distortion correction algorithm is developed to correct for the spatial and phase distortion induced by the detector optics. Further, localized phase distortion correction is implemented to correct for the remaining phase error. Unwanted ghost emission lines are removed based on two dimensional Fourier transform. In the spectral domain, the processing steps mainly consist of wavelength calibration and instrument spectral response correction, including filter response correction and modulation efficiency correction. As an in-orbit verification, the AtmoSHINE instrument was successfully deployed in space on 22th of December, 2018. In the first test phase, the functionality and the performance of the instrument in space were verified. The detector dark current measurement in orbit is consistent with the ground-based results. Based on the the calibration procedures and the developed data processing algorithms, the O2A-band emission lines can be successfully resolved. A cross-verification of the AtmoSHINE limb radiance profile with other satellite payload measurements indicates that the radiometric performance of the instrument is within the expectation. The retrieved temperature parameters are studied with respect to different number of samples and different objective functions in the optimization. This work verifies the ability of the instrument to derive the atmospheric temperature in the MLT region and its potential application in gravity wave detections