Comparing timbre estimation using auditory models with and without hearing loss

We propose a concept for evaluating signal transformations for music signals with respect to an individual hearing deficit by using an auditory model. This deficit is simulated in the model by changing specific model parameters. Our idea is extracting the musical attributes rhythm, pitch, loudness and timbre and comparing the modified model output to the original one. While rhythm, pitch, and loudness estimation are studied in previous works the focus in this paper concentrates on timbre estimation. Results are shown for the original auditory model and three models, each simulating a specific hearing loss

A multivariate approach for onset detection using supervised classification

In this paper we introduce a new onset detection approach which incorporates a supervised classification model for estimating the tone onset probability in signal frames. In contrast to the most classical strategies where only one detection function can be applied for signal feature extraction, the classification model can be fitted on a large feature set. This is meaningful since, depending on the music characteristics, some detection functions can be more advantageous that the others. Although the idea of the considering of many detection functions is not new in the literature, these functions are, so far, treated in a univariate way by, e.g., building of weighted sums. This probably lies on the difficulties of the direct transfer of the classification ideas to the onset detection task. The goodness measure of onset detection is namely based on the comparison of two time vectors while by the classification such a measure is derived from the framewise matches of predicted and true labels. In this work we first construct { based on several resent publications { a comprehensive univariate onset detection algorithm which depends on many free settable parameters. Then, the new multivariate approach also depending on many free parameters is introduced. The parameters of the both onset detection strategies are optimized for online and offline cases by utilizing an appropriate validation technique. The main funding is that the multivariate strategy outperforms the univariate one significantly regarding the F-measure. Furthermore, the multivariate approach seems to be especially beneficial in online case since it requires only the halve of the future signal information comparing to the best setting of the univariate onset detection

Model based optimization of music onset detection

In this paper a comprehensive online music onset detection algorithm is introduced where - in contrast to many other relevant publications - 14 important algorithm parameters are optimized simultaneously. For solving the optimization problem we derive an extensive tool for iterative model based optimization. In each iteration, a very time consuming evaluation has to be per- formed on a large music data base. To speed up this procedure, the expected performance of each newly proposed setting is estimated in a pretest on a representative part of the data so that just very promising points are evaluated on all data. We compare different variants of the classical and the fast optimization strategies with respect to the F-values of their best identified parameter settings. The performance of the fast approach appears to be competitive with the classical one while saving more than 80% of music piece evaluations on average. Our best found parameter settings, both for online and offline onset detection, are mainly in accordance with the usual choices in the state- of-the art literature concerning, e.g., the spectral flux detection function or preferences for window length and overlap. However, we also found unexpected results. For example, the adaptive whitening pre-processing step showed no effect

Time efficient optimization of instance based problems with application to tone onset detection

A time efficient optimization technique for instance based problems is proposed, where for each parameter setting the target function has to be evaluated on a large set of problem instances. Computational time is reduced by beginning with a performance estimation based on the evaluation of a representative subset of instances. Subsequently, only promising settings are evaluated on the whole data set. As application a comprehensive music onset detection algorithm is introduced where several numerical and categorical algorithm parameters are optimized simultaneously. Here, problem instances are music pieces of a data base. Sequential model based optimization is an appropriate technique to solve this optimization problem. The proposed optimization strategy is compared to the usual model based approach with respect to the goodness measure for tone onset detection. The performance of the proposed method appears to be competitive with the usual one while saving more than 84% of instance evaluation time on average. One other aspect is a comparison of two strategies for handling categorical parameters in Kriging based optimization

A computational study of auditory models in music recognition tasks for normalhearing and hearing-impaired listeners

The utility of auditory models for solving three music recognition tasks { onset detection, pitch estimation and instrument recognition { is analyzed. Appropriate features are introduced which enable the use of supervised classification. The auditory model-based approaches are tested in a comprehensive study and compared to state-of-the-art methods, which usually do not employ an auditory model. For this study, music data is selected according to an experimental design, which enables statements about performance differences with respect to specific music characteristics. The results confirm that the performance of music classification using the auditory model is at least comparable to the traditional methods. Furthermore, the auditory model is modified to exemplify the decrease of recognition rates in the presence of hearing deficits. The resulting system is a basis for estimating the intelligibility of music which in the future might be used for the automatic assessment of hearing instruments

Musikklassifikation mittels auditorischer Modelle zur Optimierung von Hörgeräten

In der Dissertation werden für drei Musikklassifikationsprobleme - Toneinsatzzeiterkennung, Tonhöhenschätzung und Instrumentenklassifikation - Verfahren entwickelt, die auf der Ausgabe eines Simulationsmodells des menschlichen Hörvorgangs (Ohrmodell) aufbauen. Für modifizierte Ohrmodelle, die eine Hörschädigung simulieren, kann mit Hilfe dieser Verfahren evaluiert werden, wie gut Musik differenziert wird. Ziel eines Hörgeräts ist es, die Identifizierbarkeit von Musikeigenschaften zu steigern. Durch die Verknüpfung eines Hörgerätealgorithmus mit dem Ohrmodell und den Musikklassifikationsverfahren kann somit die Güte des Hörgeräts für eine durch das Ohrmodell gegebene Hörschädigung bewertet werden. Für die Paramateroptimierung des Hörgerätealgorithmus mit Hilfe der sequentiellen modellbasierten Optimierung (MBO) wird diese Bewertung als Kostenfunktion verwendet. Für die Schätzung der drei untersuchten Klassifikationsprobleme existieren bereits umfangreiche Forschungsarbeiten, die jedoch üblicherweise nicht die Ohrmodellausgabe sondern die akustische Wellenform als Grundlage nutzen. Daher werden zunächst die entwickelten Verfahren gegen diese Standardverfahren getestet. Für die Vergleichsexperimente wird ein statistischer Versuchsplan, dem ein Plackett-Burman-Design zu Grunde liegt, aufgestellt, um die untersuchten Musikdaten in einer strukturierten Form auszuwählen. Es wird gezeigt, dass die Ohrmodellbasierte Merkmalsgrundlage keinen Nachteil darstellt, denn für die Tonhöhenschätzung und die Instrumentenklassifikation werden sogar die Ergebnisse der Standardverfahren übertroffen. Lediglich bei der Einsatzzeiterkennung schneidet das entwickelte Verfahren etwas schlechter ab, für das jedoch weitere Verbessserungsideen vorgeschlagen werden. Durch den Versuchsplan werden acht musikalische Einflussgrößen berücksichtigt. Für diese wird evaluiert, wie sie sich auf die Güte der Klassifikationsverfahren auswirken. Neben vielen erwarteten Ergebnissen, z.B. die größeren Fehlerraten bei einer Streicherbegleitung auf Grund der klanglichen Nähe zum Cello, kommen auch einige unerwartete Ergebnisse heraus. Beispielsweise sind höhere Tonhöhen und kürzere Töne vorteilhaft für die Einsatzzeiterkennung, wohingegen tiefere Tonhöhen die Ergebnisse der Instrumentenerkennung verbessern. Der Versuchsplan wird auch für einen Vergleich des normalen Ohrmodells (ohne Hörschädigung) mit drei Modellen, die unterschiedliche Hörschädigungen simulieren (Hearing Dummies), verwendet. Für all diese Modelle steigen die Fehlerraten der Musikklassifikationsverfahren in plausiblen Stärken, die abhängig von den Hörschädigungen sind. Schließlich wird die praktische Anwendbarkeit des Bewertungsverfahrens in einer leicht vereinfachten Form, die aus Rechenzeitgründen lediglich die Ergebnisse der Instrumentenerkennung berücksichtigt, für die Optimierung eines Hörgerätealgorithmus getestet. Dabei wird MBO verwendet, um das Hörgerät optimal an eine starke Hörschädigung (Hearing Dummy 1) anzupassen. Durch das optimierte Hörgerät wird die Fehlklassifikationsrate stark reduziert, und auch eine vergleichende Experteneinstellung wird geschlagen (27% ohne Hörgerät, 19% mit Hörgerät und Experteneinstellung, 14% mit optimiertem Hörgerät). Wie die Auswertung des Versuchsplans zeigt, wird am stärksten die Klassifikationsgüte für Musikstücke mit Streicherbegleitung verbessert. Am Ende der Dissertation wird noch umfangreich diskutiert, welche Möglichkeiten es gibt, die Laufzeit von MBO für das vorgestellte Optimierungsproblem zu reduzieren

Tone onset detection using an auditory model

Onset detection is an important step for music transcription and other tasks frequently encountered in music processing. Although several approaches have been developed for this task, neither of them works well under all circumstances. In Bauer et al. (2012) we investigated the influence of several factors like instrumentation on the accuracy of onset detection. In this work, this investigation is extended by a computational model of the human auditory periphery. Instead of the original signal the output of the simulated auditory nerve fibers is used. The main challenge here is combining the outputs of all auditory nerve fibers to one feature for onset detection. Different approaches are presented and compared. Our investigation shows that using the auditory model output leads to essential improvements of the onset detection rate for some instruments compared to previous results

Efficient global optimization: Motivation, variations and applications

A popular optimization method of a black box objective function is Efficient Global Optimization (EGO), also known as Sequential Model Based Optimization, SMBO, with kriging and expected improvement. EGO is a sequential design of experiments aiming at gaining as much information as possible from as few experiments as feasible by a skillful choice of the factor settings in a sequential way. In this paper we will introduce the standard procedure and some of its variants. In particular, we will propose some new variants like regression as a modeling alternative to kriging and two simple methods for the handling of categorical variables, and we will discuss focus search for the optimization of the infill criterion. Finally, we will give relevant examples for the application of the method. Moreover, in our group, we implemented all the described methods in the publicly available R package mlrMBO