12 research outputs found
Chromatic PAC-Bayes Bounds for Non-IID Data: Applications to Ranking and Stationary -Mixing Processes
Pac-Bayes bounds are among the most accurate generalization bounds for
classifiers learned from independently and identically distributed (IID) data,
and it is particularly so for margin classifiers: there have been recent
contributions showing how practical these bounds can be either to perform model
selection (Ambroladze et al., 2007) or even to directly guide the learning of
linear classifiers (Germain et al., 2009). However, there are many practical
situations where the training data show some dependencies and where the
traditional IID assumption does not hold. Stating generalization bounds for
such frameworks is therefore of the utmost interest, both from theoretical and
practical standpoints. In this work, we propose the first - to the best of our
knowledge - Pac-Bayes generalization bounds for classifiers trained on data
exhibiting interdependencies. The approach undertaken to establish our results
is based on the decomposition of a so-called dependency graph that encodes the
dependencies within the data, in sets of independent data, thanks to graph
fractional covers. Our bounds are very general, since being able to find an
upper bound on the fractional chromatic number of the dependency graph is
sufficient to get new Pac-Bayes bounds for specific settings. We show how our
results can be used to derive bounds for ranking statistics (such as Auc) and
classifiers trained on data distributed according to a stationary {\ss}-mixing
process. In the way, we show how our approach seemlessly allows us to deal with
U-processes. As a side note, we also provide a Pac-Bayes generalization bound
for classifiers learned on data from stationary -mixing distributions.Comment: Long version of the AISTATS 09 paper:
http://jmlr.csail.mit.edu/proceedings/papers/v5/ralaivola09a/ralaivola09a.pd
Optimal Window and Lattice in Gabor Transform Application to Audio Analysis
This article deals with the use of optimal lattice and optimal window in
Discrete Gabor Transform computation. In the case of a generalized Gaussian
window, extending earlier contributions, we introduce an additional local
window adaptation technique for non-stationary signals. We illustrate our
approach and the earlier one by addressing three time-frequency analysis
problems to show the improvements achieved by the use of optimal lattice and
window: close frequencies distinction, frequency estimation and SNR estimation.
The results are presented, when possible, with real world audio signals
Fenêtre et grille optimales pour la transformée de Gabor Exemples d'application à l'analyse audio
International audienceThis article deals with the use of optimal lattice and optimal window in Discrete Gabor Transform computation. In the case of a generalized Gaussian window, extending earlier contributions, we introduce an additional local window adaptation technique for non-stationary signals. We illustrate our approach and the earlier one by addressing three time-frequency analysis problems: close frequencies distinction, frequency estimation and Signal to Noise Ratio estimation. The results are presented, when possible, with real world audio signals.Cet article présente l'utilisation d'une grille optimale et d'une fenêtre optimale pour le calcul de la transformée de Gabor discrète. Dans le cas d'une Gaussienne généralisée, nous étendons des travaux précédents et proposons une fenêtre localement optimale pour des si-gnaux non-stationnaires. Nous présentons des résultats sur trois problèmes d'analyse temps-fréquence, sur des signaux réels et synthétiques : la distinction de composantes temps-fréquence proches, l'estimation de fréquence instantané et l'estimation du Rapport Signal à Bruit. Abstract – This article deals with the use of optimal lattice and optimal window in Discrete Gabor Transform computation. In the case of a generalized Gaussian window, extending earlier contributions, we introduce an additional local window adaptation technique for non-stationary signals. We illustrate our approach and the earlier one by addressing three time-frequency analysis problems: close frequencies distinction, frequency estimation and Signal to Noise Ratio estimation. The results are presented, when possible, with real world audio signals
An optimally concentrated Gabor transform for localized time-frequency components
Gabor analysis is one of the most common instances of time-frequency signal analysis. Choosing a suitable window for the Gabor transform of a signal is often a challenge for practical applications, in particular in audio signal processing. Many time-frequency (TF) patterns of different shapes may be present in a signal and they can not all be sparsely represented in the same spectrogram. We propose several algorithms, which provide optimal windows for a user-selected TF pattern with respect to different concentration criteria. We base our optimization algorithm on l p -norms as measure of TF spreading. For a given number of sampling points in the TF plane we also propose optimal lattices to be used with the obtained windows. We illustrate the potentiality of the method on selected numerical examples
An optimally concentrated Gabor transform for localized time-frequency components
Gabor analysis is one of the most common instances of time-frequency signal
analysis. Choosing a suitable window for the Gabor transform of a signal is
often a challenge for practical applications, in particular in audio signal
processing. Many time-frequency (TF) patterns of different shapes may be
present in a signal and they can not all be sparsely represented in the same
spectrogram. We propose several algorithms, which provide optimal windows for a
user-selected TF pattern with respect to different concentration criteria. We
base our optimization algorithm on -norms as measure of TF spreading. For
a given number of sampling points in the TF plane we also propose optimal
lattices to be used with the obtained windows. We illustrate the potentiality
of the method on selected numerical examples
Learning from Noisy Data using Hyperplane Sampling and Sample Averages
Abstract. We present a new classification algorithm capable of learning from data corrupted by a class dependent uniform classification noise. The produced classifier is a linear classifier, and the algorithm works seamlessly when using kernels. The algorithm relies on the sampling of random hyperplanes that help the building of new training examples of which the correct classes are known; a linear classifier (e.g. an SVM) is learned from these examples and output by the algorithm. The produced examples are sample averages computed from the data at hand with respect to areas of the space defined by the random hyperplanes and the target hyperplane. A statistical analysis of the properties of these sample averages is provided as well as results from numerical simulations conducted on synthetic datasets. These simulations show that the linear and kernelized versions of our algorithm are effective for learning from both noise-free and noisy data.
Apprentissage de SVM sur Données Bruitées
Abstract: Après avoir exhibé un exemple basique montrant que les SVM à marges douces (CSVM) ne sont pas tolérantes au bruit de classification uniforme, nous proposons une version modifiée de CSVM basée sur une fonction objectif utilisant un estimateur des slack variables du problème non bruité. Les bonnes propriétés de cet estimateur sont appuyées par une analyse théorique ainsi que par des simulations numériques effectuées sur un jeu de données synthétique
Chromatic pac bayes bounds for non-iid data
PAC-Bayes bounds are among the most accurate generalization bounds for classifiers learned with IID data, and it is particularly so for margin classifiers. However, there are many practical cases where the training data show some dependencies and where the traditional IID assumption does not apply. Stating generalization bounds for such frameworks is therefore of the utmost interest, both from theoretical and practical standpoints. In this work, we propose the first – to the best of our knowledge – PAC-Bayes generalization bounds for classifiers trained on data exhibiting dependencies. The approach undertaken to establish our results is based on the decomposition of a so-called dependency graph that encodes the dependencies within the data, in sets of independent data, through the tool of graph fractional covers. Our bounds are very general, since being able to find an upper bound on the (fractional) chromatic number of the dependency graph is sufficient to get new PAC-Bayes bounds for specific settings. We show how our results can be used to derive bounds for bipartite ranking and windowed prediction on sequential data.