29 research outputs found
Méthodes d'apprentissage statistique pour le ranking : théorie, algorithmes et applications
Multipartite ranking is a statistical learning problem that consists in ordering observations that belong to a high dimensional feature space in the same order as the labels, so that the observations with the highest label appear at the top of the list. This work aims to understand the probabilistic nature of the multipartite ranking problem in order to obtain theoretical guarantees for ranking algorithms. In this context, the output of a ranking algorithm takes the form of a scoring function, a function that maps the space of the observation to the real line which order is induced using the values on the real line. The contributions of this manuscript are the following : First, we focus on the characterization of optimal solutions to multipartite ranking. The second research theme is the design of algorithms to produce scoring functions. We offer two methods, the first using an aggregation procedure, the second an approximation scheme. Finally, we return to the binary ranking problem to establish adaptive minimax rate of convergence.Le ranking multipartite est un problème d'apprentissage statistique qui consiste à ordonner les observations qui appartiennent à un espace de grande dimension dans le même ordre que les labels, de sorte que les observations avec le label le plus élevé apparaissent en haut de la liste. Cette thèse vise à comprendre la nature probabiliste du problème de ranking multipartite afin d'obtenir des garanties théoriques pour les algorithmes de ranking. Dans ce cadre, la sortie d'un algorithme de ranking prend la forme d'une fonction de scoring, une fonction qui envoie l'espace des observations sur la droite réelle et l'ordre finale est construit en utilisant l'ordre induit par la droite réelle. Les contributions de ce manuscrit sont les suivantes : d'abord, nous nous concentrons sur la caractérisation des solutions optimales de ranking multipartite. Le deuxième thème de recherche est la conception d'algorithmes pour produire des fonctions de scoring. Nous proposons deux méthodes, la première utilisant une procédure d'agrégation, la deuxième un schema d'approximation. Enfin, nous revenons au problème de ranking binaire afin d'établir des vitesse minimax adaptives de convergences
Ordinal regression methods: survey and experimental study
Abstract—Ordinal regression problems are those machine learning problems where the objective is to classify patterns using a
categorical scale which shows a natural order between the labels. Many real-world applications present this labelling structure and
that has increased the number of methods and algorithms developed over the last years in this field. Although ordinal regression can
be faced using standard nominal classification techniques, there are several algorithms which can specifically benefit from the ordering
information. Therefore, this paper is aimed at reviewing the state of the art on these techniques and proposing a taxonomy based on
how the models are constructed to take the order into account. Furthermore, a thorough experimental study is proposed to check if
the use of the order information improves the performance of the models obtained, considering some of the approaches within the
taxonomy. The results confirm that ordering information benefits ordinal models improving their accuracy and the closeness of the
predictions to actual targets in the ordinal scal
Quadruply Stochastic Gradient Method for Large Scale Nonlinear Semi-Supervised Ordinal Regression AUC Optimization
Semi-supervised ordinal regression (SOR) problems are ubiquitous in
real-world applications, where only a few ordered instances are labeled and
massive instances remain unlabeled. Recent researches have shown that directly
optimizing concordance index or AUC can impose a better ranking on the data
than optimizing the traditional error rate in ordinal regression (OR) problems.
In this paper, we propose an unbiased objective function for SOR AUC
optimization based on ordinal binary decomposition approach. Besides, to handle
the large-scale kernelized learning problems, we propose a scalable algorithm
called QSORAO using the doubly stochastic gradients (DSG) framework for
functional optimization. Theoretically, we prove that our method can converge
to the optimal solution at the rate of , where is the number of
iterations for stochastic data sampling. Extensive experimental results on
various benchmark and real-world datasets also demonstrate that our method is
efficient and effective while retaining similar generalization performance.Comment: 12 pages, 9 figures, conferenc
Scaling-up Empirical Risk Minimization: Optimization of Incomplete U-statistics
In a wide range of statistical learning problems such as ranking, clustering
or metric learning among others, the risk is accurately estimated by
-statistics of degree , i.e. functionals of the training data with
low variance that take the form of averages over -tuples. From a
computational perspective, the calculation of such statistics is highly
expensive even for a moderate sample size , as it requires averaging
terms. This makes learning procedures relying on the optimization of
such data functionals hardly feasible in practice. It is the major goal of this
paper to show that, strikingly, such empirical risks can be replaced by
drastically computationally simpler Monte-Carlo estimates based on terms
only, usually referred to as incomplete -statistics, without damaging the
learning rate of Empirical Risk Minimization (ERM)
procedures. For this purpose, we establish uniform deviation results describing
the error made when approximating a -process by its incomplete version under
appropriate complexity assumptions. Extensions to model selection, fast rate
situations and various sampling techniques are also considered, as well as an
application to stochastic gradient descent for ERM. Finally, numerical examples
are displayed in order to provide strong empirical evidence that the approach
we promote largely surpasses more naive subsampling techniques.Comment: To appear in Journal of Machine Learning Research. 34 pages. v2:
minor correction to Theorem 4 and its proof, added 1 reference. v3: typo
corrected in Proposition 3. v4: improved presentation, added experiments on
model selection for clustering, fixed minor typo
Label Ranking with Probabilistic Models
Diese Arbeit konzentriert sich auf eine spezielle Prognoseform, das sogenannte Label Ranking. Auf den Punkt gebracht, kann Label Ranking als eine Erweiterung des herkömmlichen Klassifizierungproblems betrachtet werden. Bei einer Anfrage (z. B. durch einen Kunden) und einem vordefinierten Set von Kandidaten Labels (zB AUDI, BMW, VW), wird ein einzelnes Label (zB BMW) zur Vorhersage in der Klassifizierung benötigt, während ein komplettes Ranking aller Label (zB BMW> VW> Audi) für das Label Ranking erforderlich ist. Da Vorhersagen dieser Art, bei vielen Problemen der realen Welt nützlich sind, können Label Ranking-Methoden in mehreren Anwendungen, darunter Information Retrieval, Kundenwunsch Lernen und E-Commerce eingesetzt werden. Die vorliegende Arbeit stellt eine Auswahl an Methoden für Label-Ranking vor, die Maschinelles Lernen mit statistischen Bewertungsmodellen kombiniert.
Wir konzentrieren wir uns auf zwei statistische Ranking-Modelle, das Mallows- und das Plackett-Luce-Modell und zwei Techniken des maschinellen Lernens, das Beispielbasierte Lernen und das Verallgemeinernde Lineare Modell