Similarity Learning for Provably Accurate Sparse Linear Classification

In recent years, the crucial importance of metrics in machine learning algorithms has led to an increasing interest for optimizing distance and similarity functions. Most of the state of the art focus on learning Mahalanobis distances (requiring to fulfill a constraint of positive semi-definiteness) for use in a local k-NN algorithm. However, no theoretical link is established between the learned metrics and their performance in classification. In this paper, we make use of the formal framework of good similarities introduced by Balcan et al. to design an algorithm for learning a non PSD linear similarity optimized in a nonlinear feature space, which is then used to build a global linear classifier. We show that our approach has uniform stability and derive a generalization bound on the classification error. Experiments performed on various datasets confirm the effectiveness of our approach compared to state-of-the-art methods and provide evidence that (i) it is fast, (ii) robust to overfitting and (iii) produces very sparse classifiers.Comment: Appears in Proceedings of the 29th International Conference on Machine Learning (ICML 2012

A Survey on Metric Learning for Feature Vectors and Structured Data

The need for appropriate ways to measure the distance or similarity between data is ubiquitous in machine learning, pattern recognition and data mining, but handcrafting such good metrics for specific problems is generally difficult. This has led to the emergence of metric learning, which aims at automatically learning a metric from data and has attracted a lot of interest in machine learning and related fields for the past ten years. This survey paper proposes a systematic review of the metric learning literature, highlighting the pros and cons of each approach. We pay particular attention to Mahalanobis distance metric learning, a well-studied and successful framework, but additionally present a wide range of methods that have recently emerged as powerful alternatives, including nonlinear metric learning, similarity learning and local metric learning. Recent trends and extensions, such as semi-supervised metric learning, metric learning for histogram data and the derivation of generalization guarantees, are also covered. Finally, this survey addresses metric learning for structured data, in particular edit distance learning, and attempts to give an overview of the remaining challenges in metric learning for the years to come.Comment: Technical report, 59 pages. Changes in v2: fixed typos and improved presentation. Changes in v3: fixed typos. Changes in v4: fixed typos and new method

Decentralized Collaborative Learning of Personalized Models over Networks

We consider a set of learning agents in a col-laborative peer-to-peer network, where each agent learns a personalized model according to its own learning objective. The question addressed in this paper is: how can agents improve upon their locally trained model by communicating with other agents that have similar objectives? We introduce and analyze two asynchronous gossip algorithms running in a fully decentralized manner. Our first approach , inspired from label propagation, aims to smooth pre-trained local models over the network while accounting for the confidence that each agent has in its initial model. In our second approach, agents jointly learn and propagate their model by making iterative updates based on both their local dataset and the behavior of their neighbors. Our algorithm to optimize this challenging objective in a decentralized way is based on ADMM

Good edit similarity learning by loss minimization

International audienceSimilarity functions are a fundamental component of many learning algorithms. When dealing with string or tree-structured data, edit distancebased measures are widely used, and there exists a few methods for learning them from data. However, these methods offer no theoretical guarantee as to the generalization ability and discriminative power of the learned similarities. In this paper, we propose a loss minimization-based edit similarity learning approach, called GESL. It is driven by the notion of (e, γ, τ )-goodness, a theory that bridges the gap between the properties of a similarity function and its performance in classification. We show that our learning framework is a suitable way to deal not only with strings but also with tree-structured data. Using the notion of uniform stability, we derive generalization guarantees for a large class of loss functions. We also provide experimental results on two realworld datasets which show that edit similarities learned with GESL induce more accurate and sparser classifiers than other (standard or learned) edit similarities

Apprentissage de bonnes similarités pour la classification linéaire parcimonieuse

http://cap2012.loria.fr/pub/Papers/28.pdfNational audienceLe rôle crucial joué par les métriques au sein des processus d'apprentissage automatique a donné lieu ces dernières années à un intérêt croissant pour l'optimisation de fonctions de distances ou de similarités. La plupart des approches de l'état de l'art visent à apprendre une distance de Mahalanobis, devant satisfaire la contrainte de semi-définie positivité (SDP), exploitée in fine dans un algorithme local de type plus-proches-voisins. Cependant, aucun résultat théorique n'établit le lien entre les métriques apprises et leur comportement en classification. Dans cet article, nous exploitons le cadre formel des bonnes similarités pour proposer un algorithme d'apprentissage de similarité linéaire, optimisée dans un espace kernélisé. Nous montrons que la similarité apprise, ne requérant pas d'être SDP, possède des propriétés théoriques de stabilité permettant d'établir une borne en généralisation. Les expérimentations menées sur plusieurs jeux de données confirment son efficacité par rapport à l'état de l'art

Vote de majorité a priori contraint pour la classification binaire : spécification au cas des plus proches voisins

National audiencePour combiner différents classifieurs/votants, une solution naturelle vise à construire un vote de majorité. Un algorithme récemment introduit, MinCq, apprend un tel vote en optimisant les poids associés aux votants. Son principe repose sur la minimisation du risque du vote de majorité (la C-borne), dans le cadre de la théorie PAC-Bayes. Une limite de MinCq vient du fait qu'il ne peut tirer avantage d'une connaissance a priori sur la performance des votants (comme cela peut être le cas des classifieurs de type plus proches voisins (PPV)). Dans cet article, nous introduisons P-MinCq, une extension de MinCq, afin de considérer une contrainte a priori sur la distribution des poids des votants. Cette contrainte pouvant dépendre des exemples d'apprentissage, nous généralisons les preuves de convergence aux schémas de compression. Appliqué à un vote de majorité sur un ensemble de classifieurs PPV et évalué sur vingt jeux de données, nous montrons que P-MinCq est significativement plus performant qu'un PPV classique, un PPV symétrique et MinCq lui-même. Nous montrons finalement que combiné avec LMNN, un algorithme d'apprentissage de métrique, P-MinCq permet d'obtenir des résultats encore meilleurs

High-Dimensional Private Empirical Risk Minimization by Greedy Coordinate Descent

In this paper, we study differentially private empirical risk minimization (DP-ERM). It has been shown that the worst-case utility of DP-ERM reduces polynomially as the dimension increases. This is a major obstacle to privately learning large machine learning models. In high dimension, it is common for some model's parameters to carry more information than others. To exploit this, we propose a differentially private greedy coordinate descent (DP-GCD) algorithm. At each iteration, DP-GCD privately performs a coordinate-wise gradient step along the gradients' (approximately) greatest entry. We show theoretically that DP-GCD can achieve a logarithmic dependence on the dimension for a wide range of problems by naturally exploiting their structural properties (such as quasi-sparse solutions). We illustrate this behavior numerically, both on synthetic and real datasets

Structure development during polymer processing. Morphological and crystallographic textures in polyethylene blown films

La publication originale est disponible sur le site http://www.revue-metallurgie.orgInternational audienceThe crystallographic texture of a great number of polyethylene films manufactured by the film blowing process has been investigated by X-ray diffraction. Some films present a classical texture (c-axis in the machine direction), which can be interpreted using existing morphological models. Other exhibit an original texture (c-axis in the film thickness) and no satisfactory morphological model has been proposed until now

Modelling of the compaction phase during Hot Isostatic Pressing process at the mesoscopic scale

International audienceDuring Hot isostatic pressing (HIP) of metal powder, power-law creep is the dominant mechanism during the densification process. However, the understanding of the global impact of the thermo-mechanical boundary conditions and of the powder granulometry on the microstructure obtained after this first mechanism is not straightforward. A finite element methodology based on the use of a level set framework coupled with a remeshing technique is proposed in order to model the viscoplastic deformation of powder particles during HIP at the mesoscopic scale thanks to a Representative Elementary Volume. The methodology consists in generating, in a finite element mesh, a sphere packing of particles by representing implicitly all particles by means of a limited set of level-set functions. Mesh adaptation is also performed at particle boundaries to describe properly the particles and to manage the discontinuity of the physical properties. Such 2D scale mesoscopic densification simulations are presented and discusse

Relationships between processing conditions and mechanical properties of PA12 tubes

International audienceIn polyamide 12 (PA12) tube extrusion, calibration is the key step of the process that affects the subsequent mechanical properties. In previous work it has been shown that according to the calibration conditions, a very oriented skin layer may be created, which has been correlated to an important decrease of elongation at break. In this paper, we present new results showing a good correlation between molecular orientation and fracture toughness, as evaluated by the EWF (Essential Work of Fracture) approach. They concern notched specimens and confirm the results obtained in classical tensile testing. EWF is very sensitive to processing conditions, and especially to induced orientation: it decreases from the external to the inner regions of the tube, and increases with skin orientation