10 research outputs found
Simple, Efficient and Convenient Decentralized Multi-Task Learning for Neural Networks
Artificial intelligence relying on machine learning is increasingly used on small, personal, network-connected devices such as smartphones and vocal assistants, and these applications will likely evolve with the development of the Internet of Things. The learning process requires a lot of data, often real usersâ data, and computing power. Decentralized machine learning can help to protect usersâ privacy by keeping sensitive training data on usersâ devices, and has the potential to alleviate the cost born by service providers by off-loading some of the learning effort to user devices. Unfortunately, most approaches proposed so far for distributed learning with neural network are mono-task, and do not transfer easily to multi-tasks problems, for which users seek to solve related but distinct learning tasks and the few existing multi-task approaches have serious limitations. In this paper, we propose a novel learning method for neural networks that is decentralized, multitask, and keeps usersâ data local. Our approach works with different learning algorithms, on various types of neural networks. We formally analyze the convergence of our method, and we evaluateits efficiency in different situations on various kind of neural networks, with different learning algorithms, thus demonstrating its benefits in terms of learning quality and convergence
Foiling Sybils with HAPS in Permissionless Systems: An Address-based Peer Sampling Service
International audienceBlockchains and distributed ledgers have brought renewed interest in Byzantine fault-tolerant protocols and decentralized systems, two domains studied for several decades. Recent promising works have in particular proposed to use epidemic protocols to overcome the limitations of popular Blockchain mechanisms , such as proof-of-stake or proof-of-work. These works unfortunately assume a perfect peer-sampling service, immune to malicious attacks, a property that is difficult and costly to achieve. We revisit this fundamental problem in this paper, and propose a novel Byzantine-tolerant peer-sampling service that is resilient to Sybil attacks in open systems by exploiting the underlying structure of wide-area networks
Contributions to distributed multi-task machine learning
Lâapprentissage machine est un des domaines les plus importants et les plus actifs dans lâinformatique moderne. La plupart des systĂšmes dâapprentissage machine actuels utilisent encore une architecture essentiellement centralisĂ©e. MĂȘme si lâapplication finale doit ĂȘtre dĂ©livrĂ©e sur de nombreux systĂšmes, parfois des millions (voire des milliards) dâappareils individuels, le processus dâapprentissage est toujours centralisĂ© dans un centre de calcul. Ce peut ĂȘtre un problĂšme notamment si les donnĂ©es dâapprentissage sont sensibles, comme des conversations privĂ©es, des historiques de recherche ou des donnĂ©es mĂ©dicales. Dans cette thĂšse, nous nous intĂ©ressons au problĂšme de l'apprentissage machine distribuĂ© dans sa forme multitĂąche : une situation dans laquelle diffĂ©rents utilisateurs d'un mĂȘme systĂšme d'apprentissage machine ont des tĂąches similaires, mais diffĂ©rentes, Ă apprendre, ce qui correspond Ă des applications majeures de l'apprentissage machine moderne, comme la reconnaissance de l'Ă©criture ou de la parole. Nous proposons tout d'abord le concept d'un systĂšme d'apprentissage machine distribuĂ© multitĂąche pour les rĂ©seaux de neurones. Ensuite, nous proposons une mĂ©thode permettant d'optimiser automatiquement le processus d'apprentissage en identifiant les tĂąches les plus similaires. Enfin, nous Ă©tudions comment nos propositions correspondent aux intĂ©rĂȘts individuels des utilisateurs.Machine learning is one of the most important and active fields in present computer science. Currently, most machine learning systems are still using a mainly centralized design. Even when the final application is to be delivered in several systems, potentially millions (and even billions) of personal devices, the learning process is still centralized in a large datacenter. This can be an issue if the training data is sensitive, like private conversations, browsing histories, or health-related data. In this thesis, we tackle the problem of distributed machine learning in its multi-task form: a situation where different users of a common machine learning system have similar but different tasks to learn, which corresponds to major modern applications of machine learning, such as handwriting recognition or speech recognition. We start by proposing a design of an effective distributed multi-task machine learning system for neural networks. We then propose a method to automatically optimize the learning process based on which tasks are more similar than others. Finally, we study how our propositions fit the individual interests of users
Contributions Ă lâapprentissage machine distribuĂ© multitĂąche
Machine learning is one of the most important and active fields in present computer science. Currently, most machine learning systems are still using a mainly centralized design. Even when the final application is to be delivered in several systems, potentially millions (and even billions) of personal devices, the learning process is still centralized in a large datacenter. This can be an issue if the training data is sensitive, like private conversations, browsing histories, or health-related data. In this thesis, we tackle the problem of distributed machine learning in its multi-task form: a situation where different users of a common machine learning system have similar but different tasks to learn, which corresponds to major modern applications of machine learning, such as handwriting recognition or speech recognition. We start by proposing a design of an effective distributed multi-task machine learning system for neural networks. We then propose a method to automatically optimize the learning process based on which tasks are more similar than others. Finally, we study how our propositions fit the individual interests of users.Lâapprentissage machine est un des domaines les plus importants et les plus actifs dans lâinformatique moderne. La plupart des systĂšmes dâapprentissage machine actuels utilisent encore une architecture essentiellement centralisĂ©e. MĂȘme si lâapplication finale doit ĂȘtre dĂ©livrĂ©e sur de nombreux systĂšmes, parfois des millions (voire des milliards) dâappareils individuels, le processus dâapprentissage est toujours centralisĂ© dans un centre de calcul. Ce peut ĂȘtre un problĂšme notamment si les donnĂ©es dâapprentissage sont sensibles, comme des conversations privĂ©es, des historiques de recherche ou des donnĂ©es mĂ©dicales. Dans cette thĂšse, nous nous intĂ©ressons au problĂšme de l'apprentissage machine distribuĂ© dans sa forme multitĂąche : une situation dans laquelle diffĂ©rents utilisateurs d'un mĂȘme systĂšme d'apprentissage machine ont des tĂąches similaires, mais diffĂ©rentes, Ă apprendre, ce qui correspond Ă des applications majeures de l'apprentissage machine moderne, comme la reconnaissance de l'Ă©criture ou de la parole. Nous proposons tout d'abord le concept d'un systĂšme d'apprentissage machine distribuĂ© multitĂąche pour les rĂ©seaux de neurones. Ensuite, nous proposons une mĂ©thode permettant d'optimiser automatiquement le processus d'apprentissage en identifiant les tĂąches les plus similaires. Enfin, nous Ă©tudions comment nos propositions correspondent aux intĂ©rĂȘts individuels des utilisateurs
Contributions Ă lâapprentissage machine distribuĂ© multitĂąche
Machine learning is one of the most important and active fields in present computer science. Currently, most machine learning systems are still using a mainly centralized design. Even when the final application is to be delivered in several systems, potentially millions (and even billions) of personal devices, the learning process is still centralized in a large datacenter. This can be an issue if the training data is sensitive, like private conversations, browsing histories, or health-related data. In this thesis, we tackle the problem of distributed machine learning in its multi-task form: a situation where different users of a common machine learning system have similar but different tasks to learn, which corresponds to major modern applications of machine learning, such as handwriting recognition or speech recognition. We start by proposing a design of an effective distributed multi-task machine learning system for neural networks. We then propose a method to automatically optimize the learning process based on which tasks are more similar than others. Finally, we study how our propositions fit the individual interests of users.Lâapprentissage machine est un des domaines les plus importants et les plus actifs dans lâinformatique moderne. La plupart des systĂšmes dâapprentissage machine actuels utilisent encore une architecture essentiellement centralisĂ©e. MĂȘme si lâapplication finale doit ĂȘtre dĂ©livrĂ©e sur de nombreux systĂšmes, parfois des millions (voire des milliards) dâappareils individuels, le processus dâapprentissage est toujours centralisĂ© dans un centre de calcul. Ce peut ĂȘtre un problĂšme notamment si les donnĂ©es dâapprentissage sont sensibles, comme des conversations privĂ©es, des historiques de recherche ou des donnĂ©es mĂ©dicales. Dans cette thĂšse, nous nous intĂ©ressons au problĂšme de l'apprentissage machine distribuĂ© dans sa forme multitĂąche : une situation dans laquelle diffĂ©rents utilisateurs d'un mĂȘme systĂšme d'apprentissage machine ont des tĂąches similaires, mais diffĂ©rentes, Ă apprendre, ce qui correspond Ă des applications majeures de l'apprentissage machine moderne, comme la reconnaissance de l'Ă©criture ou de la parole. Nous proposons tout d'abord le concept d'un systĂšme d'apprentissage machine distribuĂ© multitĂąche pour les rĂ©seaux de neurones. Ensuite, nous proposons une mĂ©thode permettant d'optimiser automatiquement le processus d'apprentissage en identifiant les tĂąches les plus similaires. Enfin, nous Ă©tudions comment nos propositions correspondent aux intĂ©rĂȘts individuels des utilisateurs
Contributions Ă lâapprentissage machine distribuĂ© multitĂąche
Machine learning is one of the most important and active fields in present computer science. Currently, most machine learning systems are still using a mainly centralized design. Even when the final application is to be delivered in several systems, potentially millions (and even billions) of personal devices, the learning process is still centralized in a large datacenter. This can be an issue if the training data is sensitive, like private conversations, browsing histories, or health-related data. In this thesis, we tackle the problem of distributed machine learning in its multi-task form: a situation where different users of a common machine learning system have similar but different tasks to learn, which corresponds to major modern applications of machine learning, such as handwriting recognition or speech recognition. We start by proposing a design of an effective distributed multi-task machine learning system for neural networks. We then propose a method to automatically optimize the learning process based on which tasks are more similar than others. Finally, we study how our propositions fit the individual interests of users.Lâapprentissage machine est un des domaines les plus importants et les plus actifs dans lâinformatique moderne. La plupart des systĂšmes dâapprentissage machine actuels utilisent encore une architecture essentiellement centralisĂ©e. MĂȘme si lâapplication finale doit ĂȘtre dĂ©livrĂ©e sur de nombreux systĂšmes, parfois des millions (voire des milliards) dâappareils individuels, le processus dâapprentissage est toujours centralisĂ© dans un centre de calcul. Ce peut ĂȘtre un problĂšme notamment si les donnĂ©es dâapprentissage sont sensibles, comme des conversations privĂ©es, des historiques de recherche ou des donnĂ©es mĂ©dicales. Dans cette thĂšse, nous nous intĂ©ressons au problĂšme de l'apprentissage machine distribuĂ© dans sa forme multitĂąche : une situation dans laquelle diffĂ©rents utilisateurs d'un mĂȘme systĂšme d'apprentissage machine ont des tĂąches similaires, mais diffĂ©rentes, Ă apprendre, ce qui correspond Ă des applications majeures de l'apprentissage machine moderne, comme la reconnaissance de l'Ă©criture ou de la parole. Nous proposons tout d'abord le concept d'un systĂšme d'apprentissage machine distribuĂ© multitĂąche pour les rĂ©seaux de neurones. Ensuite, nous proposons une mĂ©thode permettant d'optimiser automatiquement le processus d'apprentissage en identifiant les tĂąches les plus similaires. Enfin, nous Ă©tudions comment nos propositions correspondent aux intĂ©rĂȘts individuels des utilisateurs
Contributions Ă lâapprentissage machine distribuĂ© multitĂąche
Machine learning is one of the most important and active fields in present computer science. Currently, most machine learning systems are still using a mainly centralized design. Even when the final application is to be delivered in several systems, potentially millions (and even billions) of personal devices, the learning process is still centralized in a large datacenter. This can be an issue if the training data is sensitive, like private conversations, browsing histories, or health-related data. In this thesis, we tackle the problem of distributed machine learning in its multi-task form: a situation where different users of a common machine learning system have similar but different tasks to learn, which corresponds to major modern applications of machine learning, such as handwriting recognition or speech recognition. We start by proposing a design of an effective distributed multi-task machine learning system for neural networks. We then propose a method to automatically optimize the learning process based on which tasks are more similar than others. Finally, we study how our propositions fit the individual interests of users.Lâapprentissage machine est un des domaines les plus importants et les plus actifs dans lâinformatique moderne. La plupart des systĂšmes dâapprentissage machine actuels utilisent encore une architecture essentiellement centralisĂ©e. MĂȘme si lâapplication finale doit ĂȘtre dĂ©livrĂ©e sur de nombreux systĂšmes, parfois des millions (voire des milliards) dâappareils individuels, le processus dâapprentissage est toujours centralisĂ© dans un centre de calcul. Ce peut ĂȘtre un problĂšme notamment si les donnĂ©es dâapprentissage sont sensibles, comme des conversations privĂ©es, des historiques de recherche ou des donnĂ©es mĂ©dicales. Dans cette thĂšse, nous nous intĂ©ressons au problĂšme de l'apprentissage machine distribuĂ© dans sa forme multitĂąche : une situation dans laquelle diffĂ©rents utilisateurs d'un mĂȘme systĂšme d'apprentissage machine ont des tĂąches similaires, mais diffĂ©rentes, Ă apprendre, ce qui correspond Ă des applications majeures de l'apprentissage machine moderne, comme la reconnaissance de l'Ă©criture ou de la parole. Nous proposons tout d'abord le concept d'un systĂšme d'apprentissage machine distribuĂ© multitĂąche pour les rĂ©seaux de neurones. Ensuite, nous proposons une mĂ©thode permettant d'optimiser automatiquement le processus d'apprentissage en identifiant les tĂąches les plus similaires. Enfin, nous Ă©tudions comment nos propositions correspondent aux intĂ©rĂȘts individuels des utilisateurs
Robust Privacy-Preserving Gossip Averaging
International audienc
AUCCCR: Agent Utility Centered Clustering for Cooperation Recommendation
International audienceProviding recommendation to agents (e.g. people or organizations) regarding whom they should collaborate with in order to reach some objective is a recurring problem in a wide range of domains. It can be useful for instance in the context of collaborative machine learning, grouped purchases, and group holidays. This problem has been modeled by hedonic games, but this generic formulation cannot easily be used to provide efficient algorithmic solutions. In this work, we define a class of hedonist games that allows us to provide an algorithmic solution to the collaboration recommendation problem by means of a clustering algorithm. We evaluate our algorithm, theoretically and experimentally and show that it performs better than other clustering algorithms in this context
Simple, Efficient and Convenient Decentralized Multi-Task Learning for Neural Networks
International audienceMachine learning requires large amounts of data, which is increasingly distributed over many systems (user devices, independent storage systems). Unfortunately aggregating this data in one site for learning is not always practical, either because of network costs or privacy concerns. Decentralized machine learning holds the potential to address these concerns, but unfortunately, most approaches proposed so far for distributed learning with neural network are mono-task, and do not transfer easily to multi-tasks problems, for which users seek to solve related but distinct learning tasks and the few existing multi-task approaches have serious limitations. In this paper, we propose a novel learning method for neural networks that is decentralized, multi-task, and keeps users' data local. Our approach works with different learning algorithms, on various types of neural networks. We formally analyze the convergence of our method, and we evaluate its efficiency in different situations on various kind of neural networks, with different learning algorithms, thus demonstrating its benefits in terms of learning quality and convergence