The Impact of LoRA on the Emergence of Clusters in Transformers

In this paper, we employ the mathematical framework on Transformers developed by \citet{sander2022sinkformers,geshkovski2023emergence,geshkovski2023mathematical} to explore how variations in attention parameters and initial token values impact the structural dynamics of token clusters. Our analysis demonstrates that while the clusters within a modified attention matrix dynamics can exhibit significant divergence from the original over extended periods, they maintain close similarities over shorter intervals, depending on the parameter differences. This work contributes to the fine-tuning field through practical applications to the LoRA algorithm \cite{hu2021lora,peft}, enhancing our understanding of the behavior of LoRA-enhanced Transformer models

Practical considerations on using private sampling for synthetic data

Artificial intelligence and data access are already mainstream. One of the main challenges when designing an artificial intelligence or disclosing content from a database is preserving the privacy of individuals who participate in the process. Differential privacy for synthetic data generation has received much attention due to the ability of preserving privacy while freely using the synthetic data. Private sampling is the first noise-free method to construct differentially private synthetic data with rigorous bounds for privacy and accuracy. However, this synthetic data generation method comes with constraints which seem unrealistic and not applicable for real-world datasets. In this paper, we provide an implementation of the private sampling algorithm and discuss the realism of its constraints in practical cases

Planification multi-agents multi-objectifs (modèle et algorithme)

Cette thèse s'intéresse à la problématique de la coordination de plusieurs agents autonomes dans un environnement réel. Cela implique la prise en compte de l'incertitude dans la réalisation des actions et du comportement des autres agents, ainsi que d'une certaine dynamicité de l'environnement. Nous avons basé notre travail sur le formalisme des processus décisionnels de Markov (MDP) qui permet d'intégrer les incertitudes dans le processus de raisonnement. Afin de prendre en compte les interactions avec les autres agents, nous avons formalisé celles-ci et intégré les interactions au sein d'un processus de décision en ligne. Ce processus est une extension des MDP où les agents cherchent à optimiser leurs gains personnels, ainsi que le bien-être du groupe. Il en découle un problème de décision multi-critères, auquel nous avons proposé une solution. Une fois ce formalisme établi, nous avons pu aborder plusieurs problèmes de coordination comme : la formation de convois, la couverture spatiale et la formation de coalitions. Ces problèmes nous ont permis d'appliquer avec succès les principes établis en début de thèse. Les extensions de ce travail traiteront l'apprentissage en ligne, et la théorie des jeux afin de permettre la détection et la résolution de cas d'inter-blocagesThis thesis deals with the coordination of a group of autonomous agents in the real world. So, we have to take into account uncertainty about action's outcome, about other agent's behavior and also the changes in the environment. We are using Markov decision processes (MDP), whose allow to manage those uncertainties in a decision process. In order to manage the interactions with the other agents, we give a formalism to express them, and also we give a solution to integrate them in a on-line decision process.This is an extension of the Markov Decision Processes where the agent are trying to optimize their own reward as well as the welfare of the group. This is a mutlicriteria decision problem, and we give it a solution. Once this formalism built, we tackle some classical coordination problems : platooning, spatial coverage, coalitions formation. Those applications allow us to apply with success the principle given at the beginning of the thesis. The extensions of this work will be dealing with on-line learning, and also game theory in order to detect and to solve deadlocks.CAEN-BU Sciences et STAPS (141182103) / SudocSudocFranceF

La décision multi-critère pour la coordination locale dans les systèmes multi-agents

A la différences des systèmes mono-agent, la plani- fication multi-agent doit résoudre des conflits entre les intérêts individuels d'un agent et l'intérêt du groupe. Dans cet article, nous utilisons un processus décisionnel de Markov décentralisé valué par des vecteurs (2V-DEC-MDP) en vue de résoudre ce problème. Le cadre formel considéré, celui des MDP à valuation vectorielle, utilise une fonction de valeur qui retourne un vecteur représentant à la fois l'intérêt personnel et l'intérêt du groupe. L'intérêts individuel d'un agent, calculé hors-ligne, repose sur sa politique optimale . L'intérêt du groupe est calculé en ligne par les agents à partir de leurs observations locales. Afin de tenir compte de ces deux critères dans un processus de décision, nous avons développé un algorithme basée sur le regret à partir de la norme de Tchebychev. L'objectif est de trouver un bon compromis entre l'intérêt du groupe et celui de l'agent. Ces résultats sont illustrés par un exemple. In spite of mono-agent systems, multi-agent planing addresses the problem of resolving conflicts between individual interests and group interest. In this paper, we are using a Decentralized Vector Valued Markov Decision Process (2V-DEC-MDP) in order to solve this problem. This formal framework, the Vector valued MDP, uses an utility function which is returning a vector representing both individual interest and group interest. The individual interest of an agent, computed off-line, is based on is optimal policy. Group interest is computed on-line by the agent using local observations. In order to take into account both criteria in a decision process, we develop a regret-based algorithm from the Tchebychev Norm. The goal is to find a good trade-off between the group interest and the agent one. This results are illustrated by an example

Collective decision-theoretic planning for robot platoon formation

National audienc

Collective Decision-Theoretic Planning for Robot Platton Formation

International audienceThe robot platooning problem has been studied extensively by the robotics community under some assumptions such as communication existance and global full observability. In this paper, we consider the platooning problem where the previous assumptions are not valid. In such a context, platooning can be considered as a specific flocking which is a collective decision model. This model can, thus, be seen as a decentralized multi-criteria decision making process. Vector-Valued Decentralized Markov Decision Process (2V-DEC-MDP) is an interesting framework for multi-criteria collective decision. It has been shown that 2V-DEC-MDP does not consider communication, local interactions and use local full observability which is a sub-class of partial observability. In this paper, we adapt this framework to consider the notion of leader and the relationship with the stochastic games. The theoretic concept of optimality used in such contexts is the Stackelberg Equilibrium (SE). We give the assumptions under which the leader follows the SE when using 2V-DEC-MDP. We present, then, the adaptation of the initial value functions of the 2V-DEC-MDP, in order to reach an SE. Experiments shown us that using the initial 2V-DEC-MDP leads to a near SE with a weak complexity while the adapted 2V-DEC-MDP leads to a SE with a very high complexity and thus a limited scalability which limits its applicability in real-life robotic applications

Rényi Pufferfish Privacy: General Additive Noise Mechanisms and Privacy Amplification by Iteration via Shift Reduction Lemmas

Pufferfish privacy is a flexible generalization of differential privacy that allows to model arbitrary secrets and adversary's prior knowledge about the data. Unfortunately, designing general and tractable Pufferfish mechanisms that do not compromise utility is challenging. Furthermore, this framework does not provide the composition guarantees needed for a direct use in iterative machine learning algorithms. To mitigate these issues, we introduce a Rényi divergence-based variant of Pufferfish and show that it allows us to extend the applicability of the Pufferfish framework. We first generalize the Wasserstein mechanism to cover a wide range of noise distributions and introduce several ways to improve its utility. We also derive stronger guarantees against out-ofdistribution adversaries. Finally, as an alternative to composition, we prove privacy amplification results for contractive noisy iterations and showcase the first use of Pufferfish in private convex optimization. A common ingredient underlying our results is the use and extension of shift reduction lemmas