Linear algebra with transformers

Transformers can learn to perform numerical computations from examples only. I study nine problems of linear algebra, from basic matrix operations to eigenvalue decomposition and inversion, and introduce and discuss four encoding schemes to represent real numbers. On all problems, transformers trained on sets of random matrices achieve high accuracies (over 90%). The models are robust to noise, and can generalize out of their training distribution. In particular, models trained to predict Laplace-distributed eigenvalues generalize to different classes of matrices: Wigner matrices or matrices with positive eigenvalues. The reverse is not true.Comment: Transactions in Machine Learning Research (TMLR), October 202

Learning advanced mathematical computations from examples

Using transformers over large generated datasets, we train models to learn mathematical properties of differential systems, such as local stability, behavior at infinity and controllability. We achieve near perfect prediction of qualitative characteristics, and good approximations of numerical features of the system. This demonstrates that neural networks can learn to perform complex computations, grounded in advanced theory, from examples, without built-in mathematical knowledge

An efficient algorithm for integer lattice reduction

A lattice of integers is the collection of all linear combinations of a set of vectors for which all entries of the vectors are integers and all coefficients in the linear combinations are also integers. Lattice reduction refers to the problem of finding a set of vectors in a given lattice such that the collection of all integer linear combinations of this subset is still the entire original lattice and so that the Euclidean norms of the subset are reduced. The present paper proposes simple, efficient iterations for lattice reduction which are guaranteed to reduce the Euclidean norms of the basis vectors (the vectors in the subset) monotonically during every iteration. Each iteration selects the basis vector for which projecting off (with integer coefficients) the components of the other basis vectors along the selected vector minimizes the Euclidean norms of the reduced basis vectors. Each iteration projects off the components along the selected basis vector and efficiently updates all information required for the next iteration to select its best basis vector and perform the associated projections.Comment: 29 pages, 20 figure

SALSA: Attacking Lattice Cryptography with Transformers

Currently deployed public-key cryptosystems will be vulnerable to attacks by full-scale quantum computers. Consequently, "quantum resistant" cryptosystems are in high demand, and lattice-based cryptosystems, based on a hard problem known as Learning With Errors (LWE), have emerged as strong contenders for standardization. In this work, we train transformers to perform modular arithmetic and combine half-trained models with statistical cryptanalysis techniques to propose SALSA: a machine learning attack on LWE-based cryptographic schemes. SALSA can fully recover secrets for small-to-mid size LWE instances with sparse binary secrets, and may scale to attack real-world LWE-based cryptosystems.Comment: Extended version of work published at NeurIPS 202

Faire coopérer des agents hétérogènes par apprentissage de médiation

Colloque avec actes et comité de lecture. nationale.National audienceDans le cadre de nos travaux sur la collaboration entre agents hétérogènes, nous avons étudié l'interaction entre un agent humain et un service de recherche d'informations. Nous avons proposé d'introduire un agent médiateur dont le rôle est de formaliser la requête d'un utilisateur, selon son profil et selon l'environnement, puis de lui donner un nombre restreint de réponses pertinentes. Nous cherchons à déterminer la meilleure stratégie de médiation (donc une séquence d'interactions) avec un modèle stochastique (MDP) dont les états sont construits sur un référentiel d'attributs et sur la capacité de la source de répondre à la requête en cours de formalisation. Les actions permettent de poser une question à l'utilisateur ou de sonder la source. Les récompenses reflètent la satisfaction de l'utilisateur, la longueur du dialogue et la quantité de résultats. Nous avons implanté un prototype de médiateur utilisant un apprentissage par renforcement (Q-Learning) car il permet une adaptation en ligne, sans modèle a priori. || In the framework of our works on collaboration between heterogeneous agents, we studied the interaction between a human agent and an information search service. We proposed to introduce a mediator agent to formalize the request of a user, according to hi

Evaluation de trois approches de thérapie génique pour le traitement des dysferlinopathies (miniprotéine, compensation et trans-épissage)

Les dysferlinopathies sont des maladies musculaires dues à une déficience en protéine dysferline, codée par le gène DYSF. Dans ce travail de thèse, trois approches thérapeutiques ont été évaluées pour ces pathologies, sur des modèles cellulaires et murins. Un variant transcriptionnel court de la dysferline a été vectorisé dans un AAV8r et injecté dans le modèle murin Bla/J, déficient en dysferline. L analyse des muscles des animaux traités montre une augmentation de la résistance des fibres musculaires au stress mécanique, mais n apporte pas de correction histologique. Cette étude souligne également la toxicité de cette miniprotéine. L anoctamine 5, impliquée dans des pathologies et des activités similaires à la dysferline, a été testée en tant que protéine compensatrice. L anoctamine 5 surexprimée dans le modèle Bla/J ne permet pas la restauration d un phénotype normal. La compensation de DYSF par ANO5 n est donc pas une voie thérapeutique à exploiter pour les dysferlinopathies. Enfin, une thérapie génique par chirurgie de l ARN dysferline a été évaluée en utilisant le trans-épissage médié par le splicéosome (SMaRT). La preuve de principe de la reprogrammation d un minigène dysferline a été faite (ARN et protéine trans-épissée obtenus in vitro). L efficacité du SMaRT dans un contexte endogène s est en revanche révélée faible, et n a pas permis la restauration d une protéine dysferline fonctionnelle dans des myoblastes humains. De plus, l observation de l expression de protéines directement à partir du RTM (RNA-trans-splicing molecule) a fait apparaître des limites à l utilisation du SMaRT pour la thérapie génique, et en particulier pour les dysferlinopathies.Dysferlinopathies are muscular diseases due to mutations in DYSF gene, inducing dysferlin protein deficiency. In this thesis, three therapeutic approaches have been investigated for these pathologies, on cell or mice models. A short transcriptional dysferlin variant has been injected into Bla/J dysferlin deficient mouse model, using AAV8r vector. Muscle fibers of treated animals displayed an increased resistance to mechanical stress without therapeutic benefit. These experiments also pointed out the toxicity of this strategy. A protein compensation approach has been tested using anoctamin 5, known to be involved in pathologies and activities similar to dysferlin s ones. AAVr mediated Anoctamin 5 overexpression in Bla/J model does not rescue their muscle phenotype. Overexpression of ANO5 does not seem to be a valuable therapeutic strategy for dysferlin deficiency. Dysferlin RNA surgery was evaluated as a possible genetic therapy using Spliceosome-Mediated RNA Trans-splicing (SMaRT). On a Minigene target, SMaRT is able to induce RNA reprogramming by trans-splicing, and produce the corresponding protein. But efficiency is by far decreased in endogenous context and not good enough to restore functional dysferlin in human myoblasts. Moreover, we described proteins resulting from RNA-trans-splicing molecule (RTM) self-expression, limiting the value of SMaRT as therapeutic strategy, especially for dysferlinopathies.EVRY-Bib. électronique (912289901) / SudocSudocFranceF

Providing users with adapted services: Dynamic building of dialogues to make heterogeneous agents cooperate

Colloque avec actes et comité de lecture. internationale.International audienceThis paper deals with a prospective way to allow the communication between heterogeneous agents in the framework of multimedia services. The focus is set on the interaction between the user of a service and his/her virtual assistant. The interaction takes the form of a dialogue where the assistant tries to identify the tasks that the current user wants to perform. After a short description of dialogue systems, we propose a way to adapt them for discovering the user's goals. Our approach to control the dialogue relies on Markov Decision Processes, which are particularly suitable to handle uncertainty that occurs at many points of the communication. We moreover collect knowledge about the users in order to build profiles that will be reused for future sessions

Length Generalization in Arithmetic Transformers

We examine how transformers cope with two challenges: learning basic integer arithmetic, and generalizing to longer sequences than seen during training. We find that relative position embeddings enable length generalization for simple tasks, such as addition: models trained on $5$-digit numbers can perform $15$-digit sums. However, this method fails for multiplication, and we propose train set priming: adding a few ($10$ to $50$) long sequences to the training set. We show that priming allows models trained on $5$-digit $\times$ $3$-digit multiplications to generalize to $35\times 3$ examples. We also show that models can be primed for different generalization lengths, and that the priming sample size scales as the logarithm of the training set size. Finally, we discuss potential applications of priming beyond arithmetic

Isolation and Characterization of New Leptospira Genotypes from Patients in Mayotte (Indian Ocean)

Leptospirosis has been recognized as an increasing public health problem affecting poor people from developing countries and tropical regions. However, the epidemiology of leptospirosis remains poorly understood in remote parts of the world. In this study of patients from the island of Mayotte, we isolated 22 strains from the blood of patients during the acute phase of illness. The pathogenic Leptospira strains were characterized by serology and various molecular typing methods. Based on serological data, serogroup Mini appears to be the dominant cause of leptospirosis in Mayotte. Further molecular characterization of these isolates allowed the identification of 10 pathogenic Leptospira genotypes that could correspond to previously unknown serovars. Further progress in our understanding of the epidemiology of Leptospira circulating genotypes in highly endemic regions should contribute to the development of novel strategies for the diagnosis and prevention of this neglected emerging disease