Stochastic Parrots Looking for Stochastic Parrots: LLMs are Easy to Fine-Tune and Hard to Detect with other LLMs

The self-attention revolution allowed generative language models to scale and achieve increasingly impressive abilities. Such models - commonly referred to as Large Language Models (LLMs) - have recently gained prominence with the general public, thanks to conversational fine-tuning, putting their behavior in line with public expectations regarding AI. This prominence amplified prior concerns regarding the misuse of LLMs and led to the emergence of numerous tools to detect LLMs in the wild. Unfortunately, most such tools are critically flawed. While major publications in the LLM detectability field suggested that LLMs were easy to detect with fine-tuned autoencoders, the limitations of their results are easy to overlook. Specifically, they assumed publicly available generative models without fine-tunes or non-trivial prompts. While the importance of these assumptions has been demonstrated, until now, it remained unclear how well such detection could be countered. Here, we show that an attacker with access to such detectors' reference human texts and output not only evades detection but can fully frustrate the detector training - with a reasonable budget and all its outputs labeled as such. Achieving it required combining common "reinforcement from critic" loss function modification and AdamW optimizer, which led to surprisingly good fine-tuning generalization. Finally, we warn against the temptation to transpose the conclusions obtained in RNN-driven text GANs to LLMs due to their better representative ability. These results have critical implications for the detection and prevention of malicious use of generative language models, and we hope they will aid the designers of generative models and detectors.Comment: 15 pages, 6 figures; 10 pages, 7 figures Supplementary Materials; under review at ECML 202

Byzantine-Resilient Learning Beyond Gradients: Distributing Evolutionary Search

Modern machine learning (ML) models are capable of impressive performances. However, their prowess is not due only to the improvements in their architecture and training algorithms but also to a drastic increase in computational power used to train them. Such a drastic increase led to a growing interest in distributed ML, which in turn made worker failures and adversarial attacks an increasingly pressing concern. While distributed byzantine resilient algorithms have been proposed in a differentiable setting, none exist in a gradient-free setting. The goal of this work is to address this shortcoming. For that, we introduce a more general definition of byzantine-resilience in ML - the \textit{model-consensus}, that extends the definition of the classical distributed consensus. We then leverage this definition to show that a general class of gradient-free ML algorithms - ($1,\lambda$)-Evolutionary Search - can be combined with classical distributed consensus algorithms to generate gradient-free byzantine-resilient distributed learning algorithms. We provide proofs and pseudo-code for two specific cases - the Total Order Broadcast and proof-of-work leader election.Comment: 10 pages, 4 listings, 2 theorem

Mécanismes moléculaires de la résistance au stress induite par l'aneuploïdie

Aneuploidy has historically been associated with detrimental phenotypes and diseases, notably cancer and Down Syndrome. However, recent experimental evidence suggests aneuploidy provides adaptation to numerous stressors, including drug resistance, making aneuploidy study critical to biomedical research. However, the molecular mechanisms underlying this process remained elusive until now. This work focused on exploring several approaches to understanding those mechanisms. Frist, we have developed a general mathematical model of organism adaptation to adverse environments. In our model, the adaptation to environments takes place as a trade-off in the space of traits, of which aneuploidy allows a more efficient and rapid sampling. This model was validated on experimental data and used to predict optimal drug combinations targeting heterogeneous populations breast tumor cells. Second, we used the framework of network biology to model biomolecular networks and apply to them results from the graph theory and existing results on weighted graphs from other domains. We were able to predict the distribution of essential genes, lethal genetic interactions and essential evolvable genes - essential genes that can be deleted in the aneuploid background. We were as well able to build a predictive model for inferring most likely pathways underlying the phenotype of large-scale genetic perturbations. Finally, we attempted to explore several possible modes besides dosage effects by which aneuploidy could impact the gene expression regulation. This required a development of an image analysis toolkit that was validated and released for as open-source software.L’aneuploïdie a été historiquement associé à des phénotypes nuisibles, notamment le cancer et le syndrome de Down. Cependant, des résultats expérimentaux récents suggèrent que l’aneuploïdie permettrait l'adaptation à des stresseurs variés, notamment résistance aux médicaments, en rendant la compréhension critique au domaine biomédical. Cependant, les mécanismes moléculaires permettant cette adaptation restaient à élucider. Une telle élucidation selon plusieurs axes a été justement l'objet de ce travail. Premièrement, nous avons développé un modèle mathématique représentant l'adaptation aux environnements adverses comme un compromis dans la position dans un espace des traits. L’aneuploïdie y permet une exploration plus rapide. Ce modèle a été validé sur des données expérimentaux et a été utilisé pour prédire une combinaison médicamenteuse ciblant les populations cellulaires hétérogènes dans le cancer du sein. Deuxièmement, nous avons utilisé les concepts du domaine de la biologie en réseaux et des résultats de théorie de graphes pour prédire la distribution des gènes essentiels, des interactions létales et des gènes essentiels évolutifs - des gènes essentiels qui peuvent être supprimés dans des organismes devenus aneuploïdes. Nous avons également construit un algorithme pour prédire les mécanismes moléculaires qui expliquerait les phénotypes associés à des perturbations génétiques à grande échelle. Finalement, nous avons exploré plusieurs mécanismes par lesquels l’aneuploïdie pourrait impacter la régulation génétique, conduisant au développement des outil informatiques publiés

Byzantine-Resilient learning beyond gradients ::distributing evolutionary search

Modern machine learning (ML) models are capable of impressive performances. However, their prowess is not due only to the improvements in their architecture and training algorithms but also to a drastic increase in computational power used to train them. Such a drastic increase led to a growing interest in distributed ML, which in turn made worker failures and adversarial attacks an increasingly pressing concern. While distributed byzantine resilient algorithms have been proposed in a differentiable setting, none exist in a gradient-free setting. The goal of this work is to address this shortcoming. For that, we introduce a more general definition of byzantine-resilience in ML- the model-consensus, that extends the definition of the classical distributed consensus. We then leverage this definition to show that a general class of gradient-free ML algorithms - (1, )-Evolutionary Search - can be combined with classical distributed consensus algorithms to generate gradient-free byzantine-resilient distributed learning algorithms. We provide proofs and pseudo-code for two specific cases - the Total Order Broadcast and proof-of-work leader election. To our knowledge, this is the first time a byzantine resilience in gradient-free ML was defined, and algorithms to achieve it – were propose

Evolutionary algorithms in the light of SGD ::limit equivalence, minima flatness, and transfer learning

Whenever applicable, the Stochastic Gradient Descent (SGD) has shown itself to be unreasonably effective. Instead of underperforming and getting trapped in local minima due to the batch noise, SGD leverages it to learn to generalize better and find minima that are good enough for the entire dataset. This led to numerous theoretical and experimental investigations, especially in the context of Artificial Neural Networks (ANNs), leading to better machine learning algorithms. However, SGD is not applicable in a non-differentiable setting, leaving all that prior research off the table. In this paper, we show that a class of evolutionary algorithms (EAs) inspired by the Gillespie-Orr Mutational Landscapes model for natural evolution is formally equivalent to SGD in certain settings and, in practice, is well adapted to large ANNs. We refer to such EAs as Gillespie-Orr EA class (GO-EAs) and empirically show how an insight transfer from SGD can work for them. We then show that for ANNs trained to near-optimality or in the transfer learning setting, the equivalence also allows transferring the insights from the Mutational Landscapes model to SGD. We then leverage this equivalence to experimentally show how SGD and GO-EAs can provide mutual insight through examples of minima flatness, transfer learning, and mixing of individuals in EAs applied to large models

LLM-based entity extraction is not for cybersecurity

The cybersecurity landscape evolves rapidly and poses threats to organizations. To enhance resilience, one needs to track the latest developments and trends in the domain. For this purpose, we use large language models (LLMs) to extract relevant knowledge entities from cybersecurity-related texts. We use a subset of arXiv preprints on cybersecurity as our data and compare different LLMs in terms of entity recognition (ER) and relevance. The results suggest that LLMs do not produce good knowledge entities that reflect the cybersecurity context

Fundamentals of Generative Large Language Models and Perspectives in Cyber-Defense

Generative Language Models gained significant attention in late 2022 / early 2023, notably with the introduction of models refined to act consistently with users' expectations of interactions with AI (conversational models). Arguably the focal point of public attention has been such a refinement of the GPT3 model -- the ChatGPT and its subsequent integration with auxiliary capabilities, including search as part of Microsoft Bing. Despite extensive prior research invested in their development, their performance and applicability to a range of daily tasks remained unclear and niche. However, their wider utilization without a requirement for technical expertise, made in large part possible through conversational fine-tuning, revealed the extent of their true capabilities in a real-world environment. This has garnered both public excitement for their potential applications and concerns about their capabilities and potential malicious uses. This review aims to provide a brief overview of the history, state of the art, and implications of Generative Language Models in terms of their principles, abilities, limitations, and future prospects -- especially in the context of cyber-defense, with a focus on the Swiss operational environment