Statistical inference of transmission fidelity of DNA methylation patterns over somatic cell divisions in mammals

We develop Bayesian inference methods for a recently-emerging type of epigenetic data to study the transmission fidelity of DNA methylation patterns over cell divisions. The data consist of parent-daughter double-stranded DNA methylation patterns with each pattern coming from a single cell and represented as an unordered pair of binary strings. The data are technically difficult and time-consuming to collect, putting a premium on an efficient inference method. Our aim is to estimate rates for the maintenance and de novo methylation events that gave rise to the observed patterns, while accounting for measurement error. We model data at multiple sites jointly, thus using whole-strand information, and considerably reduce confounding between parameters. We also adopt a hierarchical structure that allows for variation in rates across sites without an explosion in the effective number of parameters. Our context-specific priors capture the expected stationarity, or near-stationarity, of the stochastic process that generated the data analyzed here. This expected stationarity is shown to greatly increase the precision of the estimation. Applying our model to a data set collected at the human FMR1 locus, we find that measurement errors, generally ignored in similar studies, occur at a nontrivial rate (inappropriate bisulfite conversion error: 1.6$$ with 80$$ CI: 0.9--2.3$$). Accounting for these errors has a substantial impact on estimates of key biological parameters. The estimated average failure of maintenance rate and daughter de novo rate decline from 0.04 to 0.024 and from 0.14 to 0.07, respectively, when errors are accounted for. Our results also provide evidence that de novo events may occur on both parent and daughter strands: the median parent and daughter de novo rates are 0.08 (80$$ CI: 0.04--0.13) and 0.07 (80$$ CI: 0.04--0.11), respectively.Comment: Published in at http://dx.doi.org/10.1214/09-AOAS297 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Beyond generative models: superfast traversal, optimization, novelty, exploration and discovery (STONED) algorithm for molecules using SELFIES

Inverse design allows the generation of molecules with desirable physical quantities using property optimization. Deep generative models have recently been applied to tackle inverse design, as they possess the ability to optimize molecular properties directly through structure modification using gradients. While the ability to carry out direct property optimizations is promising, the use of generative deep learning models to solve practical problems requires large amounts of data and is very time-consuming. In this work, we propose STONED - a simple and efficient algorithm to perform interpolation and exploration in the chemical space, comparable to deep generative models. STONED bypasses the need for large amounts of data and training times by using string modifications in the SELFIES molecular representation. First, we achieve non-trivial performance on typical benchmarks for generative models without any training. Additionally, we demonstrate applications in high-throughput virtual screening for the design of drugs, photovoltaics, and the construction of chemical paths, allowing for both property and structure-based interpolation in the chemical space. Overall, we anticipate our results to be a stepping stone for developing more sophisticated inverse design models and benchmarking tools, ultimately helping generative models achieve wider adoption

Computer-Aided Multi-Objective Optimization in Small Molecule Discovery

Molecular discovery is a multi-objective optimization problem that requires identifying a molecule or set of molecules that balance multiple, often competing, properties. Multi-objective molecular design is commonly addressed by combining properties of interest into a single objective function using scalarization, which imposes assumptions about relative importance and uncovers little about the trade-offs between objectives. In contrast to scalarization, Pareto optimization does not require knowledge of relative importance and reveals the trade-offs between objectives. However, it introduces additional considerations in algorithm design. In this review, we describe pool-based and de novo generative approaches to multi-objective molecular discovery with a focus on Pareto optimization algorithms. We show how pool-based molecular discovery is a relatively direct extension of multi-objective Bayesian optimization and how the plethora of different generative models extend from single-objective to multi-objective optimization in similar ways using non-dominated sorting in the reward function (reinforcement learning) or to select molecules for retraining (distribution learning) or propagation (genetic algorithms). Finally, we discuss some remaining challenges and opportunities in the field, emphasizing the opportunity to adopt Bayesian optimization techniques into multi-objective de novo design

Entwicklung einer computergestützten Methode zum reaktionsbasierten De-Novo-Design wirkstoffartiger Verbindungen

A new method for computer-based de novo design of drug candidate structures is proposed. DOGS (Design of Genuine Structures) features a ligand-based strategy to suggest new molecular structures. The quality of designed compounds is assessed by a graph kernel method measuring the distance of designed molecules to a known reference ligand. Two graph representations of molecules (molecular graph and reduced graph) are implemented to feature different levels of abstraction from the molecular structure. A fully deterministic construction procedure explicitly designed to facilitate synthesizability of proposed structures is realized: DOGS uses readily available synthesis building blocks and established reaction schemes to assemble new molecules. This approach enables the software to propose not only the final compounds, but also to give suggestions for synthesis routes to generate them at the bench. The set of synthesis schemes comprises about 83 chemical reactions. Special focus was put on ring closure reactions forming drug-like substructures. The library of building blocks consists of about 25,000 readily available synthesis building blocks. DOGS builds up new structures in a stepwise process. Each virtual synthesis step adds a fragment to the growing molecule until a stop criterion (upper threshold for molecular mass or number of synthesis steps) is fulfilled. In a theoretical evaluation, a set of ~1,800 molecules proposed by DOGS is analyzed for critical properties of de novo designed compounds. The software is able to suggest drug-like molecules (79% violate less than two of Lipinski’s ‘rule of five’). In addition, a trained classifier for drug-likeness assigns a score >0.8 to 51% of the designed molecules (with 1.0 being the top score). In addition, most of the DOGS molecules are deemed to be synthesizable by a retro-synthesis descriptor (77% of molecules score in the top 10% of the decriptor’s value range). Calculated logP(o/w) values of constructed molecules resemble a unimodal distribution centred close to the mean of logP(o/w) values calculated for the reference compounds. A structural analysis of selected designs reveals that DOGS is capable of constructing molecules reflecting the overall topological arrangement of pharmacophoric features found in the reference ligands. At the same time, the DOGS designs represent innovative compounds being structurally distinct from the references. Synthesis routes for these examples are short and seem feasible in most cases. Some reaction steps might need modification by using protecting groups to avoid unwanted side reactions. Plausible bioisosters for known privileged fragments addressing the S1 pocket of trypsin were proposed by DOGS in a case study. Three of them can be found in known trypsin inhibitors as S1-adressing side chains. The software was also tested in two prospective case studies to design bioactive compounds. DOGS was applied to design ligands for human gamma-secretase and human histamine receptor subtype 4 (hH4R). Two selected designs for gamma-secretase were readily synthesizable as suggested by the software in one-step reactions. Both compounds represent inverse modulators of the target molecule. In a second case study, a ligand candidate selected for hH4R was synthesized exactly following the three-step synthesis plan suggested by DOGS. This compound showed low activity on the target structure. The concept of DOGS is able to deliver synthesizable and bioactive compounds. Suggested synthesis plans of selected compounds were readily pursuable. DOGS can therefore serve as a valuable idea generator for the design of new pharmacological active compounds.Im Rahmen der vorliegenden Arbeit wird eine neue Methode zum computergestützten de novo Design von wirkstoffartigen Molekülen vorgestellt. Ziel ist es, automatisiert und zielgerichtet neuartige Moleküle mit biologischer Aktivität zu entwerfen. Das entwickelte Programm DOGS (Design of Genuine Structures) schlägt zusätzlich zu den chemischen Verbindungen mögliche Strategien zu deren Synthese vor. Ein vollständig deterministischer Konstruktionsalgorithmus verwendet verfügbare Synthesebausteine und etablierte chemische Reaktionen zum Aufbau der neuen Moleküle. Die Bibliothek der Synthesebausteine umfasst etwa 25.000 Moleküle mit einer molekularen Masse zwischen 30 und 300 Da. Die Sammlung der Reaktionen zur Verknüpfung der Bausteine besteht aus 83 literaturbeschriebenen chemischen Reaktionen. Ein Großteil stellt Syntheseschritte zur Generierung neuer Ringsysteme dar. DOGS baut neue Moleküle schrittweise auf: In jedem virtuellen Syntheseschritt wird ein neues Fragment an das wachsende Molekül angefügt, bis eines der Stoppkriterien (Überschreitung einer maximalen molekulare Masse oder Anzahl Syntheseschritte) erfüllt ist. Zur Bewertung der Qualität der Zischen- und Endprodukte wird eine ligandenbasierte Strategie verwendet. Die entstehenden Moleküle werden mit einem bekannten Referenzliganden verglichen, welcher die gewünschte biologische Aktivität aufweist. Das Verfahren zielt dabei auf die Maximierung der Ähnlichkeit der neu konstruierten Moleküle zur Referenz ab. Eine Graphkernmethode berechnet die Ähnlichkeit zum Referenzliganden anhand des Vergleichs ihrer zweidimensionalen molekularen Struktur. In einer theoretischen Auswertung des Programms werden ca. 1.800 generierte potentielle Trypsin-Inhibitoren hinsichtlich solcher Eigenschaften analysiert, welche für neu entworfene Verbindungen kritisch sind: DOGS ist in der Lage wirkstoffartige Moleküle zu entwerfen (79% verletzen weniger als zwei von Lipinskis 'rule of five' Kriterien zur Abschätzung der oralen Bioverfügbarkeit). Zusätzlich wurde die Wirkstoffartigkeit der DOGS-Moleküle durch einen trainierten Klassifizieralgorithmus bewertet. Hierbei erhielten 51% der Verbindungen einen Wert in den oberen 20% des Wertebereichs des Klassifizierers. Weiterhin wird die synthetische Zugänglichkeit für den Großteil der computergenerierten Moleküle als hoch eingeschätzt (77% erhalten einen Wert in den oberen 10% des Wertebereichs eines Deskriptors zur Abschätzung der Synthetisierbarkeit). Die berechneten logP(o/w) Werte der konstruierten Moleküle entsprechen in ihrer Verteilung denen der Referenzliganden. Die Untersuchung der vorgeschlagenen Trypsin-Inhibitoren auf Bioisostere zur Adressierung der S1-Bindetasche zeigt, dass hierfür plausible Vorschläge von DOGS generiert werden. Der Großteil ist potentiell in der Lage eine kritische ladungsvermittelte Interaktion mit dem Protein in der S1-Bindetasche einzugehen. Unter den Vorschlägen befinden sich unter anderem auch drei Seitenketten, für die Interaktionen mit der S1-Bindetasche von Trypsin experimentell bestätigt sind. Eine Analyse ausgewählter Beispiele aus verschiedenen Läufen zum Ligandenentwurf für unterschiedliche biologische Zielmoleküle zeigt, dass das Programm in der Lage ist, die generelle topologische Anordnung potentieller Interaktionspunkte der Referenzliganden in den neu erzeugten Molekülen beizubehalten. Gleichzeitig sind diese Moleküle strukturell verschieden im Vergleich zu den Referenzliganden. Die generierten Synthesewege sind kurz und erscheinen in den meisten Fällen plausibel. Für einige der Syntheseschritte wird bei der praktischen Umsetzung der ergänzende Einsatz von Schutzgruppen notwendig sein, um unerwünschte Nebenreaktionen zu vermeiden. Die Software wurde zusätzlich zu den theoretischen Analysen in prospektiven Studien zum Ligandenentwurf praktisch evaluiert. Hierzu wurde DOGS zur Generierung von Liganden des humanen Histaminrezeptors 4 (hH4R) sowie der humanen gamma-Sekretase eingesetzt. Für hH4R wurde einer der entworfenen potentiellen Liganden synthetisiert, wobei der vorgeschlagene Syntheseweg exakt nachvollzogen werden konnte. Der Ligand weist eine geringfügige Affinität zum Histaminrezeptor auf. Für die gamma-Sekretase wurden zwei der entworfenen Moleküle zur Synthese und Testung ausgewählt. In beiden Fällen konnte auch hier die von DOGS vorgeschlagene Synthesestrategie nachvollzogen werden. Anschließende in vitro Analysen wiesen beide Verbindungen als inverse Modulatoren der gamma-Sekretase aus. Das Konstruktionskonzept von DOGS ist in der Lage, bioaktive Substanzen vorzuschlagen. Diese sind synthetisch zugänglich und können nach der vorgeschlagenen Strategie synthetisiert werden. Somit kann das Programm als Ideengenerator für den Entwurf neuer bioaktiver Moleküle dienen

Will More Expressive Graph Neural Networks do Better on Generative Tasks?

Graph generation poses a significant challenge as it involves predicting a complete graph with multiple nodes and edges based on simply a given label. This task also carries fundamental importance to numerous real-world applications, including de-novo drug and molecular design. In recent years, several successful methods have emerged in the field of graph generation. However, these approaches suffer from two significant shortcomings: (1) the underlying Graph Neural Network (GNN) architectures used in these methods are often underexplored; and (2) these methods are often evaluated on only a limited number of metrics. To fill this gap, we investigate the expressiveness of GNNs under the context of the molecular graph generation task, by replacing the underlying GNNs of graph generative models with more expressive GNNs. Specifically, we analyse the performance of six GNNs in two different generative frameworks -- autoregressive generation models, such as GCPN and GraphAF, and one-shot generation models, such as GraphEBM -- on six different molecular generative objectives on the ZINC-250k dataset. Through our extensive experiments, we demonstrate that advanced GNNs can indeed improve the performance of GCPN, GraphAF, and GraphEBM on molecular generation tasks, but GNN expressiveness is not a necessary condition for a good GNN-based generative model. Moreover, we show that GCPN and GraphAF with advanced GNNs can achieve state-of-the-art results across 17 other non-GNN-based graph generative approaches, such as variational autoencoders and Bayesian optimisation models, on the proposed molecular generative objectives (DRD2, Median1, Median2), which are important metrics for de-novo molecular design.Comment: 2nd Learning on Graphs Conference (LoG 2023). 26 pages, 5 figures, 11 table

Deep generative models for biology: represent, predict, design

Deep generative models have revolutionized the field of artificial intelligence, fundamentally changing how we generate novel objects that imitate or extrapolate from training data, and transforming how we access and consume various types of information such as texts, images, speech, and computer programs. They have the potential to radically transform other scientific disciplines, ranging from mathematical problem solving, to supporting fast and accurate simulations in high-energy physics or enabling rapid weather forecasting. In computational biology, generative models hold immense promise for improving our understanding of complex biological processes, designing new drugs and therapies, and forecasting viral evolution during pandemics, among many other applications. Biological objects pose however unique challenges due to their inherent complexity, encompassing massive spaces, multiple complementary data modalities, and a unique interplay between highly structured and relatively unstructured components. In this thesis, we develop several deep generative modeling frameworks that are motivated by key questions in computational biology. Given the interdisciplinary nature of this endeavor, we first provide a comprehensive background in generative modeling, uncertainty quantification, sequential decision making, as well as important concepts in biology and chemistry to facilitate a thorough understanding of our work. We then deep dive into the core of our contributions, which are structured around three chapters. The first chapter introduces methods for learning representations of biological sequences, laying the foundation for subsequent analyses. The second chapter illustrates how these representations can be leveraged to predict complex properties of biomolecules, focusing on three specific applications: protein fitness prediction, the effects of genetic variations on human disease risk and viral immune escape. Finally, the third chapter is dedicated to methods for designing novel biomolecules, including drug target identification, de novo molecular optimization, and protein engineering. This thesis also makes several methodological contributions to broader machine learning challenges, such as uncertainty quantification in high-dimensional spaces or efficient transformer architectures, which hold potential value in other application domains. We conclude by summarizing our key findings, highlighting shortcomings of current approaches, proposing potential avenues for future research, and discussing emerging trends within the field