Multi-Fidelity Active Learning with GFlowNets

In the last decades, the capacity to generate large amounts of data in science and engineering applications has been growing steadily. Meanwhile, the progress in machine learning has turned it into a suitable tool to process and utilise the available data. Nonetheless, many relevant scientific and engineering problems present challenges where current machine learning methods cannot yet efficiently leverage the available data and resources. For example, in scientific discovery, we are often faced with the problem of exploring very large, high-dimensional spaces, where querying a high fidelity, black-box objective function is very expensive. Progress in machine learning methods that can efficiently tackle such problems would help accelerate currently crucial areas such as drug and materials discovery. In this paper, we propose the use of GFlowNets for multi-fidelity active learning, where multiple approximations of the black-box function are available at lower fidelity and cost. GFlowNets are recently proposed methods for amortised probabilistic inference that have proven efficient for exploring large, high-dimensional spaces and can hence be practical in the multi-fidelity setting too. Here, we describe our algorithm for multi-fidelity active learning with GFlowNets and evaluate its performance in both well-studied synthetic tasks and practically relevant applications of molecular discovery. Our results show that multi-fidelity active learning with GFlowNets can efficiently leverage the availability of multiple oracles with different costs and fidelities to accelerate scientific discovery and engineering design.Comment: Code: https://github.com/nikita-0209/mf-al-gf

Bayesian neural network learning for repeat purchase modelling in direct marketing.

We focus on purchase incidence modelling for a European direct mail company. Response models based on statistical and neural network techniques are contrasted. The evidence framework of MacKay is used as an example implementation of Bayesian neural network learning, a method that is fairly robust with respect to problems typically encountered when implementing neural networks. The automatic relevance determination (ARD) method, an integrated feature of this framework, allows to assess the relative importance of the inputs. The basic response models use operationalisations of the traditionally discussed Recency, Frequency and Monetary (RFM) predictor categories. In a second experiment, the RFM response framework is enriched by the inclusion of other (non-RFM) customer profiling predictors. We contribute to the literature by providing experimental evidence that: (1) Bayesian neural networks offer a viable alternative for purchase incidence modelling; (2) a combined use of all three RFM predictor categories is advocated by the ARD method; (3) the inclusion of non-RFM variables allows to significantly augment the predictive power of the constructed RFM classifiers; (4) this rise is mainly attributed to the inclusion of customer\slash company interaction variables and a variable measuring whether a customer uses the credit facilities of the direct mailing company.Marketing; Companies; Models; Model; Problems; Neural networks; Networks; Variables; Credit;

ICU prognostication: Time to re-evaluate? Register-based studies on improving prognostication for patients admitted to the intensive care unit (ICU)

Background: ICU prognostication is difficult because of patients’ prior comorbidities and their varied reasons for admission. The model used for ICU prognostication in Sweden is the Simplified Acute Physiology Score 3 (SAPS 3), which uses information gathered within one hour of ICU admission to predict 30-day mortality. Since the SAPS 3 model was introduced, no biomarkers have been added to it to improve its prognostic performance. For comatose patients admitted to the ICU after cardiac arrest, the prognostication performed after 72 h will either result in the continued observation of the patient or the withdrawal of life-sustaining treatment.Purpose: 1) To investigate whether adding the biomarker lactate (study I) or high-sensitivity troponin T (hsTnT) (study II) to SAPS 3 adds prognostic value. 2) To investigate whether using a supervised machine learning algorithm called artificial neural networks (ANNs) can improve the prognostic performance of SAPS 3 (study III). 3) To explore whether ANNs can create reliable predictions for comatose patients at the time of hospital admission (study IV) and during the first three days after ICU admission, with or without promising biomarkers (study V).Methods: 1) To investigate whether the laboratory values of lactate or hsTnT could improve the performance of SAPS 3, we combined patients’ laboratory values on ICU admission at Skåne University Hospital with their SAPS 3 score. 2) Based on all first-time ICU admissions in Sweden from 2009–2017 as retrieved from the Swedish Intensive Care Registry (SIR), we investigated whether ANNs could improve SAPS 3 using the same variables. 3) All out-of-hospital cardiac arrest (OHCA) patients from the Target Temperature Management trial were included for data analysis. Background and prehospital data, along with clinical variables at admission, were used in study IV. Clinical variables from the first three days were used in study V along with different levels of biomarkers defined as clinically accessible (e.g. neuron-specific enolase, or NSE) and research-grade biomarkers (e.g. neurofilament light, or NFL). Patient outcome was the dichotomised Cerebral Performance Category scale (CPC); a CPC of 1–2 was considered a good outcome, and a CPC of 3–5 was considered a poor outcome.Results: 1) Both lactate and hsTnT were independent SAPS 3 predictors for 30-day mortality in the logistic regression model. In a subgroup analysis, the use of lactate improved the area under the receiver operating characteristic curve (AUROC) for cardiac arrest and septic patients, and the use of hsTnT improved the AUROC for septic patients. 2) The overall performance of the SAPS 3 model in Sweden was improved by the use of ANNs. Both the discrimination (AUROC 0.89 vs 0.85, p < 0.001) and the calibration were improved when the two models were compared on a separate test set (n = 36,214). 3) An ANN model outperformed a logistic-regression-based model in predicting poor outcome on hospital admission for OHCA patients. Incorporating biomarkers such as NSE improved the AUROC over the course of the first three days of the ICU stay; when NFL was incorporated, the prognostic performance was excellent from day 1.Conclusion: Lactate and hsTnT probably add prognostic value to SAPS 3 for patients admitted to the ICU with sepsis or after cardiac arrest (lactate only). An ANN model was found to be superior to the SAPS 3 model (Swedish modification) and corrected better for age than SAPS 3. A simplified ANN model with eight variables showed performance similar to that of the SAPS 3 model. For comatose OHCA patients, an ANN model improved the accuracy of the prediction of the long-term neurological outcome at hospital admission. Furthermore, when it used cumulative information from the first three days after ICU admission, an ANN model showed promising prognostic performance on day 3 when it incorporated clinically accessible biomarkers such as NSE, and it showed promising performance on days 1–3 when it incorporated research-grade biomarkers such as NFL

Interpretability and Explainability: A Machine Learning Zoo Mini-tour

In this review, we examine the problem of designing interpretable and explainable machine learning models. Interpretability and explainability lie at the core of many machine learning and statistical applications in medicine, economics, law, and natural sciences. Although interpretability and explainability have escaped a clear universal definition, many techniques motivated by these properties have been developed over the recent 30 years with the focus currently shifting towards deep learning methods. In this review, we emphasise the divide between interpretability and explainability and illustrate these two different research directions with concrete examples of the state-of-the-art. The review is intended for a general machine learning audience with interest in exploring the problems of interpretation and explanation beyond logistic regression or random forest variable importance. This work is not an exhaustive literature survey, but rather a primer focusing selectively on certain lines of research which the authors found interesting or informative

Discovery and development of novel inhibitors for the kinase Pim-1 and G-Protein Coupled Receptor Smoothened

Investigation of the cause of disease is no easy business. This is particularly so when one reflects upon the lessons taught us in antiquity. Prior to the beginning of the last century, diagnosis and treatment of diseases such as cancers was so bereft of hope that there was little physicians could offer in the way of comfort, let alone treatment. One of the major insights from investigations into cancers this century has been that those involved in research leading to treatments are not dealing with a singular malady but multiple families of diseases with different mechanisms and modes of action. Therefore, despite the end game being similar in cancers, that of uncontrolled growth and replication leading to cellular dysfunction, different diseases require different approaches in targeting them. This leads us to a particular broad treatment approach, that of drug design. A drug is, in the classical sense, a small molecule that, upon introduction into the body, interacts with biochemical targets to induce a wider biological effect, ideally with both an intended target and intended effect. The conceptual basis underpinning this `lock-and-key' paradigm was elucidated over a century ago and the primary occupation of those involved in biochemical research has been to determine as much information as possible about both of these protein locks and drug keys. And, as inferred from the paradigm, molecular shape is all-important in determining and controlling action against the most important locks with the most potent and specific keys. The two most important target classes in drug discovery for some time have been protein kinases and G Protein-Coupled Receptors (GPCRs). Both classes of proteins are large families that perform very different tasks within the body. Kinases activate and inactive many cellular processes by catalysing the transfer of a phosphate group from Adenosine Tri-Phosphate (ATP) to other targets. GPCRs perform the job of interacting with chemical signals and communicating them into a biological response. Dysfunction in both types of proteins in certain cells can lead to a loss of biological control and, ultimately, a cancer. Both of kinases and GPCRs have entirely different chemical structures so structural knowledge therefore becomes crucial in any approach targeting cells where dysfunction has occurred. Thus, for this thesis, a member from each class was investigated using a combination of structural approaches. From the kinase class, the kinase Proviral Integration site for MuLV (Pim-1) and from the GPCR class, the cell membrane-bound Smoothened receptor (SMO). The kinase \pimone\ was the target of various approaches in \autoref{chap:three}. Although a heavily studied target from the mid-2000's, there is a paucity of inhibitors targeting residues more remote from structural characteristics that define kinases. Further limiting extension possibilities is that \pimone\ is constitutively active so no inhibitors targeting an inactive state are possible. An initial project (\pone) used the known binding properties of small molecules, or, `fragments' to elucidate structural and dynamic information useful for targeting \pimone. This was followed by three projects, all with the goal of inhibitor discovery, all with different foci. In \ptwo, fragment binding modes from \pone\ provided the basis for the extension and development of drug-like inhibitors with a focus on synthetic feasibility. In contrast, inhibitors were found in \pthree\ via a large-scale public dataset of purchasable molecules that possess drug-like properties. Finally, \pfour\ took the truncated form of a particularly attractive fragment from \pone\ that was crystallised with \pimone, verified its binding mode and then generated extensions with, again, a focus on synthetic feasibility. The GPCR \smo\ has fewer molecular studies and much about its structural behaviour remains unknown. As the most `druggable' protein in the Hedgehog pathway, structural studies have primarily focussed on stabilising its inactive state to prevent signal transduction. Allied to this is that there are generally few inhibitors for \smo\ and the drugs for cancers related to its dysfunction are vulnerable to mutations that significantly reduce their effectiveness or abrogate it entirely. The elucidation of structural information in therefore of high priority. An initial study attempting to identify an unknown molecule from prior experiments led to insights regarding binding characteristics of specific moieties. This was particularly important to understand not just where favourable moieties bind but also sections of the \smo\ binding pocket with unfavourable binding. In both subsequent virtual screens performed in Chapter 4, the primary aim was to find new drug-like inhibitors of \smo\ using large public datasets of commercially-available molecules. The initial screen retrieved relatively few inhibitors so the binding pocket was modified to find a structural state more amenable to small molecule binding. These modifications led to a significant number of new, chemically novel inhibitors for \smo, some structural information useful for future inhibitors and the elucidation of structure-activity relationships useful for inhibitor design. This underpins the idea that structural information is of critical importance in the discovery and design of molecular inhibitors

An analysis of dose effectiveness and incidence of late rectal complications of high dose-rate brachytherapy in the radical treatment of cervical cancer

Thesis (M. Tech.) -- Central University of Technology, Free State, 200

Geometric guides for interactive evolutionary design

This thesis describes the addition of novel Geometric Guides to a generative Computer-Aided Design (CAD) application that supports early-stage concept generation. The application generates and evolves abstract 3D shapes, used to inspire the form of new product concepts. It was previously a conventional Interactive Evolutionary system where users selected shapes from evolving populations. However, design industry users wanted more control over the shapes, for example by allowing the system to influence the proportions of evolving forms. The solution researched, developed, integrated and tested is a more cooperative human-machine system combining classic user interaction with innovative geometric analysis. In the literature review, different types of Interactive Evolutionary Computation (IEC), Pose Normalisation (PN), Shape Comparison, and Minimum-Volume Bounding Box approaches are compared, with some of these technologies identified as applicable for this research. Using its Application Programming Interface, add-ins for the Siemens NX CAD system have been developed and integrated with an existing Interactive Evolutionary CAD system. These add-ins allow users to create a Geometric Guide (GG) at the start of a shape exploration session. Before evolving shapes can be compared with the GG, they must be aligned and scaled (known as Pose Normalisation in the literature). Computationally-efficient PN has been achieved using geometric functions such as Bounding Box for translation and scaling, and Principle Axes for the orientation. A shape comparison algorithm has been developed that is based on the principle of non-intersecting volumes. This algorithm is also implemented with standard, readily available geometric functions, is conceptually simple, accessible to other researchers and also offers appropriate efficacy. Objective geometric testing showed that the PN and Shape Comparison methods developed are suitable for this guiding application and can be efficiently adapted to enhance an Interactive Evolutionary Design system. System performance with different population sizes was examined to indicate how best to use the new guiding capabilities to assist users in evolutionary shape searching. This was backed up by participant testing research into two user interaction strategies. A Large Background Population (LBP) approach where the GG is used to select a sub-set of shapes to show to the user was shown to be the most effective. The inclusion of Geometric Guides has taken the research from the existing aesthetic focused tool to a system capable of application to a wider range of engineering design problems. This system supports earlier design processes and ideation in conceptual design and allows a designer to experiment with ideas freely to interactively explore populations of evolving solutions. The design approach has been further improved, and expanded beyond the previous quite limited scope of form exploration