Industry-Scale Orchestrated Federated Learning for Drug Discovery

To apply federated learning to drug discovery we developed a novel platform in the context of European Innovative Medicines Initiative (IMI) project MELLODDY (grant n{\deg}831472), which was comprised of 10 pharmaceutical companies, academic research labs, large industrial companies and startups. The MELLODDY platform was the first industry-scale platform to enable the creation of a global federated model for drug discovery without sharing the confidential data sets of the individual partners. The federated model was trained on the platform by aggregating the gradients of all contributing partners in a cryptographic, secure way following each training iteration. The platform was deployed on an Amazon Web Services (AWS) multi-account architecture running Kubernetes clusters in private subnets. Organisationally, the roles of the different partners were codified as different rights and permissions on the platform and administrated in a decentralized way. The MELLODDY platform generated new scientific discoveries which are described in a companion paper.Comment: 9 pages, 4 figures, to appear in AAAI-23 ([IAAI-23 track] Deployed Highly Innovative Applications of AI

Impairments in Episodic-Autobiographical Memory and Emotional and Social Information Processing in CADASIL during Mid-Adulthood

Staniloiu A, Woermann FG, Markowitsch HJ. Impairments in Episodic-Autobiographical Memory and Emotional and Social Information Processing in CADASIL during Mid-Adulthood. Frontiers in Behavioral Neuroscience. 2014;8: 227.Cerebral autosomal dominant arteriopathy with subcortical infarcts and leukoencephalopathy (CADASIL) – is the most common genetic source of vascular dementia in adults, being caused by a mutation in NOTCH3 gene. Spontaneous de novo mutations may occur, but their frequency is largely unknown. Ischemic strokes and cognitive impairments are the most frequent manifestations, but seizures affect up to 10% of the patients. Herein, we describe a 47-year-old male scholar with a genetically confirmed diagnosis of CADASIL (Arg133Cys mutation in the NOTCH3 gene) and a seemingly negative family history of CADASIL illness, who was investigated with a comprehensive neuropsychological testing battery and neuroimaging methods. The patient demonstrated on one hand severe and accelerated deteriorations in multiple cognitive domains such as concentration, long-term memory (including the episodic-autobiographical memory domain), problem solving, cognitive flexibility and planning, affect recognition, discrimination and matching, and social cognition (theory of mind). Some of these impairments were even captured by abbreviated instruments for investigating suspicion of dementia. On the other hand the patient still possessed high crystallized (verbal) intelligence and a capacity to put forth a façade of well-preserved intellectual functioning. Although no definite conclusions can be drawn from a single case study, our findings point to the presence of additional cognitive changes in CADASIL in middle adulthood, in particular to impairments in the episodic-autobiographical memory domain and social information processing (e.g., social cognition). Whether these identified impairments are related to the patient’s specific phenotype or to an ascertainment bias (e.g., a paucity of studies investigating these cognitive functions) requires elucidation by larger scale research

Cell-Free Protein Synthesis with Fungal Lysates for the Rapid Production of Unspecific Peroxygenases

Unspecific peroxygenases (UPOs, EC 1.11.2.1) are fungal biocatalysts that have attracted considerable interest for application in chemical syntheses due to their ability to selectively incorporate peroxide-oxygen into non-activated hydrocarbons. However, the number of available and characterized UPOs is limited, as it is difficult to produce these enzymes in homologous or hetero-logous expression systems. In the present study, we introduce a third approach for the expression of UPOs: cell-free protein synthesis using lysates from filamentous fungi. Biomass of Neurospora crassa and Aspergillus niger, respectively, was lysed by French press and tested for translational activity with a luciferase reporter enzyme. The upo1 gene from Cyclocybe (Agrocybe) aegerita (encoding the main peroxygenase, AaeUPO) was cell-free expressed with both lysates, reaching activities of up to 105 U L−1 within 24 h (measured with veratryl alcohol as substrate). The cell-free expressed enzyme (cfAaeUPO) was successfully tested in a substrate screening that included prototypical UPO substrates, as well as several pharmaceuticals. The determined activities and catalytic performance were comparable to that of the wild-type enzyme (wtAaeUPO). The results presented here suggest that cell-free expression could become a valuable tool to gain easier access to the immense pool of putative UPO genes and to expand the spectrum of these sought-after biocatalysts

Conformal efficiency as a metric for comparative model assessment befitting federated learning

As training volume increases predictive model quality, leveraging existing external data sources holds the promise of time- and cost-efficiency. In a drug discovery setting, pharmaceutical companies all own substantial but confidential datasets. The MELLODDY project develops a privacy-preserving federated machine learning solution and deploys it at an unprecedented scale (more than 100,000 tasks across ten major pharmaceutical companies), while ensuring the security and privacy of each partner’s sensitive data. Each partner builds models that benefit from a shared representation, for their own private assays. Established predictive performance metrics such as AUC ROC or AUC PR are constrained to unseen labelled chemical space. However, they cannot gauge performance gains in unlabelled chemical space. Federated learning indirectly extends labelled space, but in a privacy-preserving context, a partner cannot use this label extension for performance assessment. Metrics that estimate uncertainty on a prediction can be calculated even where no label is known. Practically, the chemical space covered with predictions of sufficient confidence, reflects the applicability domain of a model. After establishing a link to established performance metrics, we propose the efficiency from the conformal prediction framework (‘conformal efficiency’) as a proxy to the applicability domain size. A documented extension of the applicability domain would qualify as a tangible benefit from federated learning. In interim assessments, MELLODDY partners report a median increase in conformal efficiency of the federated over the single-partner model of 5.5% (with increases up to 9.7%). Subject to distributional conditions, that efficiency increase can be directly interpreted as the expected increase in conformal i.e. high confidence predictions. In conclusion, we present the first evidence that privacy-preserving federated machine learning across massive drug-discovery datasets from ten pharma partners indeed extends the applicability domain of property prediction models