Predictable Artificial Intelligence

We introduce the fundamental ideas and challenges of Predictable AI, a nascent research area that explores the ways in which we can anticipate key indicators of present and future AI ecosystems. We argue that achieving predictability is crucial for fostering trust, liability, control, alignment and safety of AI ecosystems, and thus should be prioritised over performance. While distinctive from other areas of technical and non-technical AI research, the questions, hypotheses and challenges relevant to Predictable AI were yet to be clearly described. This paper aims to elucidate them, calls for identifying paths towards AI predictability and outlines the potential impact of this emergent field.Comment: 11 pages excluding references, 4 figures, and 2 tables. Paper Under Revie

Anti-Spike antibodies 3 months after SARS-CoV-2 mRNA vaccine booster dose in patients on hemodialysis: the prospective SENCOVAC study

Background: Patients on hemodialysis are at high-risk for complications derived from coronavirus disease 2019 (COVID-19). The present analysis evaluated the impact of a booster vaccine dose and breakthrough severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infections on humoral immunity 3 months after the booster dose. Methods: This is a multicentric and prospective study assessing immunoglobulin G anti-Spike antibodies 6 and 9 months after initial SARS-CoV-2 vaccination in patients on hemodialysis that had also received a booster dose before the 6-month assessment (early booster) or between the 6- and 9-month assessments (late booster). The impact of breakthrough infections, type of vaccine, time from the booster and clinical variables were assessed. Results: A total of 711 patients [67% male, median age (range) 67 (20-89) years] were included. Of these, 545 (77%) received an early booster and the rest a late booster. At 6 months, 64 (9%) patients had negative anti-Spike antibody titers (3% of early booster and 29% of late booster patients, P =. 001). At 9 months, 91% of patients with 6-month negative response had seroconverted and there were no differences in residual prevalence of negative humoral response between early and late booster patients (0.9% vs 0.6%, P =. 693). During follow-up, 35 patients (5%) developed breakthrough SARS-CoV-2 infection. Antibody titers at 9 months were independently associated with mRNA-1273 booster (P =. 001), lower time from booster (P =. 043) and past breakthrough SARS-CoV-2 infection (P <. 001). Conclusions: In hemodialysis patients, higher titers of anti-Spike antibodies at 9 months were associated with mRNA-1273 booster, lower time from booster and past breakthrough SARS-CoV-2 infectionThe present project has been supported by Fresenius Medical Care, Diaverum, Vifor Pharma, Vircell, Fundación Renal Iñigo Álvarez de Toledo and ISCIII FEDER funds RICORS2040 (RD21/0005

Clonal chromosomal mosaicism and loss of chromosome Y in elderly men increase vulnerability for SARS-CoV-2

The pandemic caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2, COVID-19) had an estimated overall case fatality ratio of 1.38% (pre-vaccination), being 53% higher in males and increasing exponentially with age. Among 9578 individuals diagnosed with COVID-19 in the SCOURGE study, we found 133 cases (1.42%) with detectable clonal mosaicism for chromosome alterations (mCA) and 226 males (5.08%) with acquired loss of chromosome Y (LOY). Individuals with clonal mosaic events (mCA and/or LOY) showed a 54% increase in the risk of COVID-19 lethality. LOY is associated with transcriptomic biomarkers of immune dysfunction, pro-coagulation activity and cardiovascular risk. Interferon-induced genes involved in the initial immune response to SARS-CoV-2 are also down-regulated in LOY. Thus, mCA and LOY underlie at least part of the sex-biased severity and mortality of COVID-19 in aging patients. Given its potential therapeutic and prognostic relevance, evaluation of clonal mosaicism should be implemented as biomarker of COVID-19 severity in elderly people. Among 9578 individuals diagnosed with COVID-19 in the SCOURGE study, individuals with clonal mosaic events (clonal mosaicism for chromosome alterations and/or loss of chromosome Y) showed an increased risk of COVID-19 lethality

GWAS and meta-analysis identifies 49 genetic variants underlying critical COVID-19

Data availability: Downloadable summary data are available through the GenOMICC data site (https://genomicc.org/data). Summary statistics are available, but without the 23andMe summary statistics, except for the 10,000 most significant hits, for which full summary statistics are available. The full GWAS summary statistics for the 23andMe discovery dataset will be made available through 23andMe to qualified researchers under an agreement with 23andMe that protects the privacy of the 23andMe participants. For further information and to apply for access to the data, see the 23andMe website (https://research.23andMe.com/dataset-access/). All individual-level genotype and whole-genome sequencing data (for both academic and commercial uses) can be accessed through the UKRI/HDR UK Outbreak Data Analysis Platform (https://odap.ac.uk). A restricted dataset for a subset of GenOMICC participants is also available through the Genomics England data service. Monocyte RNA-seq data are available under the title ‘Monocyte gene expression data’ within the Oxford University Research Archives (https://doi.org/10.5287/ora-ko7q2nq66). Sequencing data will be made freely available to organizations and researchers to conduct research in accordance with the UK Policy Framework for Health and Social Care Research through a data access agreement. Sequencing data have been deposited at the European Genome–Phenome Archive (EGA), which is hosted by the EBI and the CRG, under accession number EGAS00001007111.Extended data figures and tables are available online at https://www.nature.com/articles/s41586-023-06034-3#Sec21 .Supplementary information is available online at https://www.nature.com/articles/s41586-023-06034-3#Sec22 .Code availability: Code to calculate the imputation of P values on the basis of SNPs in linkage disequilibrium is available at GitHub (https://github.com/baillielab/GenOMICC_GWAS).Acknowledgements: We thank the members of the Banco Nacional de ADN and the GRA@CE cohort group; and the research participants and employees of 23andMe for making this work possible. A full list of contributors who have provided data that were collated in the HGI project, including previous iterations, is available online (https://www.covid19hg.org/acknowledgements).Change history: 11 July 2023: A Correction to this paper has been published at: https://doi.org/10.1038/s41586-023-06383-z. -- In the version of this article initially published, the name of Ana Margarita Baldión-Elorza, of the SCOURGE Consortium, appeared incorrectly (as Ana María Baldion) and has now been amended in the HTML and PDF versions of the article.Copyright © The Author(s) 2023, Critical illness in COVID-19 is an extreme and clinically homogeneous disease phenotype that we have previously shown1 to be highly efficient for discovery of genetic associations2. Despite the advanced stage of illness at presentation, we have shown that host genetics in patients who are critically ill with COVID-19 can identify immunomodulatory therapies with strong beneficial effects in this group3. Here we analyse 24,202 cases of COVID-19 with critical illness comprising a combination of microarray genotype and whole-genome sequencing data from cases of critical illness in the international GenOMICC (11,440 cases) study, combined with other studies recruiting hospitalized patients with a strong focus on severe and critical disease: ISARIC4C (676 cases) and the SCOURGE consortium (5,934 cases). To put these results in the context of existing work, we conduct a meta-analysis of the new GenOMICC genome-wide association study (GWAS) results with previously published data. We find 49 genome-wide significant associations, of which 16 have not been reported previously. To investigate the therapeutic implications of these findings, we infer the structural consequences of protein-coding variants, and combine our GWAS results with gene expression data using a monocyte transcriptome-wide association study (TWAS) model, as well as gene and protein expression using Mendelian randomization. We identify potentially druggable targets in multiple systems, including inflammatory signalling (JAK1), monocyte–macrophage activation and endothelial permeability (PDE4A), immunometabolism (SLC2A5 and AK5), and host factors required for viral entry and replication (TMPRSS2 and RAB2A).GenOMICC was funded by Sepsis Research (the Fiona Elizabeth Agnew Trust), the Intensive Care Society, a Wellcome Trust Senior Research Fellowship (to J.K.B., 223164/Z/21/Z), the Department of Health and Social Care (DHSC), Illumina, LifeArc, the Medical Research Council, UKRI, a BBSRC Institute Program Support Grant to the Roslin Institute (BBS/E/D/20002172, BBS/E/D/10002070 and BBS/E/D/30002275) and UKRI grants MC_PC_20004, MC_PC_19025, MC_PC_1905 and MRNO2995X/1. A.D.B. acknowledges funding from the Wellcome PhD training fellowship for clinicians (204979/Z/16/Z), the Edinburgh Clinical Academic Track (ECAT) programme. This research is supported in part by the Data and Connectivity National Core Study, led by Health Data Research UK in partnership with the Office for National Statistics and funded by UK Research and Innovation (grant MC_PC_20029). Laboratory work was funded by a Wellcome Intermediate Clinical Fellowship to B.F. (201488/Z/16/Z). We acknowledge the staff at NHS Digital, Public Health England and the Intensive Care National Audit and Research Centre who provided clinical data on the participants; and the National Institute for Healthcare Research Clinical Research Network (NIHR CRN) and the Chief Scientist’s Office (Scotland), who facilitate recruitment into research studies in NHS hospitals, and to the global ISARIC and InFACT consortia. GenOMICC genotype controls were obtained using UK Biobank Resource under project 788 funded by Roslin Institute Strategic Programme Grants from the BBSRC (BBS/E/D/10002070 and BBS/E/D/30002275) and Health Data Research UK (HDR-9004 and HDR-9003). UK Biobank data were used in the GSMR analyses presented here under project 66982. The UK Biobank was established by the Wellcome Trust medical charity, Medical Research Council, Department of Health, Scottish Government and the Northwest Regional Development Agency. It has also had funding from the Welsh Assembly Government, British Heart Foundation and Diabetes UK. The work of L.K. was supported by an RCUK Innovation Fellowship from the National Productivity Investment Fund (MR/R026408/1). J.Y. is supported by the Westlake Education Foundation. SCOURGE is funded by the Instituto de Salud Carlos III (COV20_00622 to A.C., PI20/00876 to C.F.), European Union (ERDF) ‘A way of making Europe’, Fundación Amancio Ortega, Banco de Santander (to A.C.), Cabildo Insular de Tenerife (CGIEU0000219140 ‘Apuestas científicas del ITER para colaborar en la lucha contra la COVID-19’ to C.F.) and Fundación Canaria Instituto de Investigación Sanitaria de Canarias (PIFIISC20/57 to C.F.). We also acknowledge the contribution of the Centro National de Genotipado (CEGEN) and Centro de Supercomputación de Galicia (CESGA) for funding this project by providing supercomputing infrastructures. A.D.L. is a recipient of fellowships from the National Council for Scientific and Technological Development (CNPq)-Brazil (309173/2019-1 and 201527/2020-0)

Beyond the imitation game: Quantifying and extrapolating the capabilities of language models

Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabilities are as yet poorly characterized. In order to inform future research, prepare for disruptive new model capabilities, and ameliorate socially harmful effects, it is vital that we understand the present and near-future capabilities and limitations of language models. To address this challenge, we introduce the Beyond the Imitation Game benchmark (BIG-bench). BIG-bench currently consists of 204 tasks, contributed by 442 authors across 132 institutions. Task topics are diverse, drawing problems from linguistics, childhood development, math, common-sense reasoning, biology, physics, social bias, software development, and beyond. BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models. We evaluate the behavior of OpenAI's GPT models, Google-internal dense transformer architectures, and Switch-style sparse transformers on BIG-bench, across model sizes spanning millions to hundreds of billions of parameters. In addition, a team of human expert raters performed all tasks in order to provide a strong baseline. Findings include: model performance and calibration both improve with scale, but are poor in absolute terms (and when compared with rater performance); performance is remarkably similar across model classes, though with benefits from sparsity; tasks that improve gradually and predictably commonly involve a large knowledge or memorization component, whereas tasks that exhibit "breakthrough" behavior at a critical scale often involve multiple steps or components, or brittle metrics; social bias typically increases with scale in settings with ambiguous context, but this can be improved with prompting

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabilities are as yet poorly characterized. In order to inform future research, prepare for disruptive new model capabilities, and ameliorate socially harmful effects, it is vital that we understand the present and near-future capabilities and limitations of language models. To address this challenge, we introduce the Beyond the Imitation Game benchmark (BIG-bench). BIG-bench currently consists of 204 tasks, contributed by 442 authors across 132 institutions. Task topics are diverse, drawing problems from linguistics, childhood development, math, common-sense reasoning, biology, physics, social bias, software development, and beyond. BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models. We evaluate the behavior of OpenAI's GPT models, Google-internal dense transformer architectures, and Switch-style sparse transformers on BIG-bench, across model sizes spanning millions to hundreds of billions of parameters. In addition, a team of human expert raters performed all tasks in order to provide a strong baseline. Findings include: model performance and calibration both improve with scale, but are poor in absolute terms (and when compared with rater performance); performance is remarkably similar across model classes, though with benefits from sparsity; tasks that improve gradually and predictably commonly involve a large knowledge or memorization component, whereas tasks that exhibit "breakthrough" behavior at a critical scale often involve multiple steps or components, or brittle metrics; social bias typically increases with scale in settings with ambiguous context, but this can be improved with prompting.Comment: 27 pages, 17 figures + references and appendices, repo: https://github.com/google/BIG-benc