BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License

Mapping local patterns of childhood overweight and wasting in low- and middle-income countries between 2000 and 2017

A double burden of malnutrition occurs when individuals, household members or communities experience both undernutrition and overweight. Here, we show geospatial estimates of overweight and wasting prevalence among children under 5 years of age in 105 low- and middle-income countries (LMICs) from 2000 to 2017 and aggregate these to policy-relevant administrative units. Wasting decreased overall across LMICs between 2000 and 2017, from 8.4% (62.3 (55.1–70.8) million) to 6.4% (58.3 (47.6–70.7) million), but is predicted to remain above the World Health Organization’s Global Nutrition Target of <5% in over half of LMICs by 2025. Prevalence of overweight increased from 5.2% (30 (22.8–38.5) million) in 2000 to 6.0% (55.5 (44.8–67.9) million) children aged under 5 years in 2017. Areas most affected by double burden of malnutrition were located in Indonesia, Thailand, southeastern China, Botswana, Cameroon and central Nigeria. Our estimates provide a new perspective to researchers, policy makers and public health agencies in their efforts to address this global childhood syndemic

Global, regional, and national incidence, prevalence, and years lived with disability for 328 diseases and injuries for 195 countries, 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016

As mortality rates decline, life expectancy increases, and populations age, non-fatal outcomes of diseases and injuries are becoming a larger component of the global burden of disease. The Global Burden of Diseases, Injuries, and Risk Factors Study 2016 (GBD 2016) provides a comprehensive assessment of prevalence, incidence, and years lived with disability (YLDs) for 328 causes in 195 countries and territories from 1990 to 2016

Global burden of 369 diseases and injuries in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019

Background: In an era of shifting global agendas and expanded emphasis on non-communicable diseases and injuries along with communicable diseases, sound evidence on trends by cause at the national level is essential. The Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) provides a systematic scientific assessment of published, publicly available, and contributed data on incidence, prevalence, and mortality for a mutually exclusive and collectively exhaustive list of diseases and injuries. Methods: GBD estimates incidence, prevalence, mortality, years of life lost (YLLs), years lived with disability (YLDs), and disability-adjusted life-years (DALYs) due to 369 diseases and injuries, for two sexes, and for 204 countries and territories. Input data were extracted from censuses, household surveys, civil registration and vital statistics, disease registries, health service use, air pollution monitors, satellite imaging, disease notifications, and other sources. Cause-specific death rates and cause fractions were calculated using the Cause of Death Ensemble model and spatiotemporal Gaussian process regression. Cause-specific deaths were adjusted to match the total all-cause deaths calculated as part of the GBD population, fertility, and mortality estimates. Deaths were multiplied by standard life expectancy at each age to calculate YLLs. A Bayesian meta-regression modelling tool, DisMod-MR 2.1, was used to ensure consistency between incidence, prevalence, remission, excess mortality, and cause-specific mortality for most causes. Prevalence estimates were multiplied by disability weights for mutually exclusive sequelae of diseases and injuries to calculate YLDs. We considered results in the context of the Socio-demographic Index (SDI), a composite indicator of income per capita, years of schooling, and fertility rate in females younger than 25 years. Uncertainty intervals (UIs) were generated for every metric using the 25th and 975th ordered 1000 draw values of the posterior distribution. Findings: Global health has steadily improved over the past 30 years as measured by age-standardised DALY rates. After taking into account population growth and ageing, the absolute number of DALYs has remained stable. Since 2010, the pace of decline in global age-standardised DALY rates has accelerated in age groups younger than 50 years compared with the 1990–2010 time period, with the greatest annualised rate of decline occurring in the 0–9-year age group. Six infectious diseases were among the top ten causes of DALYs in children younger than 10 years in 2019: lower respiratory infections (ranked second), diarrhoeal diseases (third), malaria (fifth), meningitis (sixth), whooping cough (ninth), and sexually transmitted infections (which, in this age group, is fully accounted for by congenital syphilis; ranked tenth). In adolescents aged 10–24 years, three injury causes were among the top causes of DALYs: road injuries (ranked first), self-harm (third), and interpersonal violence (fifth). Five of the causes that were in the top ten for ages 10–24 years were also in the top ten in the 25–49-year age group: road injuries (ranked first), HIV/AIDS (second), low back pain (fourth), headache disorders (fifth), and depressive disorders (sixth). In 2019, ischaemic heart disease and stroke were the top-ranked causes of DALYs in both the 50–74-year and 75-years-and-older age groups. Since 1990, there has been a marked shift towards a greater proportion of burden due to YLDs from non-communicable diseases and injuries. In 2019, there were 11 countries where non-communicable disease and injury YLDs constituted more than half of all disease burden. Decreases in age-standardised DALY rates have accelerated over the past decade in countries at the lower end of the SDI range, while improvements have started to stagnate or even reverse in countries with higher SDI. Interpretation: As disability becomes an increasingly large component of disease burden and a larger component of health expenditure, greater research and developm nt investment is needed to identify new, more effective intervention strategies. With a rapidly ageing global population, the demands on health services to deal with disabling outcomes, which increase with age, will require policy makers to anticipate these changes. The mix of universal and more geographically specific influences on health reinforces the need for regular reporting on population health in detail and by underlying cause to help decision makers to identify success stories of disease control to emulate, as well as opportunities to improve. Funding: Bill & Melinda Gates Foundation. © 2020 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 licens

Global age-sex-specific fertility, mortality, healthy life expectancy (HALE), and population estimates in 204 countries and territories, 1950-2019 : a comprehensive demographic analysis for the Global Burden of Disease Study 2019

Background: Accurate and up-to-date assessment of demographic metrics is crucial for understanding a wide range of social, economic, and public health issues that affect populations worldwide. The Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) 2019 produced updated and comprehensive demographic assessments of the key indicators of fertility, mortality, migration, and population for 204 countries and territories and selected subnational locations from 1950 to 2019. Methods: 8078 country-years of vital registration and sample registration data, 938 surveys, 349 censuses, and 238 other sources were identified and used to estimate age-specific fertility. Spatiotemporal Gaussian process regression (ST-GPR) was used to generate age-specific fertility rates for 5-year age groups between ages 15 and 49 years. With extensions to age groups 10–14 and 50–54 years, the total fertility rate (TFR) was then aggregated using the estimated age-specific fertility between ages 10 and 54 years. 7417 sources were used for under-5 mortality estimation and 7355 for adult mortality. ST-GPR was used to synthesise data sources after correction for known biases. Adult mortality was measured as the probability of death between ages 15 and 60 years based on vital registration, sample registration, and sibling histories, and was also estimated using ST-GPR. HIV-free life tables were then estimated using estimates of under-5 and adult mortality rates using a relational model life table system created for GBD, which closely tracks observed age-specific mortality rates from complete vital registration when available. Independent estimates of HIV-specific mortality generated by an epidemiological analysis of HIV prevalence surveys and antenatal clinic serosurveillance and other sources were incorporated into the estimates in countries with large epidemics. Annual and single-year age estimates of net migration and population for each country and territory were generated using a Bayesian hierarchical cohort component model that analysed estimated age-specific fertility and mortality rates along with 1250 censuses and 747 population registry years. We classified location-years into seven categories on the basis of the natural rate of increase in population (calculated by subtracting the crude death rate from the crude birth rate) and the net migration rate. We computed healthy life expectancy (HALE) using years lived with disability (YLDs) per capita, life tables, and standard demographic methods. Uncertainty was propagated throughout the demographic estimation process, including fertility, mortality, and population, with 1000 draw-level estimates produced for each metric. Findings: The global TFR decreased from 2·72 (95% uncertainty interval [UI] 2·66–2·79) in 2000 to 2·31 (2·17–2·46) in 2019. Global annual livebirths increased from 134·5 million (131·5–137·8) in 2000 to a peak of 139·6 million (133·0–146·9) in 2016. Global livebirths then declined to 135·3 million (127·2–144·1) in 2019. Of the 204 countries and territories included in this study, in 2019, 102 had a TFR lower than 2·1, which is considered a good approximation of replacement-level fertility. All countries in sub-Saharan Africa had TFRs above replacement level in 2019 and accounted for 27·1% (95% UI 26·4–27·8) of global livebirths. Global life expectancy at birth increased from 67·2 years (95% UI 66·8–67·6) in 2000 to 73·5 years (72·8–74·3) in 2019. The total number of deaths increased from 50·7 million (49·5–51·9) in 2000 to 56·5 million (53·7–59·2) in 2019. Under-5 deaths declined from 9·6 million (9·1–10·3) in 2000 to 5·0 million (4·3–6·0) in 2019. Global population increased by 25·7%, from 6·2 billion (6·0–6·3) in 2000 to 7·7 billion (7·5–8·0) in 2019. In 2019, 34 countries had negative natural rates of increase; in 17 of these, the population declined because immigration was not sufficient to counteract the negative rate of decline. Globally, HALE increased from 58·6 years (56·1–60·8) in 2000 to 63·5 years (60·8–66·1) in 2019. HALE increased in 202 of 204 countries and territories between 2000 and 2019

A reference map of the human binary protein interactome.

Global insights into cellular organization and genome function require comprehensive understanding of the interactome networks that mediate genotype-phenotype relationships(1,2). Here we present a human 'all-by-all' reference interactome map of human binary protein interactions, or 'HuRI'. With approximately 53,000 protein-protein interactions, HuRI has approximately four times as many such interactions as there are high-quality curated interactions from small-scale studies. The integration of HuRI with genome(3), transcriptome(4) and proteome(5) data enables cellular function to be studied within most physiological or pathological cellular contexts. We demonstrate the utility of HuRI in identifying the specific subcellular roles of protein-protein interactions. Inferred tissue-specific networks reveal general principles for the formation of cellular context-specific functions and elucidate potential molecular mechanisms that might underlie tissue-specific phenotypes of Mendelian diseases. HuRI is a systematic proteome-wide reference that links genomic variation to phenotypic outcomes

Healthcare Access and Quality Index based on mortality from causes amenable to personal health care in 195 countries and territories, 1990-2015 : a novel analysis from the Global Burden of Disease Study 2015

Background National levels of personal health-care access and quality can be approximated by measuring mortality rates from causes that should not be fatal in the presence of effective medical care (ie, amenable mortality). Previous analyses of mortality amenable to health care only focused on high-income countries and faced several methodological challenges. In the present analysis, we use the highly standardised cause of death and risk factor estimates generated through the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) to improve and expand the quantification of personal health-care access and quality for 195 countries and territories from 1990 to 2015. Methods We mapped the most widely used list of causes amenable to personal health care developed by Nolte and McKee to 32 GBD causes. We accounted for variations in cause of death certification and misclassifications through the extensive data standardisation processes and redistribution algorithms developed for GBD. To isolate the effects of personal health-care access and quality, we risk-standardised cause-specific mortality rates for each geography-year by removing the joint effects of local environmental and behavioural risks, and adding back the global levels of risk exposure as estimated for GBD 2015. We employed principal component analysis to create a single, interpretable summary measure-the Healthcare Quality and Access (HAQ) Index-on a scale of 0 to 100. The HAQ Index showed strong convergence validity as compared with other health-system indicators, including health expenditure per capita (r= 0.88), an index of 11 universal health coverage interventions (r= 0.83), and human resources for health per 1000 (r= 0.77). We used free disposal hull analysis with bootstrapping to produce a frontier based on the relationship between the HAQ Index and the Socio-demographic Index (SDI), a measure of overall development consisting of income per capita, average years of education, and total fertility rates. This frontier allowed us to better quantify the maximum levels of personal health-care access and quality achieved across the development spectrum, and pinpoint geographies where gaps between observed and potential levels have narrowed or widened over time. Findings Between 1990 and 2015, nearly all countries and territories saw their HAQ Index values improve; nonetheless, the difference between the highest and lowest observed HAQ Index was larger in 2015 than in 1990, ranging from 28.6 to 94.6. Of 195 geographies, 167 had statistically significant increases in HAQ Index levels since 1990, with South Korea, Turkey, Peru, China, and the Maldives recording among the largest gains by 2015. Performance on the HAQ Index and individual causes showed distinct patterns by region and level of development, yet substantial heterogeneities emerged for several causes, including cancers in highest-SDI countries; chronic kidney disease, diabetes, diarrhoeal diseases, and lower respiratory infections among middle-SDI countries; and measles and tetanus among lowest-SDI countries. While the global HAQ Index average rose from 40.7 (95% uncertainty interval, 39.0-42.8) in 1990 to 53.7 (52.2-55.4) in 2015, far less progress occurred in narrowing the gap between observed HAQ Index values and maximum levels achieved; at the global level, the difference between the observed and frontier HAQ Index only decreased from 21.2 in 1990 to 20.1 in 2015. If every country and territory had achieved the highest observed HAQ Index by their corresponding level of SDI, the global average would have been 73.8 in 2015. Several countries, particularly in eastern and western sub-Saharan Africa, reached HAQ Index values similar to or beyond their development levels, whereas others, namely in southern sub-Saharan Africa, the Middle East, and south Asia, lagged behind what geographies of similar development attained between 1990 and 2015. Interpretation This novel extension of the GBD Study shows the untapped potential for personal health-care access and quality improvement across the development spectrum. Amid substantive advances in personal health care at the national level, heterogeneous patterns for individual causes in given countries or territories suggest that few places have consistently achieved optimal health-care access and quality across health-system functions and therapeutic areas. This is especially evident in middle-SDI countries, many of which have recently undergone or are currently experiencing epidemiological transitions. The HAQ Index, if paired with other measures of health-systemcharacteristics such as intervention coverage, could provide a robust avenue for tracking progress on universal health coverage and identifying local priorities for strengthening personal health-care quality and access throughout the world. Copyright (C) The Author(s). Published by Elsevier Ltd.Peer reviewe

Derandomized constructions of k-wise (almost) independent permutations

Abstract Constructions of k-wise almost independent permutations have been receiving a growingamount of attention in recent years. However, unlike the case of k-wise independent functions,the size of previously constructed families of such permutations is far from optimal. This paper gives a new method for reducing the size of families given by previous constructions. Ourmethod relies on pseudorandom generators for space-bounded computations. In fact, all we need is a generator, that produces &amp;quot;pseudorandom walks &amp;quot; on undirected graphs with a consis-tent labelling. One such generator is implied by Reingold&apos;s log-space algorithm for undirected connectivity [35, 36]. We obtain families of k-wise almost independent permutations, with anoptimal description length, up to a constant factor. More precisely, if the distance from uniform for any k tuple should be at most ffi, then the size of the description of a permutation inthe family is O(kn + log 1ffi). 1 Introduction In explicit constructions of pseudorandom objects, we are interested in simulating a large randomobject using a succinct one and would like to capture some essential properties of the former. A natural way to phrase such a requirement is via limited access. Suppose the object that we areinterested in simulating is a random function f: {0, 1}n 7! {0, 1}n and we want to come up witha small family of functions G that simulates it. The k-wise independence requirement in this caseis that a function g chosen at random from G be completely indistinguishable from a function fchosen at random from the set of all functions, for any process that receives the value of eithe

Lossy Chains and Fractional Secret Sharing ∗

Motivated by the goal of controlling the amount of work required to access a shared resource or to solve a cryptographic puzzle, we introduce and study the related notions of lossy chains and fractional secret sharing. Fractional secret sharing generalizes traditional secret sharing by allowing a finegrained control over the amount of uncertainty about the secret. More concretely, a fractional secret sharing scheme realizes a fractional access structure f: 2 [n] → [m] by guaranteeing that from the point of view of each set T ⊆ [n] of parties, the secret is uniformly distributed over a set of f(T) potential secrets. We show that every (monotone) fractional access structure can be realized. For symmetric structures, in which f(T) depends only on the size of T, we give an efficient construction with share size poly(n, log m). Our construction of fractional secret sharing schemes is based on the new notion of lossy chains which may be of independent interest. A lossy chain is a Markov chain (X0,..., Xn) which starts with a random secret X0 and gradually loses information about it at a rate which is specified by a loss function g. Concretely, in every step t, the distribution of X0 conditioned on the value of Xt should always be uniformly distributed over a set of size g(t). We show how to construct such lossy chains efficiently for any possible loss function g, and prove that our construction achieves an optimal asymptotic information rate.