Zero-Shot Learning with Common Sense Knowledge Graphs

Zero-shot learning relies on semantic class representations such as hand-engineered attributes or learned embeddings to predict classes without any labeled examples. We propose to learn class representations from common sense knowledge graphs. Common sense knowledge graphs are an untapped source of explicit high-level knowledge that requires little human effort to apply to a range of tasks. To capture the knowledge in the graph, we introduce ZSL-KG, a general-purpose framework with a novel transformer graph convolutional network (TrGCN) for generating class representations. Our proposed TrGCN architecture computes non-linear combinations of the node neighbourhood and shows improvements on zero-shot learning tasks in language and vision. Our results show ZSL-KG outperforms the best performing graph-based zero-shot learning framework by an average of 2.1 accuracy points with improvements as high as 3.4 accuracy points. Our ablation study on ZSL-KG with alternate graph neural networks shows that our TrGCN adds up to 1.2 accuracy points improvement on these tasks

Does CLIP Bind Concepts? Probing Compositionality in Large Image Models

Large-scale neural network models combining text and images have made incredible progress in recent years. However, it remains an open question to what extent such models encode compositional representations of the concepts over which they operate, such as correctly identifying ''red cube'' by reasoning over the constituents ''red'' and ''cube''. In this work, we focus on the ability of a large pretrained vision and language model (CLIP) to encode compositional concepts and to bind variables in a structure-sensitive way (e.g., differentiating ''cube behind sphere'' from ''sphere behind cube''). In order to inspect the performance of CLIP, we compare several architectures from research on compositional distributional semantics models (CDSMs), a line of research that attempts to implement traditional compositional linguistic structures within embedding spaces. We find that CLIP can compose concepts in a single-object setting, but in situations where concept binding is needed, performance drops dramatically. At the same time, CDSMs also perform poorly, with best performance at chance level

Multitask Prompted Training Enables Zero-Shot Task Generalization

International audienceLarge language models have recently been shown to attain reasonable zero-shot generalization on a diverse set of tasks (Brown et al., 2020). It has been hypothesized that this is a consequence of implicit multitask learning in language models’ pretraining (Radford et al., 2019). Can zero-shot generalization instead be directly induced by explicit multitask learning? To test this question at scale, we develop a system for easily mapping any natural language tasks into a human-readable prompted form. We convert a large set of supervised datasets, each with multiple prompts with diverse wording. These prompted datasets allow for benchmarking the ability of a model to perform completely held-out tasks. We fine-tune a pre-trained encoder-decoder model (Raffel et al., 2020; Lester et al., 2021) on this multitask mixture covering a wide variety of tasks. The model attains strong zero-shot performance on several standard datasets, often outperforming models up to 16x its size. Further, our approach attains strong performance on a subset of tasks from the BIG-bench benchmark, outperforming models up to 6x its size. All trained models are available at https://github.com/bigscience-workshop/t-zero, and all prompts are available at https://github.com/bigscience-workshop/promptsource

Alcohol use and burden for 195 countries and territories, 1990-2016 : a systematic analysis for the Global Burden of Disease Study 2016

Background Alcohol use is a leading risk factor for death and disability, but its overall association with health remains complex given the possible protective effects of moderate alcohol consumption on some conditions. With our comprehensive approach to health accounting within the Global Burden of Diseases, Injuries, and Risk Factors Study 2016, we generated improved estimates of alcohol use and alcohol-attributable deaths and disability-adjusted life-years (DALYs) for 195 locations from 1990 to 2016, for both sexes and for 5-year age groups between the ages of 15 years and 95 years and older. Methods Using 694 data sources of individual and population-level alcohol consumption, along with 592 prospective and retrospective studies on the risk of alcohol use, we produced estimates of the prevalence of current drinking, abstention, the distribution of alcohol consumption among current drinkers in standard drinks daily (defined as 10 g of pure ethyl alcohol), and alcohol-attributable deaths and DALYs. We made several methodological improvements compared with previous estimates: first, we adjusted alcohol sales estimates to take into account tourist and unrecorded consumption; second, we did a new meta-analysis of relative risks for 23 health outcomes associated with alcohol use; and third, we developed a new method to quantify the level of alcohol consumption that minimises the overall risk to individual health. Findings Globally, alcohol use was the seventh leading risk factor for both deaths and DALYs in 2016, accounting for 2.2% (95% uncertainty interval [UI] 1.5-3.0) of age-standardised female deaths and 6.8% (5.8-8.0) of age-standardised male deaths. Among the population aged 15-49 years, alcohol use was the leading risk factor globally in 2016, with 3.8% (95% UI 3.2-4-3) of female deaths and 12.2% (10.8-13-6) of male deaths attributable to alcohol use. For the population aged 15-49 years, female attributable DALYs were 2.3% (95% UI 2.0-2.6) and male attributable DALYs were 8.9% (7.8-9.9). The three leading causes of attributable deaths in this age group were tuberculosis (1.4% [95% UI 1. 0-1. 7] of total deaths), road injuries (1.2% [0.7-1.9]), and self-harm (1.1% [0.6-1.5]). For populations aged 50 years and older, cancers accounted for a large proportion of total alcohol-attributable deaths in 2016, constituting 27.1% (95% UI 21.2-33.3) of total alcohol-attributable female deaths and 18.9% (15.3-22.6) of male deaths. The level of alcohol consumption that minimised harm across health outcomes was zero (95% UI 0.0-0.8) standard drinks per week. Interpretation Alcohol use is a leading risk factor for global disease burden and causes substantial health loss. We found that the risk of all-cause mortality, and of cancers specifically, rises with increasing levels of consumption, and the level of consumption that minimises health loss is zero. These results suggest that alcohol control policies might need to be revised worldwide, refocusing on efforts to lower overall population-level consumption.Peer reviewe

The global burden of adolescent and young adult cancer in 2019 : a systematic analysis for the Global Burden of Disease Study 2019

Background In estimating the global burden of cancer, adolescents and young adults with cancer are often overlooked, despite being a distinct subgroup with unique epidemiology, clinical care needs, and societal impact. Comprehensive estimates of the global cancer burden in adolescents and young adults (aged 15-39 years) are lacking. To address this gap, we analysed results from the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) 2019, with a focus on the outcome of disability-adjusted life-years (DALYs), to inform global cancer control measures in adolescents and young adults. Methods Using the GBD 2019 methodology, international mortality data were collected from vital registration systems, verbal autopsies, and population-based cancer registry inputs modelled with mortality-to-incidence ratios (MIRs). Incidence was computed with mortality estimates and corresponding MIRs. Prevalence estimates were calculated using modelled survival and multiplied by disability weights to obtain years lived with disability (YLDs). Years of life lost (YLLs) were calculated as age-specific cancer deaths multiplied by the standard life expectancy at the age of death. The main outcome was DALYs (the sum of YLLs and YLDs). Estimates were presented globally and by Socio-demographic Index (SDI) quintiles (countries ranked and divided into five equal SDI groups), and all estimates were presented with corresponding 95% uncertainty intervals (UIs). For this analysis, we used the age range of 15-39 years to define adolescents and young adults. Findings There were 1.19 million (95% UI 1.11-1.28) incident cancer cases and 396 000 (370 000-425 000) deaths due to cancer among people aged 15-39 years worldwide in 2019. The highest age-standardised incidence rates occurred in high SDI (59.6 [54.5-65.7] per 100 000 person-years) and high-middle SDI countries (53.2 [48.8-57.9] per 100 000 person-years), while the highest age-standardised mortality rates were in low-middle SDI (14.2 [12.9-15.6] per 100 000 person-years) and middle SDI (13.6 [12.6-14.8] per 100 000 person-years) countries. In 2019, adolescent and young adult cancers contributed 23.5 million (21.9-25.2) DALYs to the global burden of disease, of which 2.7% (1.9-3.6) came from YLDs and 97.3% (96.4-98.1) from YLLs. Cancer was the fourth leading cause of death and tenth leading cause of DALYs in adolescents and young adults globally. Interpretation Adolescent and young adult cancers contributed substantially to the overall adolescent and young adult disease burden globally in 2019. These results provide new insights into the distribution and magnitude of the adolescent and young adult cancer burden around the world. With notable differences observed across SDI settings, these estimates can inform global and country-level cancer control efforts. Copyright (C) 2021 The Author(s). Published by Elsevier Ltd.Peer reviewe

Adolescent transport and unintentional injuries: a systematic analysis using the Global Burden of Disease Study 2019

Background: Globally, transport and unintentional injuries persist as leading preventable causes of mortality and morbidity for adolescents. We sought to report comprehensive trends in injury-related mortality and morbidity for adolescents aged 10–24 years during the past three decades. Methods: Using the Global Burden of Disease, Injuries, and Risk Factors 2019 Study, we analysed mortality and disability-adjusted life-years (DALYs) attributed to transport and unintentional injuries for adolescents in 204 countries. Burden is reported in absolute numbers and age-standardised rates per 100 000 population by sex, age group (10–14, 15–19, and 20–24 years), and sociodemographic index (SDI) with 95% uncertainty intervals (UIs). We report percentage changes in deaths and DALYs between 1990 and 2019. Findings: In 2019, 369 061 deaths (of which 214 337 [58%] were transport related) and 31·1 million DALYs (of which 16·2 million [52%] were transport related) among adolescents aged 10–24 years were caused by transport and unintentional injuries combined. If compared with other causes, transport and unintentional injuries combined accounted for 25% of deaths and 14% of DALYs in 2019, and showed little improvement from 1990 when such injuries accounted for 26% of adolescent deaths and 17% of adolescent DALYs. Throughout adolescence, transport and unintentional injury fatality rates increased by age group. The unintentional injury burden was higher among males than females for all injury types, except for injuries related to fire, heat, and hot substances, or to adverse effects of medical treatment. From 1990 to 2019, global mortality rates declined by 34·4% (from 17·5 to 11·5 per 100 000) for transport injuries, and by 47·7% (from 15·9 to 8·3 per 100 000) for unintentional injuries. However, in low-SDI nations the absolute number of deaths increased (by 80·5% to 42 774 for transport injuries and by 39·4% to 31 961 for unintentional injuries). In the high-SDI quintile in 2010–19, the rate per 100 000 of transport injury DALYs was reduced by 16·7%, from 838 in 2010 to 699 in 2019. This was a substantially slower pace of reduction compared with the 48·5% reduction between 1990 and 2010, from 1626 per 100 000 in 1990 to 838 per 100 000 in 2010. Between 2010 and 2019, the rate of unintentional injury DALYs per 100 000 also remained largely unchanged in high-SDI countries (555 in 2010 vs 554 in 2019; 0·2% reduction). The number and rate of adolescent deaths and DALYs owing to environmental heat and cold exposure increased for the high-SDI quintile during 2010–19. Interpretation: As other causes of mortality are addressed, inadequate progress in reducing transport and unintentional injury mortality as a proportion of adolescent deaths becomes apparent. The relative shift in the burden of injury from high-SDI countries to low and low–middle-SDI countries necessitates focused action, including global donor, government, and industry investment in injury prevention. The persisting burden of DALYs related to transport and unintentional injuries indicates a need to prioritise innovative measures for the primary prevention of adolescent injury. Funding: Bill & Melinda Gates Foundation

Global burden of 369 diseases and injuries in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019

Background: In an era of shifting global agendas and expanded emphasis on non-communicable diseases and injuries along with communicable diseases, sound evidence on trends by cause at the national level is essential. The Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) provides a systematic scientific assessment of published, publicly available, and contributed data on incidence, prevalence, and mortality for a mutually exclusive and collectively exhaustive list of diseases and injuries. Methods: GBD estimates incidence, prevalence, mortality, years of life lost (YLLs), years lived with disability (YLDs), and disability-adjusted life-years (DALYs) due to 369 diseases and injuries, for two sexes, and for 204 countries and territories. Input data were extracted from censuses, household surveys, civil registration and vital statistics, disease registries, health service use, air pollution monitors, satellite imaging, disease notifications, and other sources. Cause-specific death rates and cause fractions were calculated using the Cause of Death Ensemble model and spatiotemporal Gaussian process regression. Cause-specific deaths were adjusted to match the total all-cause deaths calculated as part of the GBD population, fertility, and mortality estimates. Deaths were multiplied by standard life expectancy at each age to calculate YLLs. A Bayesian meta-regression modelling tool, DisMod-MR 2.1, was used to ensure consistency between incidence, prevalence, remission, excess mortality, and cause-specific mortality for most causes. Prevalence estimates were multiplied by disability weights for mutually exclusive sequelae of diseases and injuries to calculate YLDs. We considered results in the context of the Socio-demographic Index (SDI), a composite indicator of income per capita, years of schooling, and fertility rate in females younger than 25 years. Uncertainty intervals (UIs) were generated for every metric using the 25th and 975th ordered 1000 draw values of the posterior distribution. Findings: Global health has steadily improved over the past 30 years as measured by age-standardised DALY rates. After taking into account population growth and ageing, the absolute number of DALYs has remained stable. Since 2010, the pace of decline in global age-standardised DALY rates has accelerated in age groups younger than 50 years compared with the 1990–2010 time period, with the greatest annualised rate of decline occurring in the 0–9-year age group. Six infectious diseases were among the top ten causes of DALYs in children younger than 10 years in 2019: lower respiratory infections (ranked second), diarrhoeal diseases (third), malaria (fifth), meningitis (sixth), whooping cough (ninth), and sexually transmitted infections (which, in this age group, is fully accounted for by congenital syphilis; ranked tenth). In adolescents aged 10–24 years, three injury causes were among the top causes of DALYs: road injuries (ranked first), self-harm (third), and interpersonal violence (fifth). Five of the causes that were in the top ten for ages 10–24 years were also in the top ten in the 25–49-year age group: road injuries (ranked first), HIV/AIDS (second), low back pain (fourth), headache disorders (fifth), and depressive disorders (sixth). In 2019, ischaemic heart disease and stroke were the top-ranked causes of DALYs in both the 50–74-year and 75-years-and-older age groups. Since 1990, there has been a marked shift towards a greater proportion of burden due to YLDs from non-communicable diseases and injuries. In 2019, there were 11 countries where non-communicable disease and injury YLDs constituted more than half of all disease burden. Decreases in age-standardised DALY rates have accelerated over the past decade in countries at the lower end of the SDI range, while improvements have started to stagnate or even reverse in countries with higher SDI. Interpretation: As disability becomes an increasingly large component of disease burden and a larger component of health expenditure, greater research and developm nt investment is needed to identify new, more effective intervention strategies. With a rapidly ageing global population, the demands on health services to deal with disabling outcomes, which increase with age, will require policy makers to anticipate these changes. The mix of universal and more geographically specific influences on health reinforces the need for regular reporting on population health in detail and by underlying cause to help decision makers to identify success stories of disease control to emulate, as well as opportunities to improve. Funding: Bill & Melinda Gates Foundation. © 2020 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 licens

Mapping local patterns of childhood overweight and wasting in low- and middle-income countries between 2000 and 2017

A double burden of malnutrition occurs when individuals, household members or communities experience both undernutrition and overweight. Here, we show geospatial estimates of overweight and wasting prevalence among children under 5 years of age in 105 low- and middle-income countries (LMICs) from 2000 to 2017 and aggregate these to policy-relevant administrative units. Wasting decreased overall across LMICs between 2000 and 2017, from 8.4% (62.3 (55.1–70.8) million) to 6.4% (58.3 (47.6–70.7) million), but is predicted to remain above the World Health Organization’s Global Nutrition Target of <5% in over half of LMICs by 2025. Prevalence of overweight increased from 5.2% (30 (22.8–38.5) million) in 2000 to 6.0% (55.5 (44.8–67.9) million) children aged under 5 years in 2017. Areas most affected by double burden of malnutrition were located in Indonesia, Thailand, southeastern China, Botswana, Cameroon and central Nigeria. Our estimates provide a new perspective to researchers, policy makers and public health agencies in their efforts to address this global childhood syndemic

Burden of disease scenarios for 204 countries and territories, 2022–2050: a forecasting analysis for the Global Burden of Disease Study 2021

Background: Future trends in disease burden and drivers of health are of great interest to policy makers and the public at large. This information can be used for policy and long-term health investment, planning, and prioritisation. We have expanded and improved upon previous forecasts produced as part of the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) and provide a reference forecast (the most likely future), and alternative scenarios assessing disease burden trajectories if selected sets of risk factors were eliminated from current levels by 2050. Methods: Using forecasts of major drivers of health such as the Socio-demographic Index (SDI; a composite measure of lag-distributed income per capita, mean years of education, and total fertility under 25 years of age) and the full set of risk factor exposures captured by GBD, we provide cause-specific forecasts of mortality, years of life lost (YLLs), years lived with disability (YLDs), and disability-adjusted life-years (DALYs) by age and sex from 2022 to 2050 for 204 countries and territories, 21 GBD regions, seven super-regions, and the world. All analyses were done at the cause-specific level so that only risk factors deemed causal by the GBD comparative risk assessment influenced future trajectories of mortality for each disease. Cause-specific mortality was modelled using mixed-effects models with SDI and time as the main covariates, and the combined impact of causal risk factors as an offset in the model. At the all-cause mortality level, we captured unexplained variation by modelling residuals with an autoregressive integrated moving average model with drift attenuation. These all-cause forecasts constrained the cause-specific forecasts at successively deeper levels of the GBD cause hierarchy using cascading mortality models, thus ensuring a robust estimate of cause-specific mortality. For non-fatal measures (eg, low back pain), incidence and prevalence were forecasted from mixed-effects models with SDI as the main covariate, and YLDs were computed from the resulting prevalence forecasts and average disability weights from GBD. Alternative future scenarios were constructed by replacing appropriate reference trajectories for risk factors with hypothetical trajectories of gradual elimination of risk factor exposure from current levels to 2050. The scenarios were constructed from various sets of risk factors: environmental risks (Safer Environment scenario), risks associated with communicable, maternal, neonatal, and nutritional diseases (CMNNs; Improved Childhood Nutrition and Vaccination scenario), risks associated with major non-communicable diseases (NCDs; Improved Behavioural and Metabolic Risks scenario), and the combined effects of these three scenarios. Using the Shared Socioeconomic Pathways climate scenarios SSP2-4.5 as reference and SSP1-1.9 as an optimistic alternative in the Safer Environment scenario, we accounted for climate change impact on health by using the most recent Intergovernmental Panel on Climate Change temperature forecasts and published trajectories of ambient air pollution for the same two scenarios. Life expectancy and healthy life expectancy were computed using standard methods. The forecasting framework includes computing the age-sex-specific future population for each location and separately for each scenario. 95% uncertainty intervals (UIs) for each individual future estimate were derived from the 2·5th and 97·5th percentiles of distributions generated from propagating 500 draws through the multistage computational pipeline. Findings: In the reference scenario forecast, global and super-regional life expectancy increased from 2022 to 2050, but improvement was at a slower pace than in the three decades preceding the COVID-19 pandemic (beginning in 2020). Gains in future life expectancy were forecasted to be greatest in super-regions with comparatively low life expectancies (such as sub-Saharan Africa) compared with super-regions with higher life expectancies (such as the high-income super-region), leading to a trend towards convergence in life expectancy across locations between now and 2050. At the super-region level, forecasted healthy life expectancy patterns were similar to those of life expectancies. Forecasts for the reference scenario found that health will improve in the coming decades, with all-cause age-standardised DALY rates decreasing in every GBD super-region. The total DALY burden measured in counts, however, will increase in every super-region, largely a function of population ageing and growth. We also forecasted that both DALY counts and age-standardised DALY rates will continue to shift from CMNNs to NCDs, with the most pronounced shifts occurring in sub-Saharan Africa (60·1% [95% UI 56·8–63·1] of DALYs were from CMNNs in 2022 compared with 35·8% [31·0–45·0] in 2050) and south Asia (31·7% [29·2–34·1] to 15·5% [13·7–17·5]). This shift is reflected in the leading global causes of DALYs, with the top four causes in 2050 being ischaemic heart disease, stroke, diabetes, and chronic obstructive pulmonary disease, compared with 2022, with ischaemic heart disease, neonatal disorders, stroke, and lower respiratory infections at the top. The global proportion of DALYs due to YLDs likewise increased from 33·8% (27·4–40·3) to 41·1% (33·9–48·1) from 2022 to 2050, demonstrating an important shift in overall disease burden towards morbidity and away from premature death. The largest shift of this kind was forecasted for sub-Saharan Africa, from 20·1% (15·6–25·3) of DALYs due to YLDs in 2022 to 35·6% (26·5–43·0) in 2050. In the assessment of alternative future scenarios, the combined effects of the scenarios (Safer Environment, Improved Childhood Nutrition and Vaccination, and Improved Behavioural and Metabolic Risks scenarios) demonstrated an important decrease in the global burden of DALYs in 2050 of 15·4% (13·5–17·5) compared with the reference scenario, with decreases across super-regions ranging from 10·4% (9·7–11·3) in the high-income super-region to 23·9% (20·7–27·3) in north Africa and the Middle East. The Safer Environment scenario had its largest decrease in sub-Saharan Africa (5·2% [3·5–6·8]), the Improved Behavioural and Metabolic Risks scenario in north Africa and the Middle East (23·2% [20·2–26·5]), and the Improved Nutrition and Vaccination scenario in sub-Saharan Africa (2·0% [–0·6 to 3·6]). Interpretation: Globally, life expectancy and age-standardised disease burden were forecasted to improve between 2022 and 2050, with the majority of the burden continuing to shift from CMNNs to NCDs. That said, continued progress on reducing the CMNN disease burden will be dependent on maintaining investment in and policy emphasis on CMNN disease prevention and treatment. Mostly due to growth and ageing of populations, the number of deaths and DALYs due to all causes combined will generally increase. By constructing alternative future scenarios wherein certain risk exposures are eliminated by 2050, we have shown that opportunities exist to substantially improve health outcomes in the future through concerted efforts to prevent exposure to well established risk factors and to expand access to key health interventions