Zero-Shot Learning with Common Sense Knowledge Graphs

Zero-shot learning relies on semantic class representations such as hand-engineered attributes or learned embeddings to predict classes without any labeled examples. We propose to learn class representations from common sense knowledge graphs. Common sense knowledge graphs are an untapped source of explicit high-level knowledge that requires little human effort to apply to a range of tasks. To capture the knowledge in the graph, we introduce ZSL-KG, a general-purpose framework with a novel transformer graph convolutional network (TrGCN) for generating class representations. Our proposed TrGCN architecture computes non-linear combinations of the node neighbourhood and shows improvements on zero-shot learning tasks in language and vision. Our results show ZSL-KG outperforms the best performing graph-based zero-shot learning framework by an average of 2.1 accuracy points with improvements as high as 3.4 accuracy points. Our ablation study on ZSL-KG with alternate graph neural networks shows that our TrGCN adds up to 1.2 accuracy points improvement on these tasks

Does CLIP Bind Concepts? Probing Compositionality in Large Image Models

Large-scale neural network models combining text and images have made incredible progress in recent years. However, it remains an open question to what extent such models encode compositional representations of the concepts over which they operate, such as correctly identifying ''red cube'' by reasoning over the constituents ''red'' and ''cube''. In this work, we focus on the ability of a large pretrained vision and language model (CLIP) to encode compositional concepts and to bind variables in a structure-sensitive way (e.g., differentiating ''cube behind sphere'' from ''sphere behind cube''). In order to inspect the performance of CLIP, we compare several architectures from research on compositional distributional semantics models (CDSMs), a line of research that attempts to implement traditional compositional linguistic structures within embedding spaces. We find that CLIP can compose concepts in a single-object setting, but in situations where concept binding is needed, performance drops dramatically. At the same time, CDSMs also perform poorly, with best performance at chance level

Multitask Prompted Training Enables Zero-Shot Task Generalization

International audienceLarge language models have recently been shown to attain reasonable zero-shot generalization on a diverse set of tasks (Brown et al., 2020). It has been hypothesized that this is a consequence of implicit multitask learning in language models’ pretraining (Radford et al., 2019). Can zero-shot generalization instead be directly induced by explicit multitask learning? To test this question at scale, we develop a system for easily mapping any natural language tasks into a human-readable prompted form. We convert a large set of supervised datasets, each with multiple prompts with diverse wording. These prompted datasets allow for benchmarking the ability of a model to perform completely held-out tasks. We fine-tune a pre-trained encoder-decoder model (Raffel et al., 2020; Lester et al., 2021) on this multitask mixture covering a wide variety of tasks. The model attains strong zero-shot performance on several standard datasets, often outperforming models up to 16x its size. Further, our approach attains strong performance on a subset of tasks from the BIG-bench benchmark, outperforming models up to 6x its size. All trained models are available at https://github.com/bigscience-workshop/t-zero, and all prompts are available at https://github.com/bigscience-workshop/promptsource

Global burden of 369 diseases and injuries in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019

Background: In an era of shifting global agendas and expanded emphasis on non-communicable diseases and injuries along with communicable diseases, sound evidence on trends by cause at the national level is essential. The Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) provides a systematic scientific assessment of published, publicly available, and contributed data on incidence, prevalence, and mortality for a mutually exclusive and collectively exhaustive list of diseases and injuries. Methods: GBD estimates incidence, prevalence, mortality, years of life lost (YLLs), years lived with disability (YLDs), and disability-adjusted life-years (DALYs) due to 369 diseases and injuries, for two sexes, and for 204 countries and territories. Input data were extracted from censuses, household surveys, civil registration and vital statistics, disease registries, health service use, air pollution monitors, satellite imaging, disease notifications, and other sources. Cause-specific death rates and cause fractions were calculated using the Cause of Death Ensemble model and spatiotemporal Gaussian process regression. Cause-specific deaths were adjusted to match the total all-cause deaths calculated as part of the GBD population, fertility, and mortality estimates. Deaths were multiplied by standard life expectancy at each age to calculate YLLs. A Bayesian meta-regression modelling tool, DisMod-MR 2.1, was used to ensure consistency between incidence, prevalence, remission, excess mortality, and cause-specific mortality for most causes. Prevalence estimates were multiplied by disability weights for mutually exclusive sequelae of diseases and injuries to calculate YLDs. We considered results in the context of the Socio-demographic Index (SDI), a composite indicator of income per capita, years of schooling, and fertility rate in females younger than 25 years. Uncertainty intervals (UIs) were generated for every metric using the 25th and 975th ordered 1000 draw values of the posterior distribution. Findings: Global health has steadily improved over the past 30 years as measured by age-standardised DALY rates. After taking into account population growth and ageing, the absolute number of DALYs has remained stable. Since 2010, the pace of decline in global age-standardised DALY rates has accelerated in age groups younger than 50 years compared with the 1990–2010 time period, with the greatest annualised rate of decline occurring in the 0–9-year age group. Six infectious diseases were among the top ten causes of DALYs in children younger than 10 years in 2019: lower respiratory infections (ranked second), diarrhoeal diseases (third), malaria (fifth), meningitis (sixth), whooping cough (ninth), and sexually transmitted infections (which, in this age group, is fully accounted for by congenital syphilis; ranked tenth). In adolescents aged 10–24 years, three injury causes were among the top causes of DALYs: road injuries (ranked first), self-harm (third), and interpersonal violence (fifth). Five of the causes that were in the top ten for ages 10–24 years were also in the top ten in the 25–49-year age group: road injuries (ranked first), HIV/AIDS (second), low back pain (fourth), headache disorders (fifth), and depressive disorders (sixth). In 2019, ischaemic heart disease and stroke were the top-ranked causes of DALYs in both the 50–74-year and 75-years-and-older age groups. Since 1990, there has been a marked shift towards a greater proportion of burden due to YLDs from non-communicable diseases and injuries. In 2019, there were 11 countries where non-communicable disease and injury YLDs constituted more than half of all disease burden. Decreases in age-standardised DALY rates have accelerated over the past decade in countries at the lower end of the SDI range, while improvements have started to stagnate or even reverse in countries with higher SDI. Interpretation: As disability becomes an increasingly large component of disease burden and a larger component of health expenditure, greater research and developm nt investment is needed to identify new, more effective intervention strategies. With a rapidly ageing global population, the demands on health services to deal with disabling outcomes, which increase with age, will require policy makers to anticipate these changes. The mix of universal and more geographically specific influences on health reinforces the need for regular reporting on population health in detail and by underlying cause to help decision makers to identify success stories of disease control to emulate, as well as opportunities to improve. Funding: Bill & Melinda Gates Foundation. © 2020 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 licens

Alcohol use and burden for 195 countries and territories, 1990-2016 : a systematic analysis for the Global Burden of Disease Study 2016

Background Alcohol use is a leading risk factor for death and disability, but its overall association with health remains complex given the possible protective effects of moderate alcohol consumption on some conditions. With our comprehensive approach to health accounting within the Global Burden of Diseases, Injuries, and Risk Factors Study 2016, we generated improved estimates of alcohol use and alcohol-attributable deaths and disability-adjusted life-years (DALYs) for 195 locations from 1990 to 2016, for both sexes and for 5-year age groups between the ages of 15 years and 95 years and older. Methods Using 694 data sources of individual and population-level alcohol consumption, along with 592 prospective and retrospective studies on the risk of alcohol use, we produced estimates of the prevalence of current drinking, abstention, the distribution of alcohol consumption among current drinkers in standard drinks daily (defined as 10 g of pure ethyl alcohol), and alcohol-attributable deaths and DALYs. We made several methodological improvements compared with previous estimates: first, we adjusted alcohol sales estimates to take into account tourist and unrecorded consumption; second, we did a new meta-analysis of relative risks for 23 health outcomes associated with alcohol use; and third, we developed a new method to quantify the level of alcohol consumption that minimises the overall risk to individual health. Findings Globally, alcohol use was the seventh leading risk factor for both deaths and DALYs in 2016, accounting for 2.2% (95% uncertainty interval [UI] 1.5-3.0) of age-standardised female deaths and 6.8% (5.8-8.0) of age-standardised male deaths. Among the population aged 15-49 years, alcohol use was the leading risk factor globally in 2016, with 3.8% (95% UI 3.2-4-3) of female deaths and 12.2% (10.8-13-6) of male deaths attributable to alcohol use. For the population aged 15-49 years, female attributable DALYs were 2.3% (95% UI 2.0-2.6) and male attributable DALYs were 8.9% (7.8-9.9). The three leading causes of attributable deaths in this age group were tuberculosis (1.4% [95% UI 1. 0-1. 7] of total deaths), road injuries (1.2% [0.7-1.9]), and self-harm (1.1% [0.6-1.5]). For populations aged 50 years and older, cancers accounted for a large proportion of total alcohol-attributable deaths in 2016, constituting 27.1% (95% UI 21.2-33.3) of total alcohol-attributable female deaths and 18.9% (15.3-22.6) of male deaths. The level of alcohol consumption that minimised harm across health outcomes was zero (95% UI 0.0-0.8) standard drinks per week. Interpretation Alcohol use is a leading risk factor for global disease burden and causes substantial health loss. We found that the risk of all-cause mortality, and of cancers specifically, rises with increasing levels of consumption, and the level of consumption that minimises health loss is zero. These results suggest that alcohol control policies might need to be revised worldwide, refocusing on efforts to lower overall population-level consumption.Peer reviewe

The global burden of adolescent and young adult cancer in 2019 : a systematic analysis for the Global Burden of Disease Study 2019

Background In estimating the global burden of cancer, adolescents and young adults with cancer are often overlooked, despite being a distinct subgroup with unique epidemiology, clinical care needs, and societal impact. Comprehensive estimates of the global cancer burden in adolescents and young adults (aged 15-39 years) are lacking. To address this gap, we analysed results from the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) 2019, with a focus on the outcome of disability-adjusted life-years (DALYs), to inform global cancer control measures in adolescents and young adults. Methods Using the GBD 2019 methodology, international mortality data were collected from vital registration systems, verbal autopsies, and population-based cancer registry inputs modelled with mortality-to-incidence ratios (MIRs). Incidence was computed with mortality estimates and corresponding MIRs. Prevalence estimates were calculated using modelled survival and multiplied by disability weights to obtain years lived with disability (YLDs). Years of life lost (YLLs) were calculated as age-specific cancer deaths multiplied by the standard life expectancy at the age of death. The main outcome was DALYs (the sum of YLLs and YLDs). Estimates were presented globally and by Socio-demographic Index (SDI) quintiles (countries ranked and divided into five equal SDI groups), and all estimates were presented with corresponding 95% uncertainty intervals (UIs). For this analysis, we used the age range of 15-39 years to define adolescents and young adults. Findings There were 1.19 million (95% UI 1.11-1.28) incident cancer cases and 396 000 (370 000-425 000) deaths due to cancer among people aged 15-39 years worldwide in 2019. The highest age-standardised incidence rates occurred in high SDI (59.6 [54.5-65.7] per 100 000 person-years) and high-middle SDI countries (53.2 [48.8-57.9] per 100 000 person-years), while the highest age-standardised mortality rates were in low-middle SDI (14.2 [12.9-15.6] per 100 000 person-years) and middle SDI (13.6 [12.6-14.8] per 100 000 person-years) countries. In 2019, adolescent and young adult cancers contributed 23.5 million (21.9-25.2) DALYs to the global burden of disease, of which 2.7% (1.9-3.6) came from YLDs and 97.3% (96.4-98.1) from YLLs. Cancer was the fourth leading cause of death and tenth leading cause of DALYs in adolescents and young adults globally. Interpretation Adolescent and young adult cancers contributed substantially to the overall adolescent and young adult disease burden globally in 2019. These results provide new insights into the distribution and magnitude of the adolescent and young adult cancer burden around the world. With notable differences observed across SDI settings, these estimates can inform global and country-level cancer control efforts. Copyright (C) 2021 The Author(s). Published by Elsevier Ltd.Peer reviewe

Adolescent transport and unintentional injuries: a systematic analysis using the Global Burden of Disease Study 2019

Background: Globally, transport and unintentional injuries persist as leading preventable causes of mortality and morbidity for adolescents. We sought to report comprehensive trends in injury-related mortality and morbidity for adolescents aged 10–24 years during the past three decades. Methods: Using the Global Burden of Disease, Injuries, and Risk Factors 2019 Study, we analysed mortality and disability-adjusted life-years (DALYs) attributed to transport and unintentional injuries for adolescents in 204 countries. Burden is reported in absolute numbers and age-standardised rates per 100 000 population by sex, age group (10–14, 15–19, and 20–24 years), and sociodemographic index (SDI) with 95% uncertainty intervals (UIs). We report percentage changes in deaths and DALYs between 1990 and 2019. Findings: In 2019, 369 061 deaths (of which 214 337 [58%] were transport related) and 31·1 million DALYs (of which 16·2 million [52%] were transport related) among adolescents aged 10–24 years were caused by transport and unintentional injuries combined. If compared with other causes, transport and unintentional injuries combined accounted for 25% of deaths and 14% of DALYs in 2019, and showed little improvement from 1990 when such injuries accounted for 26% of adolescent deaths and 17% of adolescent DALYs. Throughout adolescence, transport and unintentional injury fatality rates increased by age group. The unintentional injury burden was higher among males than females for all injury types, except for injuries related to fire, heat, and hot substances, or to adverse effects of medical treatment. From 1990 to 2019, global mortality rates declined by 34·4% (from 17·5 to 11·5 per 100 000) for transport injuries, and by 47·7% (from 15·9 to 8·3 per 100 000) for unintentional injuries. However, in low-SDI nations the absolute number of deaths increased (by 80·5% to 42 774 for transport injuries and by 39·4% to 31 961 for unintentional injuries). In the high-SDI quintile in 2010–19, the rate per 100 000 of transport injury DALYs was reduced by 16·7%, from 838 in 2010 to 699 in 2019. This was a substantially slower pace of reduction compared with the 48·5% reduction between 1990 and 2010, from 1626 per 100 000 in 1990 to 838 per 100 000 in 2010. Between 2010 and 2019, the rate of unintentional injury DALYs per 100 000 also remained largely unchanged in high-SDI countries (555 in 2010 vs 554 in 2019; 0·2% reduction). The number and rate of adolescent deaths and DALYs owing to environmental heat and cold exposure increased for the high-SDI quintile during 2010–19. Interpretation: As other causes of mortality are addressed, inadequate progress in reducing transport and unintentional injury mortality as a proportion of adolescent deaths becomes apparent. The relative shift in the burden of injury from high-SDI countries to low and low–middle-SDI countries necessitates focused action, including global donor, government, and industry investment in injury prevention. The persisting burden of DALYs related to transport and unintentional injuries indicates a need to prioritise innovative measures for the primary prevention of adolescent injury. Funding: Bill & Melinda Gates Foundation

Mapping local patterns of childhood overweight and wasting in low- and middle-income countries between 2000 and 2017

A double burden of malnutrition occurs when individuals, household members or communities experience both undernutrition and overweight. Here, we show geospatial estimates of overweight and wasting prevalence among children under 5 years of age in 105 low- and middle-income countries (LMICs) from 2000 to 2017 and aggregate these to policy-relevant administrative units. Wasting decreased overall across LMICs between 2000 and 2017, from 8.4% (62.3 (55.1–70.8) million) to 6.4% (58.3 (47.6–70.7) million), but is predicted to remain above the World Health Organization’s Global Nutrition Target of <5% in over half of LMICs by 2025. Prevalence of overweight increased from 5.2% (30 (22.8–38.5) million) in 2000 to 6.0% (55.5 (44.8–67.9) million) children aged under 5 years in 2017. Areas most affected by double burden of malnutrition were located in Indonesia, Thailand, southeastern China, Botswana, Cameroon and central Nigeria. Our estimates provide a new perspective to researchers, policy makers and public health agencies in their efforts to address this global childhood syndemic

Mapping inequalities in exclusive breastfeeding in low- and middle-income countries, 2000–2018

Exclusive breastfeeding (EBF)—giving infants only breast-milk for the first 6 months of life—is a component of optimal breastfeeding practices effective in preventing child morbidity and mortality. EBF practices are known to vary by population and comparable subnational estimates of prevalence and progress across low- and middle-income countries (LMICs) are required for planning policy and interventions. Here we present a geospatial analysis of EBF prevalence estimates from 2000 to 2018 across 94 LMICs mapped to policy-relevant administrative units (for example, districts), quantify subnational inequalities and their changes over time, and estimate probabilities of meeting the World Health Organization’s Global Nutrition Target (WHO GNT) of ≥70% EBF prevalence by 2030. While six LMICs are projected to meet the WHO GNT of ≥70% EBF prevalence at a national scale, only three are predicted to meet the target in all their district-level units by 2030