Natural Language to Code Translation with Execution

Generative models of code, pretrained on large corpora of programs, have shown great success in translating natural language to code (Chen et al., 2021; Austin et al., 2021; Li et al., 2022, inter alia). While these models do not explicitly incorporate program semantics (i.e., execution results) during training, they are able to generate correct solutions for many problems. However, choosing a single correct program from a generated set for each problem remains challenging. In this work, we introduce execution result--based minimum Bayes risk decoding (MBR-EXEC) for program selection and show that it improves the few-shot performance of pretrained code models on natural-language-to-code tasks. We select output programs from a generated candidate set by marginalizing over program implementations that share the same semantics. Because exact equivalence is intractable, we execute each program on a small number of test inputs to approximate semantic equivalence. Across datasets, execution or simulated execution significantly outperforms the methods that do not involve program semantics. We find that MBR-EXEC consistently improves over all execution-unaware selection methods, suggesting it as an effective approach for natural language to code translation. We open-source our code at github.com/facebookresearch/mbr-exec and data at dl.fbaipublicfiles.com/mbr-exec/mbr-exec-release.zipComment: EMNLP 202

Large Language Models Can Be Easily Distracted by Irrelevant Context

Large language models have achieved impressive performance on various natural language processing tasks. However, so far they have been evaluated primarily on benchmarks where all information in the input context is relevant for solving the task. In this work, we investigate the distractibility of large language models, i.e., how the model problem-solving accuracy can be influenced by irrelevant context. In particular, we introduce Grade-School Math with Irrelevant Context (GSM-IC), an arithmetic reasoning dataset with irrelevant information in the problem description. We use this benchmark to measure the distractibility of cutting-edge prompting techniques for large language models, and find that the model performance is dramatically decreased when irrelevant information is included. We also identify several approaches for mitigating this deficiency, such as decoding with self-consistency and adding to the prompt an instruction that tells the language model to ignore the irrelevant information

InCoder: A Generative Model for Code Infilling and Synthesis

Code is seldom written in a single left-to-right pass and is instead repeatedly edited and refined. We introduce InCoder, a unified generative model that can perform program synthesis (via left-to-right generation) as well as editing (via infilling). InCoder is trained to generate code files from a large corpus of permissively licensed code, where regions of code have been randomly masked and moved to the end of each file, allowing code infilling with bidirectional context. Our model is the first generative model that is able to directly perform zero-shot code infilling, which we evaluate on challenging tasks such as type inference, comment generation, and variable re-naming. We find that the ability to condition on bidirectional context substantially improves performance on these tasks, while still performing comparably on standard program synthesis benchmarks in comparison to left-to-right only models pretrained at similar scale. The InCoder models and code are publicly released. https://sites.google.com/view/incoder-code-modelsComment: 25 pages, 13 figures. v2: added NeoX-20B results & StackOverflow corpus inf

Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models

While large language models (LLMs) have demonstrated remarkable capabilities across a range of downstream tasks, a significant concern revolves around their propensity to exhibit hallucinations: LLMs occasionally generate content that diverges from the user input, contradicts previously generated context, or misaligns with established world knowledge. This phenomenon poses a substantial challenge to the reliability of LLMs in real-world scenarios. In this paper, we survey recent efforts on the detection, explanation, and mitigation of hallucination, with an emphasis on the unique challenges posed by LLMs. We present taxonomies of the LLM hallucination phenomena and evaluation benchmarks, analyze existing approaches aiming at mitigating LLM hallucination, and discuss potential directions for future research.Comment: work in progress; 32 page

A Rare Functional Noncoding Variant at the GWAS-Implicated MIR137/MIR2682 Locus Might Confer Risk to Schizophrenia and Bipolar Disorder

Schizophrenia (SZ) genome-wide association studies (GWASs) have identified common risk variants in >100 susceptibility loci; however, the contribution of rare variants at these loci remains largely unexplored. One of the strongly associated loci spans MIR137 (miR137) and MIR2682 (miR2682), two microRNA genes important for neuronal function. We sequenced ∼6.9 kb MIR137/MIR2682 and upstream regulatory sequences in 2,610 SZ cases and 2,611 controls of European ancestry. We identified 133 rare variants with minor allele frequency (MAF) <0.5%. The rare variant burden in promoters and enhancers, but not insulators, was associated with SZ (p = 0.021 for MAF < 0.5%, p = 0.003 for MAF < 0.1%). A rare enhancer SNP, 1:g.98515539A>T, presented exclusively in 11 SZ cases (nominal p = 4.8 × 10−4). We further identified its risk allele T in 2 of 2,434 additional SZ cases, 11 of 4,339 bipolar (BP) cases, and 3 of 3,572 SZ/BP study controls and 1,688 population controls; yielding combined p values of 0.0007, 0.0013, and 0.0001 for SZ, BP, and SZ/BP, respectively. The risk allele T of 1:g.98515539A>T reduced enhancer activity of its flanking sequence by >50% in human neuroblastoma cells, predicting lower expression of MIR137/MIR2682. Both empirical and computational analyses showed weaker transcription factor (YY1) binding by the risk allele. Chromatin conformation capture (3C) assay further indicated that 1:g.98515539A>T influenced MIR137/MIR2682, but not the nearby DPYD or LOC729987. Our results suggest that rare noncoding risk variants are associated with SZ and BP at MIR137/MIR2682 locus, with risk alleles decreasing MIR137/MIR2682 expression

Novel Insights into Pituitary Tumorigenesis: Genetic and Epigenetic Mechanisms.

Substantial advances have been made recently in the pathobiology of pituitary tumors. Similar to many other endocrine tumors, over the last few years we have recognized the role of germline and somatic mutations in a number of syndromic or nonsyndromic conditions with pituitary tumor predisposition. These include the identification of novel germline variants in patients with familial or simplex pituitary tumors and establishment of novel somatic variants identified through next generation sequencing. Advanced techniques have allowed the exploration of epigenetic mechanisms mediated through DNA methylation, histone modifications and noncoding RNAs, such as microRNA, long noncoding RNAs and circular RNAs. These mechanisms can influence tumor formation, growth, and invasion. While genetic and epigenetic mechanisms often disrupt similar pathways, such as cell cycle regulation, in pituitary tumors there is little overlap between genes altered by germline, somatic, and epigenetic mechanisms. The interplay between these complex mechanisms driving tumorigenesis are best studied in the emerging multiomics studies. Here, we summarize insights from the recent developments in the regulation of pituitary tumorigenesis

Height and body-mass index trajectories of school-aged children and adolescents from 1985 to 2019 in 200 countries and territories: a pooled analysis of 2181 population-based studies with 65 million participants

Summary Background Comparable global data on health and nutrition of school-aged children and adolescents are scarce. We aimed to estimate age trajectories and time trends in mean height and mean body-mass index (BMI), which measures weight gain beyond what is expected from height gain, for school-aged children and adolescents. Methods For this pooled analysis, we used a database of cardiometabolic risk factors collated by the Non-Communicable Disease Risk Factor Collaboration. We applied a Bayesian hierarchical model to estimate trends from 1985 to 2019 in mean height and mean BMI in 1-year age groups for ages 5–19 years. The model allowed for non-linear changes over time in mean height and mean BMI and for non-linear changes with age of children and adolescents, including periods of rapid growth during adolescence. Findings We pooled data from 2181 population-based studies, with measurements of height and weight in 65 million participants in 200 countries and territories. In 2019, we estimated a difference of 20 cm or higher in mean height of 19-year-old adolescents between countries with the tallest populations (the Netherlands, Montenegro, Estonia, and Bosnia and Herzegovina for boys; and the Netherlands, Montenegro, Denmark, and Iceland for girls) and those with the shortest populations (Timor-Leste, Laos, Solomon Islands, and Papua New Guinea for boys; and Guatemala, Bangladesh, Nepal, and Timor-Leste for girls). In the same year, the difference between the highest mean BMI (in Pacific island countries, Kuwait, Bahrain, The Bahamas, Chile, the USA, and New Zealand for both boys and girls and in South Africa for girls) and lowest mean BMI (in India, Bangladesh, Timor-Leste, Ethiopia, and Chad for boys and girls; and in Japan and Romania for girls) was approximately 9–10 kg/m2. In some countries, children aged 5 years started with healthier height or BMI than the global median and, in some cases, as healthy as the best performing countries, but they became progressively less healthy compared with their comparators as they grew older by not growing as tall (eg, boys in Austria and Barbados, and girls in Belgium and Puerto Rico) or gaining too much weight for their height (eg, girls and boys in Kuwait, Bahrain, Fiji, Jamaica, and Mexico; and girls in South Africa and New Zealand). In other countries, growing children overtook the height of their comparators (eg, Latvia, Czech Republic, Morocco, and Iran) or curbed their weight gain (eg, Italy, France, and Croatia) in late childhood and adolescence. When changes in both height and BMI were considered, girls in South Korea, Vietnam, Saudi Arabia, Turkey, and some central Asian countries (eg, Armenia and Azerbaijan), and boys in central and western Europe (eg, Portugal, Denmark, Poland, and Montenegro) had the healthiest changes in anthropometric status over the past 3·5 decades because, compared with children and adolescents in other countries, they had a much larger gain in height than they did in BMI. The unhealthiest changes—gaining too little height, too much weight for their height compared with children in other countries, or both—occurred in many countries in sub-Saharan Africa, New Zealand, and the USA for boys and girls; in Malaysia and some Pacific island nations for boys; and in Mexico for girls. Interpretation The height and BMI trajectories over age and time of school-aged children and adolescents are highly variable across countries, which indicates heterogeneous nutritional quality and lifelong health advantages and risks

Worldwide trends in hypertension prevalence and progress in treatment and control from 1990 to 2019: a pooled analysis of 1201 population-representative studies with 104 million participants.

BACKGROUND: Hypertension can be detected at the primary health-care level and low-cost treatments can effectively control hypertension. We aimed to measure the prevalence of hypertension and progress in its detection, treatment, and control from 1990 to 2019 for 200 countries and territories. METHODS: We used data from 1990 to 2019 on people aged 30-79 years from population-representative studies with measurement of blood pressure and data on blood pressure treatment. We defined hypertension as having systolic blood pressure 140 mm Hg or greater, diastolic blood pressure 90 mm Hg or greater, or taking medication for hypertension. We applied a Bayesian hierarchical model to estimate the prevalence of hypertension and the proportion of people with hypertension who had a previous diagnosis (detection), who were taking medication for hypertension (treatment), and whose hypertension was controlled to below 140/90 mm Hg (control). The model allowed for trends over time to be non-linear and to vary by age. FINDINGS: The number of people aged 30-79 years with hypertension doubled from 1990 to 2019, from 331 (95% credible interval 306-359) million women and 317 (292-344) million men in 1990 to 626 (584-668) million women and 652 (604-698) million men in 2019, despite stable global age-standardised prevalence. In 2019, age-standardised hypertension prevalence was lowest in Canada and Peru for both men and women; in Taiwan, South Korea, Japan, and some countries in western Europe including Switzerland, Spain, and the UK for women; and in several low-income and middle-income countries such as Eritrea, Bangladesh, Ethiopia, and Solomon Islands for men. Hypertension prevalence surpassed 50% for women in two countries and men in nine countries, in central and eastern Europe, central Asia, Oceania, and Latin America. Globally, 59% (55-62) of women and 49% (46-52) of men with hypertension reported a previous diagnosis of hypertension in 2019, and 47% (43-51) of women and 38% (35-41) of men were treated. Control rates among people with hypertension in 2019 were 23% (20-27) for women and 18% (16-21) for men. In 2019, treatment and control rates were highest in South Korea, Canada, and Iceland (treatment >70%; control >50%), followed by the USA, Costa Rica, Germany, Portugal, and Taiwan. Treatment rates were less than 25% for women and less than 20% for men in Nepal, Indonesia, and some countries in sub-Saharan Africa and Oceania. Control rates were below 10% for women and men in these countries and for men in some countries in north Africa, central and south Asia, and eastern Europe. Treatment and control rates have improved in most countries since 1990, but we found little change in most countries in sub-Saharan Africa and Oceania. Improvements were largest in high-income countries, central Europe, and some upper-middle-income and recently high-income countries including Costa Rica, Taiwan, Kazakhstan, South Africa, Brazil, Chile, Turkey, and Iran. INTERPRETATION: Improvements in the detection, treatment, and control of hypertension have varied substantially across countries, with some middle-income countries now outperforming most high-income nations. The dual approach of reducing hypertension prevalence through primary prevention and enhancing its treatment and control is achievable not only in high-income countries but also in low-income and middle-income settings. FUNDING: WHO

Worldwide trends in hypertension prevalence and progress in treatment and control from 1990 to 2019: a pooled analysis of 1201 population-representative studies with 104 million participants

Background Hypertension can be detected at the primary health-care level and low-cost treatments can effectively control hypertension. We aimed to measure the prevalence of hypertension and progress in its detection, treatment, and control from 1990 to 2019 for 200 countries and territories. Methods We used data from 1990 to 2019 on people aged 30-79 years from population-representative studies with measurement of blood pressure and data on blood pressure treatment. We defined hypertension as having systolic blood pressure 140 mm Hg or greater, diastolic blood pressure 90 mm Hg or greater, or taking medication for hypertension. We applied a Bayesian hierarchical model to estimate the prevalence of hypertension and the proportion of people with hypertension who had a previous diagnosis (detection), who were taking medication for hypertension (treatment), and whose hypertension was controlled to below 140/90 mm Hg (control). The model allowed for trends over time to be non-linear and to vary by age. Findings The number of people aged 30-79 years with hypertension doubled from 1990 to 2019, from 331 (95% credible interval 306-359) million women and 317 (292-344) million men in 1990 to 626 (584-668) million women and 652 (604-698) million men in 2019, despite stable global age-standardised prevalence. In 2019, age-standardised hypertension prevalence was lowest in Canada and Peru for both men and women; in Taiwan, South Korea, Japan, and some countries in western Europe including Switzerland, Spain, and the UK for women; and in several low-income and middle-income countries such as Eritrea, Bangladesh, Ethiopia, and Solomon Islands for men. Hypertension prevalence surpassed 50% for women in two countries and men in nine countries, in central and eastern Europe, central Asia, Oceania, and Latin America. Globally, 59% (55-62) of women and 49% (46-52) of men with hypertension reported a previous diagnosis of hypertension in 2019, and 47% (43-51) of women and 38% (35-41) of men were treated. Control rates among people with hypertension in 2019 were 23% (20-27) for women and 18% (16-21) for men. In 2019, treatment and control rates were highest in South Korea, Canada, and Iceland (treatment >70%; control >50%), followed by the USA, Costa Rica, Germany, Portugal, and Taiwan. Treatment rates were less than 25% for women and less than 20% for men in Nepal, Indonesia, and some countries in sub-Saharan Africa and Oceania. Control rates were below 10% for women and men in these countries and for men in some countries in north Africa, central and south Asia, and eastern Europe. Treatment and control rates have improved in most countries since 1990, but we found little change in most countries in sub-Saharan Africa and Oceania. Improvements were largest in high-income countries, central Europe, and some upper-middle-income and recently high-income countries including Costa Rica, Taiwan, Kazakhstan, South Africa, Brazil, Chile, Turkey, and Iran. Interpretation Improvements in the detection, treatment, and control of hypertension have varied substantially across countries, with some middle-income countries now outperforming most high-income nations. The dual approach of reducing hypertension prevalence through primary prevention and enhancing its treatment and control is achievable not only in high-income countries but also in low-income and middle-income settings. Copyright (C) 2021 World Health Organization; licensee Elsevier