Automatic Construction of Evaluation Suites for Natural Language Generation Datasets

Machine learning approaches applied to NLP are often evaluated by summarizing their performance in a single number, for example accuracy. Since most test sets are constructed as an i.i.d. sample from the overall data, this approach overly simplifies the complexity of language and encourages overfitting to the head of the data distribution. As such, rare language phenomena or text about underrepresented groups are not equally included in the evaluation. To encourage more in-depth model analyses, researchers have proposed the use of multiple test sets, also called challenge sets, that assess specific capabilities of a model. In this paper, we develop a framework based on this idea which is able to generate controlled perturbations and identify subsets in text-to-scalar, text-to-text, or data-to-text settings. By applying this framework to the GEM generation benchmark, we propose an evaluation suite made of 80 challenge sets, demonstrate the kinds of analyses that it enables and shed light onto the limits of current generation models

Loss of p53 triggers Wnt-dependent systemic inflammation to drive breast cancer metastasis

Cancer-associated systemic inflammation is strongly linked to poor disease outcome in patients with cancer1,2. For most human epithelial tumour types, high systemic neutrophil-to-lymphocyte ratios are associated with poor overall survival3, and experimental studies have demonstrated a causal relationship between neutrophils and metastasis4,5. However, the cancer-cell-intrinsic mechanisms that dictate the substantial heterogeneity in systemic neutrophilic inflammation between tumour-bearing hosts are largely unresolved. Here, using a panel of 16 distinct genetically engineered mouse models for breast cancer, we uncover a role for cancer-cell-intrinsic p53 as a key regulator of pro-metastatic neutrophils. Mechanistically, loss of p53 in cancer cells induced the secretion of WNT ligands that stimulate tumour-associated macrophages to produce IL-1β, thus driving systemic inflammation. Pharmacological and genetic blockade of WNT secretion in p53-null cancer cells reverses macrophage production of IL-1β and subsequent neutrophilic inflammation, resulting in reduced metastasis formation. Collectively, we demonstrate a mechanistic link between the loss of p53 in cancer cells, secretion of WNT ligands and systemic neutrophilia that potentiates metastatic progression. These insights illustrate the importance of the genetic makeup of breast tumours in dictating pro-metastatic systemic inflammation, and set the stage for personalized immune intervention strategies for patients with cancer

Loss of p53 triggers Wnt-dependent systemic inflammation to drive breast cancer metastasis

Cancer-associated systemic inflammation is strongly linked to poor disease outcome in patients with cancer1,2. For most human epithelial tumour types, high systemic neutrophil-to-lymphocyte ratios are associated with poor overall survival3, and experimental studies have demonstrated a causal relationship between neutrophils and metastasis4,5. However, the cancer-cell-intrinsic mechanisms that dictate the substantial heterogeneity in systemic neutrophilic inflammation between tumour-bearing hosts are largely unresolved. Here, using a panel of 16 distinct genetically engineered mouse models for breast cancer, we uncover a role for cancer-cell-intrinsic p53 as a key regulator of pro-metastatic neutrophils. Mechanistically, loss of p53 in cancer cells induced the secretion of WNT ligands that stimulate tumour-associated macrophages to produce IL-1β, thus driving systemic inflammation. Pharmacological and genetic blockade of WNT secretion in p53-null cancer cells reverses macrophage production of IL-1β and subsequent neutrophilic inflammation, resulting in reduced metastasis formation. Collectively, we demonstrate a mechanistic link between the loss of p53 in cancer cells, secretion of WNT ligands and systemic neutrophilia that potentiates metastatic progression. These insights illustrate the importance of the genetic makeup of breast tumours in dictating pro-metastatic systemic inflammation, and set the stage for personalized immune intervention strategies for patients with cancer

Birth preparedness and complication readiness (BPCR) among pregnant women in hard-to-reach areas in Bangladesh:BPCR in hard-to-reach areas of Bangladesh

Birth preparedness and complication readiness aims to reduce delays in care seeking, promote skilled birth attendance, and facility deliveries. Little is known about birth preparedness practices among populations living in hard-to-reach areas in Bangladesh.To describe levels of birth preparedness and complication readiness among recently delivered women, identify determinants of being better prepared for birth, and assess the impact of greater birth preparedness on maternal and neonatal health practices.A cross-sectional survey with 2,897 recently delivered women was undertaken in 2012 as part of an evaluation trial done in five hard-to-reach districts in rural Bangladesh. Mothers were considered well prepared for birth if they adopted two or more of the four birth preparedness components. Descriptive statistics and multivariable logistic regression were used for analysis.Less than a quarter (24.5%) of women were considered well prepared for birth. Predictors of being well-prepared included: husband's education (OR = 1.3; CI: 1.1-1.7), district of residence, exposure to media in the form of reading a newspaper (OR = 2.2; CI: 1.2-3.9), receiving home visit by a health worker during pregnancy (OR = 1.5; CI: 1.2-1.8), and receiving at least 3 antenatal care visits from a qualified provider (OR = 1.4; CI: 1.0-1.9). Well-prepared women were more likely to deliver at a health facility (OR = 2.4; CI: 1.9-3.1), use a skilled birth attendant (OR = 2.4, CI: 1.9-3.1), practice clean cord care (OR = 1.3, CI: 1.0-1.5), receive post-natal care from a trained provider within two days of birth for themselves (OR = 2.6, CI: 2.0-3.2) or their newborn (OR = 2.6, CI: 2.1-3.3), and seek care for delivery complications (OR = 1.8, CI: 1.3-2.6).Greater emphasis on BPCR interventions tailored for hard to reach areas is needed to improve skilled birth attendance, care seeking for complications and essential newborn care and facilitate reductions in maternal and neonatal mortality in low performing districts in Bangladesh

Insights into udder health and intramammary antibiotic usage on Irish dairy farms during 2003-2010

By international standards, Ireland is a relatively small dairy producer. However, the industry plays a critical role to the national economy, accounting for approximately 3% of national gross domestic product. This paper presents insights into udder health and intramammary antibiotic usage on Irish dairy farms during 2003-2010, based on data from several sources. Three data sources were used, including data on milk recording data, intramammary antibiotic sales and animal health assessment. The milk recording data included a single unadjusted herd-level somatic cell count (SCC) value for each herd at each milk recording, being the arithmetic mean of cow-level SCC of each cow at that recording, weighted by cow-level yield. These data were used to calculate the percentage of herds each month where the unadjusted herd SCC exceeded 200,000 and 400,000 cells/mL. Two logistic generalised estimating-equations (GEE) models were developed, the outcome variable being either the probability that the monthly SCC of a herd was greater than 400,000 cells/mL or less than or equal to 200,000 cells/mL. Spring herds had a lower probability of a high SCC (> 400,000 cells/mL) during February to October compared to non-Spring herds but a higher probability between November to January. The odds of a high SCC were greater in 2005, 2006, 2009 and 2010 but less in 2007 and 2008 compared to 2004. Smaller herds had higher odds of having a high SCC compared to larger herds. We present the number of intramammary tubes and the quantity of active substance (kg) sold annually in Ireland during 2003-2010. We infer an incidence of clinical mastitis of 54.0 cases per 100 cow-years at risk, assuming 4 tubes per treatment regime, one affected quarter per cow, tubes restricted to clinical cases only and 100% of treated cases considered new cases, based on data collected on sales of in-lactation intra-mammary antibiotics. With differing assumptions, this estimate varied between 25.8 and 77.0 cases per 100 cow-years at risk. Using data on sales of dry cow therapy intra-mammary antibiotics, we also infer that most Irish dairy farmers use blanket dry cow therapy. It is important that Ireland has an objective understanding of current levels of udder health, to facilitate benchmarking and improvement into the future. Udder health is a concern on a number of Irish dairy farms. High SCC results were present throughout the year, but more marked towards the start and end of each milking season. Animal Health Ireland recently commenced a major national programme, CellCheck, in collaboration with a broad range of stakeholders, to support national SCC improvement. In this paper, relevant European and national legislation is also reviewed