BLiMP: A Benchmark of Linguistic Minimal Pairs for English

We introduce BLiMP (The Benchmark of Linguistic Minimal Pairs), a human-solvable challenge set for evaluating language models (LMs) that covers a broad range of major grammatical phenomena in English. BLiMP consists of over 30 datasets, each containing 1000 minimal pairs isolating specific contrasts in syntax, morphology, or semantics. Like GLUE (Wang et al., 2018), BLiMP makes it easy to directly compare models. Evaluating n-gram, LSTM, and Transformer LMs (GPT-2 and TransformerXL), we find that transformers are strongest overall, achieving (near) human performance on agreement and binding. However, phenomena like wh-islands and NPI licensing remain challenging even for state-of-the-art LMs

BLiMP: The Benchmark of Linguistic Minimal Pairs for English

We introduce The Benchmark of Linguistic Minimal Pairs (shortened to BLiMP), a challenge set for evaluating what language models (LMs) know about major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each containing 1000 minimal pairs isolating specific contrasts in syntax, morphology, or semantics. The data is automatically generated according to expert-crafted grammars, and aggregate human agreement with the labels is 96.4%. We use it to evaluate n-gram, LSTM, and Transformer (GPT-2 and Transformer-XL) LMs. We find that state-of-the-art models identify morphological contrasts reliably, but they struggle with semantic restrictions on the distribution of quantifiers and negative polarity items and subtle syntactic phenomena such as extraction islands.Comment: To appear in TAC

Case Report Esophageal Rupture as a Primary Manifestation in Eosinophilic Esophagitis

Eosinophilic esophagitis (EoE) is a chronic inflammatory process characterized by symptoms of esophageal dysfunction and, histologically, by eosinophilic infiltration of the esophagus. In adults, it commonly presents with dysphagia, food impaction, and chest or abdominal pain. Chronic inflammation can lead to diffuse narrowing of the esophageal lumen which may cause food impaction. Endoscopic procedures to relieve food impaction may lead to complications such as esophageal perforation due to the friability of the esophageal mucosa. Spontaneous transmural esophageal rupture, also known as Boerhaave's syndrome, as a primary manifestation of EoE is rare. In this paper, we present two adult patients who presented with esophageal perforation as the initial manifestation of EoE. This rare complication of EoE has been documented in 13 other reports (11 adults, 2 children) and only 1 of the patients had been previously diagnosed with EoE. A history of dysphagia was present in 1 of our patients and in the majority of previously documented patients. Esophageal perforation is a potentially severe complication of EoE. Patients with a history of dysphagia and patients with spontaneous esophageal perforation should warrant an evaluation for EoE

Abstract 13825: Characteristics of Patients With Exercise-Induced Left Ventricular Systolic Dysfunction and Non-Significant Coronary Artery Disease

Introduction: Prior studies have shown 30-35% of patients with an abnormal stress test had non-obstructive coronary artery disease. The characteristics and outcomes of this patient population have not been well studied. We tested the hypothesis that there is a distinct group of individuals who have an exercise-induced cardiomyopathy not related to the presence of obstructive coronary artery disease or pre-existing structural heart disease. Methods: Our cohort included adults 18 years and older who underwent stress echocardiography at the Mayo Clinic. Patients were included if their resting left ventricular ejection fraction (LVEF) was greater than 50% and decreased by more than 5% to under 50% with exercise. We excluded patients who did not have coronary angiography in the 90 days following stress echocardiography and those with prior diagnosis of coronary artery disease or structural/valvular heart disease. Results: A total of 205627 stress echocardiograms were performed between 2003 and 2019, of which 127 patients were analyzed in our cohort. The median age was 68 years, and 76% were female. The median LVEF was 55% at rest and 45% with stress. Left bundle branch block was present in 9% of patients. Median peak systolic blood pressure was 174 mm Hg. Stress electrocardiogram was positive in 14% of patients. Median functional aerobic capacity [interquartile range] as a percentage of age/sex predicted capacity was 101% [73%, 117%]. Conclusion: We describe a cohort of patients with exercise-induced left ventricular systolic dysfunction not explained by obstructive coronary artery disease or structural heart disease. Forthcoming analyses of long term outcomes will be important to understand how these patients compare to patients without exercise-induced cardiac dysfunction. </jats:p