Evaluating NLG Evaluation Metrics: A Measurement Theory Perspective

We address the fundamental challenge in Natural Language Generation (NLG) model evaluation, the design and validation of evaluation metrics. Recognizing the limitations of existing metrics and issues with human judgment, we propose using measurement theory, the foundation of test design, as a framework for conceptualizing and evaluating the validity and reliability of NLG evaluation metrics. This approach offers a systematic method for defining "good" metrics, developing robust metrics, and assessing metric performance. In this paper, we introduce core concepts in measurement theory in the context of NLG evaluation and key methods to evaluate the performance of NLG metrics. Through this framework, we aim to promote the design, evaluation, and interpretation of valid and reliable metrics, ultimately contributing to the advancement of robust and effective NLG models in real-world settings

Co-targeting HGF/cMET Signaling with MEK Inhibitors in Metastatic Uveal Melanoma.

Patients with metastatic uveal melanoma usually die within 1 year of diagnosis, emphasizing an urgent need to develop new treatment strategies. The liver is the most common site of metastasis. Mitogen-activated protein kinase kinase (MEK) inhibitors improve survival in V600 BRAF-mutated cutaneous melanoma patients but have limited efficacy in patients with uveal melanoma. Our previous work showed that hepatocyte growth factor (HGF) signaling elicits resistance to MEK inhibitors in metastatic uveal melanoma. In this study, we demonstrate that expression of two BH3-only family proteins, Bim-EL and Bmf, contributes to HGF-mediated resistance to MEK inhibitors. Targeting HGF/cMET signaling with LY2875358, a neutralizing and internalizing anti-cMET bivalent antibody, and LY2801653, a dual cMET/RON inhibitor, overcomes resistance to trametinib provided by exogenous HGF and by conditioned medium from primary hepatic stellate cells. We further determined that activation of PI3Kα/γ/δ isoforms mediates the resistance to MEK inhibitors by HGF. Combination of LY2801653 with trametinib decreases AKT phosphorylation and promotes proapoptotic PARP cleavage in metastatic uveal melanoma explants. Together, our data support the notion that selectively blocking cMET signaling or PI3K isoforms in metastatic uveal melanoma may break the intrinsic resistance to MEK inhibitors provided by factors from stromal cells in the liver

A new Rogue-like Escherichia phage UDF157lw to control Escherichia coli O157:H7

IntroductionShiga toxin-producing Escherichia coli (STEC) O157:H7 is one of the notorious foodborne pathogens causing high mortality through the consumption of contaminated food items. The food safety risk from STEC pathogens could escalate when a group of bacterial cells aggregates to form a biofilm. Bacterial biofilm can diminish the effects of various antimicrobial interventions and enhance the pathogenicity of the pathogens. Therefore, there is an urgent need to have effective control measurements. Bacteriophages can kill the target bacterial cells through lytic infection, and some enzymes produced during the infection have the capability to penetrate the biofilm for mitigation compared to traditional interventions. This study aimed to characterize a new Escherichia phage vB_EcoS-UDF157lw (or UDF157lw) and determine its antimicrobial efficacy against E. coli O157:H7.MethodsPhage characterization included biological approaches, including phage morphology, one-step growth curve, stability tests (pH and temperature), and genomic approaches (whole-genome sequencing). Later, antimicrobial activity tests, including productive infection against susceptible bacterial strains, in vitro antimicrobial activity, and anti-biofilm, were conducted.ResultsUDF157lw is a new member of the phages belonging to the Rogunavirus genus, comprising a long and non-contractile tail, isolated from bovine feces and shares close genomic evolutionary similarities with Escherichia phages vB_EcoS-BECP10 and bV_EcoS_AKS96. When used against E. coli O157:H7 (ATCC35150), phage UDF157lw exhibited a latent period of 14 min and a burst size of 110 PFU per infected cell. The phage remained viable in a wide range of pH values (pH 4–11) and temperatures (4–60°C). No virulence genes, such as stx, lysogenic genes, and antibiotic resistance genes, were found. Phage UDF157lw demonstrated high infection efficiencies against different E. coli O157:H7 and generic E. coli strains. In addition, UDF157lw encoded a unique major tail protein (ORF_26) with prominent depolymerase enzyme activity against various E. coli O157:H7 strains, causing large plaque sizes. In contrast to the phage without encoding depolymerase gene, UDF157lw was able to reduce the 24-h and 48-h E. coli O157:H7 biofilm after 1-h phage treatment.DiscussionThe findings of this study provide insights into a new member of the Rogunavirus phages and demonstrate its antimicrobial potential against E. coli O157:H7 in vitro

Concatenated γ-aminobutyric acid type A receptors revisited: Finding order in chaos

Publisher's version (útgefin grein)γ-aminobutyric acid type A receptors (GABAARs), the major inhibitory neurotransmitter receptors in the mammalian central nervous system, are arguably the most challenging member of the pentameric Cys-loop receptors to study due to their heteromeric structure. When two or more subunits are expressed together in heterologous systems, receptors of variable subunit type, ratio, and orientation can form, precluding accurate interpretation of data from functional studies. Subunit concatenation is a technique that involves the linking of individual subunits and in theory allows the precise control of the uniformity of expressed receptors. In reality, the resulting concatemers from widely used constructs are flexible in their orientation and may therefore assemble with themselves or free GABAAR subunits in unexpected ways. In this study, we examine functional responses of receptors from existing concatenated constructs and describe refinements necessary to allow expression of uniform receptor populations. We find that dimers from two commonly used concatenated constructs, β-23-α and α-10-β, assemble readily in both the clockwise and the counterclockwise orientations when coexpressed with free subunits. Furthermore, we show that concatemers formed from new tetrameric α-10-β-α-β and α-10-β-α-γ constructs also assemble in both orientations with free subunits to give canonical αβγ receptors. To restrict linker flexibility, we systematically shorten linker lengths of dimeric and pentameric constructs and find optimized constructs that direct the assembly of GABAARs only in one orientation, thus eliminating the ambiguity associated with previously described concatemers. Based on our data, we revisit some noncanonical GABAAR configurations proposed in recent years and explain how the use of some concatenated constructs may have led to wrong conclusions. Our results help clarify current contradictions in the literature regarding GABAAR subunit stoichiometry and arrangement. The lessons learned from this study may guide future efforts in understanding other related heteromeric receptors.This work was supported by the Australian Research Council (LP160100560) and the Australian National Health and Medical Research Council (APP1124567 and APP1081733 to M. Chebib, P.K. Ahring, and T. Balle). N.M. Kowal was supported by the Icelandic Research Fund (grant number 152604). The authors declare no competing financial interests. Author contributions: P.K. Ahring designed the research; V.W.Y. Liao, P.K. Ahring, H.C. Chua, N.M. Kowal, and T. Balle performed the research; P.K. Ahring analyzed the data and wrote the manuscript. P.K. Ahring, T. Balle, and M. Chebib acquired the necessary funding. All authors approved the final version of the manuscript. Richard W. Aldrich served as editor.Peer Reviewe

Thromboprophylaxis Is Associated With Reduced Post-hospitalization Venous Thromboembolic Events in Patients With Inflammatory Bowel Diseases

Background & Aims Patients with inflammatory bowel diseases (IBDs) have increased risk for venous thromboembolism (VTE); those who require hospitalization have particularly high risk. Few hospitalized patients with IBD receive thromboprophylaxis. We analyzed the frequency of VTE after IBD-related hospitalization, risk factors for post-hospitalization VTE, and the efficacy of prophylaxis in preventing post-hospitalization VTE. Methods In a retrospective study, we analyzed data from a multi-institutional cohort of patients with Crohn's disease or ulcerative colitis and at least 1 IBD-related hospitalization. Our primary outcome was a VTE event. All patients contributed person-time from the date of the index hospitalization to development of VTE, subsequent hospitalization, or end of follow-up. Our main predictor variable was pharmacologic thromboprophylaxis. Cox proportional hazard models adjusting for potential confounders were used to estimate hazard ratios (HRs) and 95% confidence intervals (CIs). Results From a cohort of 2788 patients with at least 1 IBD-related hospitalization, 62 patients developed VTE after discharge (2%). Incidences of VTE at 30, 60, 90, and 180 days after the index hospitalization were 3.7/1000, 4.1/1000, 5.4/1000, and 9.4/1000 person-days, respectively. Pharmacologic thromboprophylaxis during the index hospital stay was associated with a significantly lower risk of post-hospitalization VTE (HR, 0.46; 95% CI, 0.22–0.97). Increased numbers of comorbidities (HR, 1.30; 95% CI, 1.16–1.47) and need for corticosteroids before hospitalization (HR, 1.71; 95% CI, 1.02–2.87) were also independently associated with risk of VTE. Length of hospitalization or surgery during index hospitalization was not associated with post-hospitalization VTE. Conclusions Pharmacologic thromboprophylaxis during IBD-related hospitalization is associated with reduced risk of post-hospitalization VTE.National Institutes of Health (U.S.) (U54-LM008748

Improving Case Definition of Crohnʼs Disease and Ulcerative Colitis in Electronic Medical Records Using Natural Language Processing

available in PMC 2014 June 01Background: Previous studies identifying patients with inflammatory bowel disease using administrative codes have yielded inconsistent results. Our objective was to develop a robust electronic medical record–based model for classification of inflammatory bowel disease leveraging the combination of codified data and information from clinical text notes using natural language processing. Methods: Using the electronic medical records of 2 large academic centers, we created data marts for Crohn’s disease (CD) and ulcerative colitis (UC) comprising patients with ≥1 International Classification of Diseases, 9th edition, code for each disease. We used codified (i.e., International Classification of Diseases, 9th edition codes, electronic prescriptions) and narrative data from clinical notes to develop our classification model. Model development and validation was performed in a training set of 600 randomly selected patients for each disease with medical record review as the gold standard. Logistic regression with the adaptive LASSO penalty was used to select informative variables. Results: We confirmed 399 CD cases (67%) in the CD training set and 378 UC cases (63%) in the UC training set. For both, a combined model including narrative and codified data had better accuracy (area under the curve for CD 0.95; UC 0.94) than models using only disease International Classification of Diseases, 9th edition codes (area under the curve 0.89 for CD; 0.86 for UC). Addition of natural language processing narrative terms to our final model resulted in classification of 6% to 12% more subjects with the same accuracy. Conclusions: Inclusion of narrative concepts identified using natural language processing improves the accuracy of electronic medical records case definition for CD and UC while simultaneously identifying more subjects compared with models using codified data alone.National Institutes of Health (U.S.) (NIH U54-LM008748)American Gastroenterological AssociationNational Institutes of Health (U.S.) (NIH K08 AR060257)Beth Isreal Deaconess Medical Center (Katherine Swan Ginsburg Fund)National Institutes of Health (U.S.) (NIH R01-AR056768)Burroughs Wellcome Fund (Career Award for Medical Scientists)National Institutes of Health (U.S.) (NIH U01-GM092691)National Institutes of Health (U.S.) (NIH R01-AR059648

The synthesis and evaluation of fluoro-, trifluoromethyl, and iodomuscimols as GABA agonists

MAFAM acknowledges the Ministry of Education of Malaysia and Universiti Teknologi MARA for a scholarship. MC acknowledges the funding support of the Australian National Health and Medical Research Council (APP1081733).Halogenated analogues of the neurotoxic alkaloid muscimol were prepared with fluorine, iodine or trifluoromethyl at the 4 position of the isoxazole ring system. These compounds were investigated as agonists for GABAA receptors. Only the C-4 fluorine containing analogue proved to be an active compound in these assays. The fluoro analogue was less active than muscimol, however it showed differential activity between synaptic (α1β2γ2) and extrasynaptic (α4β2γ) GABAA receptors, having a similar potency to the neurotransmitter GABA for the extrasynaptic (α4β2γ) receptor.PostprintPostprintPeer reviewe

Stromal cell-derived factor and granulocyte-monocyte colony-stimulating factor form a combined neovasculogenic therapy for ischemic cardiomyopathy

ObjectiveIschemic heart failure is an increasingly prevalent global health concern with major morbidity and mortality. Currently, therapies are limited, and novel revascularization methods might have a role. This study examined enhancing endogenous myocardial revascularization by expanding bone marrow-derived endothelial progenitor cells with the marrow stimulant granulocyte-monocyte colony-stimulating factor and recruiting the endothelial progenitor cells with intramyocardial administration of the potent endothelial progenitor cell chemokine stromal cell-derived factor.MethodsIschemic cardiomyopathy was induced in Lewis rats (n = 40) through left anterior descending coronary artery ligation. After 3 weeks, animals were randomized into 4 groups: saline control, granulocyte-monocyte colony-stimulating factor only (GM-CSF only), stromal cell-derived factor only (SDF only), and combined stromal cell-derived factor/granulocyte-monocyte colony-stimulating factor (SDF/GM-CSF) (n = 10 each). After another 3 weeks, hearts were analyzed for endothelial progenitor cell density by endothelial progenitor cell marker colocalization immunohistochemistry, vasculogenesis by von Willebrand immunohistochemistry, ventricular geometry by hematoxylin-and-eosin microscopy, and in vivo myocardial function with an intracavitary pressure-volume conductance microcatheter.ResultsThe saline control, GM-CSF only, and SDF only groups were equivalent. Compared with the saline control group, animals in the SDF/GM-CSF group exhibited increased endothelial progenitor cell density (21.7 ± 3.2 vs 9.6 ± 3.1 CD34+/vascular endothelial growth factor receptor 2–positive cells per high-power field, P = .01). There was enhanced vascularity (44.1 ± 5.5 versus 23.8 ± 2.2 von Willebrand factor-positive vessels per high-power field, P = .007). SDF/GM-CSF group animals experienced less adverse ventricular remodeling, as manifested by less cavitary dilatation (9.8 ± 0.1 mm vs 10.1 ± 0.1 mm [control], P = .04) and increased border-zone wall thickness (1.78 ± 0.19 vs 1.41 ± 0.16 mm [control], P = .03). (SDF/GM-CSF group animals had improved cardiac function compared with animals in the saline control group (maximum pressure: 93.9 ± 3.2 vs 71.7 ± 3.1 mm Hg, P < .001; maximum dP/dt: 3513 ± 303 vs 2602 ± 201 mm Hg/s, P < .05; cardiac output: 21.3 ± 2.7 vs 13.3 ± 1.3 mL/min, P < .01; end-systolic pressure-volume relationship slope: 1.7 ± 0.4 vs 0.5 ± 0.2 mm Hg/μL, P < .01.)ConclusionThis novel revascularization strategy of bone marrow stimulation and intramyocardial delivery of the endothelial progenitor cell chemokine stromal cell-derived factor yielded significantly enhanced myocardial endothelial progenitor cell density, vasculogenesis, geometric preservation, and contractility in a model of ischemic cardiomyopathy

Normalization of Plasma 25-Hydroxy Vitamin D Is Associated with Reduced Risk of Surgery in Crohn’s Disease

available in PMC 2014 August 01AB Background: Vitamin D may have an immunologic role in Crohn's disease (CD) and ulcerative colitis (UC). Retrospective studies suggested a weak association between vitamin D status and disease activity but have significant limitations. Methods: Using a multi-institution inflammatory bowel disease cohort, we identified all patients with CD and UC who had at least one measured plasma 25-hydroxy vitamin D (25(OH)D). Plasma 25(OH)D was considered sufficient at levels >=30 ng/mL. Logistic regression models adjusting for potential confounders were used to identify impact of measured plasma 25(OH)D on subsequent risk of inflammatory bowel disease-related surgery or hospitalization. In a subset of patients where multiple measures of 25(OH)D were available, we examined impact of normalization of vitamin D status on study outcomes. Results: Our study included 3217 patients (55% CD; mean age, 49 yr). The median lowest plasma 25(OH)D was 26 ng/mL (interquartile range, 17-35 ng/mL). In CD, on multivariable analysis, plasma 25(OH)D =30 ng/mL. Similar estimates were also seen for UC. Furthermore, patients with CD who had initial levels <30 ng/mL but subsequently normalized their 25(OH)D had a reduced likelihood of surgery (odds ratio, 0.56; 95% confidence interval, 0.32-0.98) compared with those who remained deficient. Conclusion: Low plasma 25(OH)D is associated with increased risk of surgery and hospitalizations in both CD and UC, and normalization of 25(OH)D status is associated with a reduction in the risk of CD-related surgery. (C) Crohn's & Colitis Foundation of America, Inc