BUMP: A Benchmark of Unfaithful Minimal Pairs for Meta-Evaluation of Faithfulness Metrics

The proliferation of automatic faithfulness metrics for summarization has produced a need for benchmarks to evaluate them. While existing benchmarks measure the correlation with human judgements of faithfulness on model-generated summaries, they are insufficient for diagnosing whether metrics are: 1) consistent, i.e., indicate lower faithfulness as errors are introduced into a summary, 2) effective on human-written texts, and 3) sensitive to different error types (as summaries can contain multiple errors). To address these needs, we present a benchmark of unfaithful minimal pairs (BUMP), a dataset of 889 human-written, minimally different summary pairs, where a single error is introduced to a summary from the CNN/DailyMail dataset to produce an unfaithful summary. We find BUMP complements existing benchmarks in a number of ways: 1) the summaries in BUMP are harder to discriminate and less probable under SOTA summarization models, 2) unlike non-pair-based datasets, BUMP can be used to measure the consistency of metrics, and reveals that the most discriminative metrics tend not to be the most consistent, and 3) unlike datasets containing generated summaries with multiple errors, BUMP enables the measurement of metrics' performance on individual error types.Comment: Accepted as a long main conference paper at ACL 202

Insecticide susceptibility status of Phlebotomus (Paraphlebotomus) sergenti and Phlebotomus (Phlebotomus) papatasi in endemic foci of cutaneous leishmaniasis in Morocco

<p>Abstract</p> <p>Background</p> <p>In Morocco, cutaneous leishmaniasis is transmitted by <it>Phlebotomus sergenti </it>and <it>Ph. papatasi</it>. Vector control is mainly based on environmental management but indoor residual spraying with synthetic pyrethroids is applied in many foci of <it>Leishmania tropica</it>. However, the levels and distribution of sandfly susceptibility to insecticides currently used has not been studied yet. Hence, this study was undertaken to establish the susceptibility status of <it>Ph. sergenti </it>and <it>Ph. papatasi </it>to lambdacyhalothrin, DDT and malathion.</p> <p>Methods</p> <p>The insecticide susceptibility status of <it>Ph. sergenti </it>and <it>Ph. papatasi </it>was assessed during 2011, following the standard WHO technique based on discriminating dosage. A series of twenty-five susceptibility tests were carried out on wild populations of <it>Ph. sergenti </it>and <it>Ph. papatasi </it>collected by CDC light traps from seven villages in six different provinces. Knockdown rates (KDT) were noted at 5 min intervals during the exposure to DDT and to lambdacyhalothrin. After one hour of exposure, sandflies were transferred to the observation tubes for 24 hours. After this period, mortality rate was calculated. Data were analyzed by Probit analysis program to determine the knockdown time 50% and 90% (KDT50 and KDT90) values.</p> <p>Results</p> <p>Study results showed that <it>Ph.sergenti </it>and <it>Ph. papatasi </it>were susceptible to all insecticides tested. Comparison of KDT values showed a clear difference between the insecticide knockdown effect in studied villages. This effect was lower in areas subject to high selective public health insecticide pressure in the framework of malaria or leishmaniasis control.</p> <p>Conclusion</p> <p><it>Phlebotomus sergenti </it>and <it>Ph. papatasi </it>are susceptible to the insecticides tested in the seven studied villages but they showed a low knockdown effect in Azilal, Chichaoua and Settat. Therefore, a study of insecticide susceptibility of these vectors in other foci of leishmaniasis is recommended and the level of their susceptibility should be regularly monitored.</p

The PTEN Phosphatase Controls Intestinal Epithelial Cell Polarity and Barrier Function: Role in Colorectal Cancer Progression

The PTEN phosphatase acts on phosphatidylinositol 3,4,5-triphosphates resulting from phosphatidylinositol 3-kinase (PI3K) activation. PTEN expression has been shown to be decreased in colorectal cancer. Little is known however as to the specific cellular role of PTEN in human intestinal epithelial cells. The aim of this study was to investigate the role of PTEN in human colorectal cancer cells.Caco-2/15, HCT116 and CT26 cells were infected with recombinant lentiviruses expressing a shRNA specifically designed to knock-down PTEN. The impact of PTEN downregulation was analyzed on cell polarization and differentiation, intercellular junction integrity (expression of cell-cell adhesion proteins, barrier function), migration (wound assay), invasion (matrigel-coated transwells) and on tumor and metastasis formation in mice. Electron microscopy analysis showed that lentiviral infection of PTEN shRNA significantly inhibited Caco-2/15 cell polarization, functional differentiation and brush border development. A strong reduction in claudin 1, 3, 4 and 8 was also observed as well as a decrease in transepithelial resistance. Loss of PTEN expression increased the spreading, migration and invasion capacities of colorectal cancer cells in vitro. PTEN downregulation also increased tumor size following subcutaneous injection of colorectal cancer cells in nude mice. Finally, loss of PTEN expression in HCT116 and CT26, but not in Caco-2/15, led to an increase in their metastatic potential following tail-vein injections in mice.Altogether, these results indicate that PTEN controls cellular polarity, establishment of cell-cell junctions, paracellular permeability, migration and tumorigenic/metastatic potential of human colorectal cancer cells