Reranking Overgenerated Responses for End-to-End Task-Oriented Dialogue Systems

End-to-end (E2E) task-oriented dialogue (ToD) systems are prone to fall into the so-called 'likelihood trap', resulting in generated responses which are dull, repetitive, and often inconsistent with dialogue history. Comparing ranked lists of multiple generated responses against the 'gold response' (from training data) reveals a wide diversity in response quality, with many good responses placed lower in the ranked list. The main challenge, addressed in this work, is then how to reach beyond greedily generated system responses, that is, how to obtain and select such high-quality responses from the list of overgenerated responses at inference without availability of the gold response. To this end, we propose a simple yet effective reranking method which aims to select high-quality items from the lists of responses initially overgenerated by the system. The idea is to use any sequence-level (similarity) scoring function to divide the semantic space of responses into high-scoring versus low-scoring partitions. At training, the high-scoring partition comprises all generated responses whose similarity to the gold response is higher than the similarity of the greedy response to the gold response. At inference, the aim is to estimate the probability that each overgenerated response belongs to the high-scoring partition, given only previous dialogue history. We validate the robustness and versatility of our proposed method on the standard MultiWOZ dataset: our methods improve a state-of-the-art E2E ToD system by 2.4 BLEU, 3.2 ROUGE, and 2.8 METEOR scores, achieving new peak results. Additional experiments on the BiTOD dataset and human evaluation further ascertain the generalisability and effectiveness of the proposed framework.Comment: 22 pages, 10 figure

Specific Targeting MRI of Chitosan Oligosaccharide Modified Fe3O4 Nanoprobe on Macrophage and the Inhibition of Macrophage Foaming Induced by ox-LDL

Atherosclerosis (AS) is a primary cause of morbidity and mortality all over the world. Molecular imaging techniques can enable early localization and diagnosis of atherosclerosis plaques. Recent newly developed chitooligosaccharides (CSO) is considered to be capable of target mannose receptors on the surface of macrophages and to inhibit foam cell formation. Here we present a targeting magnetic resonance imaging (MRI) nanoprobe, which was successfully constructed with polyacrylic acid (PAA) modified nanometer iron oxide (Fe3O4) as the core, and coating with CSO molecules, possessing the abilities of targeted MRI and specifically inhibition of the formation of foamy macrophages in the atherosclerotic process. The experimental results showed that the distributions of PAA-Fe3O4 and CSO-PAA-Fe3O4 were uniform and the corresponding sizes were about 5.93 nm and 8.15 nm, respectively. The Fourier transform infrared spectra (FTIR) testified the CSO was coupled with PAA-Fe3O4 successfully. After coupled with CSO, the r1 of PAA-Fe3O4 was increased from 5.317 mM s-1 to 6.147 mM s-1, indicating their potential as MRI contrast agent. Oil Red O staining and total cholesterols (TC) determination showed that CSO-PAA-Fe3O4 could significantly inhibit the foaming process of RAW264.7 cells induced by oxidatively modified low density lipoprotein (ox-LDL). In vitro cellular MRI displayed that, compared with PAA-Fe3O4,CSO-PAA-Fe3O4 could lower the T1 relaxation time of RAW264.7 cells better. In summary, construction of CSO-PAA-Fe3O4 nanoprobe in this study could realize the targeted MRI of macrophages and inhibition of ox-LDL induced macrophage foaming process. This will provide a new avenue in the diagnosis and treatment of AS

Spontaneously immortalised bovine mammary epithelial cells exhibit a distinct gene expression pattern from the breast cancer cells

<p>Abstract</p> <p>Background</p> <p>Spontaneous immortalisation of cultured mammary epithelial cells (MECs) is an extremely rare event, and the molecular mechanism behind spontaneous immortalisation of MECs is unclear. Here, we report the establishment of a spontaneously immortalised bovine mammary epithelial cell line (BME65Cs) and the changes in gene expression associated with BME65Cs cells.</p> <p>Results</p> <p>BME65Cs cells maintain the general characteristics of normal mammary epithelial cells in morphology, karyotype and immunohistochemistry, and are accompanied by the activation of endogenous <it>bTERT </it>(bovine Telomerase Reverse Transcriptase) and stabilisation of the telomere. Currently, BME65Cs cells have been passed for more than 220 generations, and these cells exhibit non-malignant transformation. The expression of multiple genes was investigated in BME65Cs cells, senescent BMECs (bovine MECs) cells, early passage BMECs cells and MCF-7 cells (a human breast cancer cell line). In comparison with early passage BMECs cells, the expression of senescence-relevant apoptosis-related gene were significantly changed in BME65Cs cells. P16^INK4awas downregulated, p53 was low expressed and Bax/Bcl-2 ratio was reversed. Moreover, a slight upregulation of the oncogene <it>c-Myc</it>, along with an undetectable level of breast tumor-related gene <it>Bag-1 </it>and <it>TRPS-1</it>, was observed in BME65Cs cells while these genes are all highly expressed in MCF-7. In addition, <it>DNMT1 </it>is upregulated in BME65Cs. These results suggest that the inhibition of both senescence and mitochondrial apoptosis signalling pathways contribute to the immortality of BME65Cs cells. The expression of <it>p53 </it>and <it>p16</it>^{<it>INK4a </it>}in BME65Cs was altered in the pattern of down-regulation but not "loss", suggesting that this spontaneous immortalization is possibly initiated by other mechanism rather than gene mutation of <it>p53 </it>or <it>p16</it>^{<it>INK4a</it>}.</p> <p>Conclusions</p> <p>Spontaneously immortalised BME65Cs cells maintain many characteristics of normal BMEC cells and exhibit non-malignant transformation. Although this cell line displays altered patterns of gene expression, it is clearly distinct from malignant breast cancer cell line. It showed that co-inhibition of cellular senescence and mitochondrial apoptosis pathways coordinates BME65Cs cells immortalisation. Additionally, mechanisms other than gene mutation are likely to be involved in regulation of cellular functions. This study provides an insight into the relationship between cell senescence and immortalisation. BME65Cs cells will be useful in future studies of cellular senescence and tumorigenesis.</p

Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding

Visually-situated language is ubiquitous -- sources range from textbooks with diagrams to web pages with images and tables, to mobile apps with buttons and forms. Perhaps due to this diversity, previous work has typically relied on domain-specific recipes with limited sharing of the underlying data, model architectures, and objectives. We present Pix2Struct, a pretrained image-to-text model for purely visual language understanding, which can be finetuned on tasks containing visually-situated language. Pix2Struct is pretrained by learning to parse masked screenshots of web pages into simplified HTML. The web, with its richness of visual elements cleanly reflected in the HTML structure, provides a large source of pretraining data well suited to the diversity of downstream tasks. Intuitively, this objective subsumes common pretraining signals such as OCR, language modeling, image captioning. In addition to the novel pretraining strategy, we introduce a variable-resolution input representation and a more flexible integration of language and vision inputs, where language prompts such as questions are rendered directly on top of the input image. For the first time, we show that a single pretrained model can achieve state-of-the-art results in six out of nine tasks across four domains: documents, illustrations, user interfaces, and natural images

Integrative transcriptomic profiling of mRNA, miRNA, circRNA, and lncRNA in alveolar macrophages isolated from PRRSV-infected porcine

IntroductionThe porcine reproductive and respiratory syndrome virus (PRRSV) continues to pose a significant threat to the global swine industry, attributed largely to its immunosuppressive properties and the chronic nature of its infection. The absence of effective vaccines and therapeutics amplifies the urgency to deepen our comprehension of PRRSV’s intricate pathogenic mechanisms. Previous transcriptomic studies, although informative, are partially constrained by their predominant reliance on in vitro models or lack of long-term infections. Moreover, the role of circular RNAs (circRNAs) during PRRSV invasion is yet to be elucidated.MethodsIn this study, we employed an in vivo approach, exposing piglets to a PRRSV challenge over varied durations of 3, 7, or 21 days. Subsequently, porcine alveolar macrophages were isolated for a comprehensive transcriptomic investigation, examining the expression patterns of mRNAs, miRNAs, circRNAs, and long non-coding RNAs (lncRNAs).ResultsDifferentially expressed RNAs from all four categories were identified, underscoring the dynamic interplay among these RNA species during PRRSV infection. Functional enrichment analyses indicate that these differentially expressed RNAs, as well as their target genes, play a pivotal role in immune related pathways. For the first time, we integrated circRNAs into the lncRNA-miRNA-mRNA relationship, constructing a competitive endogenous RNA (ceRNA) network. Our findings highlight the immune-related genes, CTLA4 and SAMHD1, as well as their associated miRNAs, lncRNAs, and circRNAs, suggesting potential therapeutic targets for PRRS. Importantly, we corroborated the expression patterns of selected RNAs through RT-qPCR, ensuring consistency with our transcriptomic sequencing data.DiscussionThis study sheds lights on the intricate RNA interplay during PRRSV infection and provides a solid foundation for future therapeutic strategizing

A novel fusion protein consisting of anti-ANGPTL3 antibody and interleukin-22 ameliorates diabetic nephropathy in mice

IntroductionThe pathogenic mechanisms of diabetic nephropathy (DN) include podocyte injury, inflammatory responses and metabolic disorders. Although the antagonism of Angiopoietin-like protein 3 (ANGPTL3) can alleviate proteinuria symptoms by inhibiting the activation of integrin αvβ3 on the surface of podocytes, it can not impede other pathological processes, such as inflammatory responses and metabolic dysfunction of glucolipid. Interleukin-22 (IL-22) is considered to be a pivotal molecule involved in suppressing inflammatory responses, initiating regenerative repair, and regulating glucolipid metabolism.MethodsGenes encoding the mIL22IgG2aFc and two chains of anti-ANGPTL3 antibody and bifunctional protein were synthesized. Then, the DN mice were treated with intraperitoneal injection of normal saline, anti-ANGPTL3 (20 mg/kg), mIL22Fc (12 mg/kg) or anti-ANGPTL3 /IL22 (25.3 mg/kg) and irrigation of positive drug losartan (20mg/kg/d) twice a week for 8 weeks.ResultsIn this research, a novel bifunctional fusion protein (anti-ANGPTL3/IL22) formed by the fusion of IL-22 with the C-terminus of anti-ANGPTL3 antibody exhibited favorable stability and maintained the biological activity of anti-ANGPTL3 and IL-22, respectively. The fusion protein showed a more pronounced attenuation of proteinuria and improved dysfunction of glucolipid metabolism compared with mIL22Fc or anti-ANGPTL3. Our results also indicated that anti-ANGPTL3/IL22 intervention significantly alleviated renal fibrosis via inhibiting the expression of the inflammatory response-related protein nuclear factor kappa light-chain enhancer of activated B cells (NF-κB) p65 and NOD-like receptor family pyrin domain-containing protein 3 (NLRP3) inflammasome. Moreover, transcriptome analysis revealed the downregulation of signaling pathways associated with injury and dysfunction of the renal parenchymal cell indicating the possible protective mechanisms of anti-ANGPTL3/IL22 in DN.ConclusionCollectively, anti-ANGPTL3/IL22 bifunctional fusion protein can be a promising novel therapeutic strategy for DN by reducing podocyte injury, ameliorating inflammatory response, and enhancing renal tissue recovery