DialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterances

Recent advances in pre-trained language models have significantly improved neural response generation. However, existing methods usually view the dialogue context as a linear sequence of tokens and learn to generate the next word through token-level self-attention. Such token-level encoding hinders the exploration of discourse-level coherence among utterances. This paper presents DialogBERT, a novel conversational response generation model that enhances previous PLM-based dialogue models. DialogBERT employs a hierarchical Transformer architecture. To efficiently capture the discourse-level coherence among utterances, we propose two training objectives, including masked utterance regression and distributed utterance order ranking in analogy to the original BERT training. Experiments on three multi-turn conversation datasets show that our approach remarkably outperforms the baselines, such as BART and DialoGPT, in terms of quantitative evaluation. The human evaluation suggests that DialogBERT generates more coherent, informative, and human-like responses than the baselines with significant margins.Comment: Published as a conference paper at AAAI 202

Karyotype and nucleic acid content in Zantedeschia aethiopica Spr. and Zantedeschia elliottiana Engl.

Analysis of karyotype, nucleic deoxyribonucleic acid (DNA) content and sodium dodecyl sulfate polyacrylamide gel electrophoresis (SDS-PAGE) were performed in Zantedeschia aethiopica and Zantedeschia elliottiana. Mitotic metaphase in both species showed 2n=32. The chromosomes of both species were quite similar with medium length ranging from 1.55 ± 0.04 to 3.85 ± 0.12 μM in Z. aethiopica and 2.15 ± 0.04 to 3.90 ± 0.12 μM in Z. elliottiana. However, some differences were found in morphology and centromeric position among the chromosomes. Identification of individual chromosomes was carried out using chromosomes length, and centromeric positions. The karyotype of Z. aethiopica was determined to be 2n = 32 = 14 m + 18 sm and of Z. elliottiana to be 2n = 32 = 10 m + 22 sm. The 2C nuclear DNA content was found to be 3.72 ± 0.10 picograms (equivalent to 3638.16 mega base pairs) for Z. aethiopica and 1144.26 ± 0.05 picograms (equivalent to 1144.26 mega base pairs) for Z. elliottiana. Leaf protein analysis showed 11 and 9 bands for Z. aethiopica and Z. elliottiana, respectively, among which some were species specific. These results may provide useful information regarding Zantedeschia for the study of taxonomic relationships, genetics and breeding.Keywords: Zantedeschia, karyotype, mitotic metaphase, chromosomes, flow cytometr

Continuous Decomposition of Granularity for Neural Paraphrase Generation

While Transformers have had significant success in paragraph generation, they treat sentences as linear sequences of tokens and often neglect their hierarchical information. Prior work has shown that decomposing the levels of granularity~(e.g., word, phrase, or sentence) for input tokens has produced substantial improvements, suggesting the possibility of enhancing Transformers via more fine-grained modeling of granularity. In this work, we propose a continuous decomposition of granularity for neural paraphrase generation (C-DNPG). In order to efficiently incorporate granularity into sentence encoding, C-DNPG introduces a granularity-aware attention (GA-Attention) mechanism which extends the multi-head self-attention with: 1) a granularity head that automatically infers the hierarchical structure of a sentence by neurally estimating the granularity level of each input token; and 2) two novel attention masks, namely, granularity resonance and granularity scope, to efficiently encode granularity into attention. Experiments on two benchmarks, including Quora question pairs and Twitter URLs have shown that C-DNPG outperforms baseline models by a remarkable margin and achieves state-of-the-art results in terms of many metrics. Qualitative analysis reveals that C-DNPG indeed captures fine-grained levels of granularity with effectiveness.Comment: Accepted to be published in COLING 202

A Distributed ADMM Approach to Non-Myopic Path Planning for Multi-Target Tracking

This paper investigates non-myopic path planning of mobile sensors for multi-target tracking. Such problem has posed a high computational complexity issue and/or the necessity of high-level decision making. Existing works tackle these issues by heuristically assigning targets to each sensing agent and solving the split problem for each agent. However, such heuristic methods reduce the target estimation performance in the absence of considering the changes of target state estimation along time. In this work, we detour the task-assignment problem by reformulating the general non-myopic planning problem to a distributed optimization problem with respect to targets. By combining alternating direction method of multipliers (ADMM) and local trajectory optimization method, we solve the problem and induce consensus (i.e., high-level decisions) automatically among the targets. In addition, we propose a modified receding-horizon control (RHC) scheme and edge-cutting method for efficient real-time operation. The proposed algorithm is validated through simulations in various scenarios.Comment: Copyright 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other work

Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning

Recent studies have proposed unified user modeling frameworks that leverage user behavior data from various applications. Many of them benefit from utilizing users' behavior sequences as plain texts, representing rich information in any domain or system without losing generality. Hence, a question arises: Can language modeling for user history corpus help improve recommender systems? While its versatile usability has been widely investigated in many domains, its applications to recommender systems still remain underexplored. We show that language modeling applied directly to task-specific user histories achieves excellent results on diverse recommendation tasks. Also, leveraging additional task-agnostic user histories delivers significant performance benefits. We further demonstrate that our approach can provide promising transfer learning capabilities for a broad spectrum of real-world recommender systems, even on unseen domains and services.Comment: 14 pages, 5 figures, 9 table

Solitary Necrotic Nodules of the Liver Mimicking Hepatic Metastasis: Report of Two Cases

We present two cases of solitary necrotic nodules of the liver which on radiologic images mimicked hepatic metastasis. Solitary necrotic nodule of the liver is a rare but benign entity which histopathologically consists of an outer fibrotic capsule with inflammatory cells and a central core of amorphous necrotic material. The lesion was seen on contrast-enhanced CT as an ovoid-shaped hypoattenuating nodule; on CT during hepatic arteriography as enhancing nodule; on intraoperative US as a target-appearing hypoechoic nodule; on T2WI as a hyperintensity nodule, and on dynamic MR as a subtle peripheral enhancing nodule. Although the radiologic features are not specific, solitary necrotic nodule of the liver should be included in the differential diagnosis of hepatic metastasis