Goal-oriented Dialogue Policy Learning from Failures

Reinforcement learning methods have been used for learning dialogue policies. However, learning an effective dialogue policy frequently requires prohibitively many conversations. This is partly because of the sparse rewards in dialogues, and the very few successful dialogues in early learning phase. Hindsight experience replay (HER) enables learning from failures, but the vanilla HER is inapplicable to dialogue learning due to the implicit goals. In this work, we develop two complex HER methods providing different trade-offs between complexity and performance, and, for the first time, enabled HER-based dialogue policy learning. Experiments using a realistic user simulator show that our HER methods perform better than existing experience replay methods (as applied to deep Q-networks) in learning rate

SourceP: Smart Ponzi Schemes Detection on Ethereum Using Pre-training Model with Data Flow

As blockchain technology becomes more and more popular, a typical financial scam, the Ponzi scheme, has also emerged in the blockchain platform Ethereum. This Ponzi scheme deployed through smart contracts, also known as the smart Ponzi scheme, has caused a lot of economic losses and negative impacts. Existing methods for detecting smart Ponzi schemes on Ethereum mainly rely on bytecode features, opcode features, account features, and transaction behavior features of smart contracts, and such methods lack interpretability and sustainability. In this paper, we propose SourceP, a method to detect smart Ponzi schemes on the Ethereum platform using pre-training models and data flow, which only requires using the source code of smart contracts as features to explore the possibility of detecting smart Ponzi schemes from another direction. SourceP reduces the difficulty of data acquisition and feature extraction of existing detection methods while increasing the interpretability of the model. Specifically, we first convert the source code of a smart contract into a data flow graph and then introduce a pre-training model based on learning code representations to build a classification model to identify Ponzi schemes in smart contracts. The experimental results show that SourceP achieves 87.2\% recall and 90.7\% F-score for detecting smart Ponzi schemes within Ethereum's smart contract dataset, outperforming state-of-the-art methods in terms of performance and sustainability. We also demonstrate through additional experiments that pre-training models and data flow play an important contribution to SourceP, as well as proving that SourceP has a good generalization ability.Comment: 12 page

Learning and Reasoning for Robot Dialog and Navigation Tasks

You are viewing an article from the Proceedings of the 21st Annual Meeting of the Special Interest Group on Discourse and Dialogue that was in the Good Systems Network Digest in 2020.Office of the VP for Researc

Efficient Dialog Policy Learning by Reasoning with Contextual Knowledge

Goal-oriented dialog policy learning algorithms aim to learn a dialog policy for selecting language actions based on the current dialog state. Deep reinforcement learning methods have been used for dialog policy learning. This work is motivated by the observation that, although dialog is a domain with rich contextual knowledge, reinforcement learning methods are ill-equipped to incorporate such knowledge into the dialog policy learning process. In this paper, we develop a deep reinforcement learning framework for goal-oriented dialog policy learning that learns user preferences from user goal data, while leveraging commonsense knowledge from people. The developed framework has been evaluated using a realistic dialog simulation platform. Compared with baselines from the literature and the ablations of our approach, we see significant improvements in learning efficiency and the quality of the computed action policies

CcpA Regulates <i>Staphylococcus aureus</i> Biofilm Formation through Direct Repression of Staphylokinase Expression

Staphylococcus aureus represents a notorious opportunistic pathogen causing various infections in biofilm nature, imposing remarkable therapeutic challenges worldwide. The catabolite control protein A (CcpA), a major regulator of carbon catabolite repression (CCR), has been recognized to modulate S. aureus biofilm formation, while the underlying mechanism remains to be fully elucidated. In this study, the reduced biofilm was firstly determined in the ccpA deletion mutant of S. aureus clinical isolate XN108 using both crystal violet staining and confocal laser scanning microscopy. RNA-seq analysis suggested that sak-encoding staphylokinase (Sak) was significantly upregulated in the mutant ∆ccpA, which was further confirmed by RT-qPCR. Consistently, the induced Sak production correlated the elevated promoter activity of sak and increased secretion in the supernatants, as demonstrated by Psak-lacZ reporter fusion expression and chromogenic detection, respectively. Notably, electrophoretic mobility shift assays showed that purified recombinant protein CcpA binds directly to the promoter region of sak, suggesting the direct negative control of sak expression by CcpA. Double isogenic deletion of ccpA and sak restored biofilm formation for mutant ∆ccpA, which could be diminished by trans-complemented sak. Furthermore, the exogenous addition of recombinant Sak inhibited biofilm formation for XN108 in a dose-dependent manner. Together, this study delineates a novel model of CcpA-controlled S. aureus biofilm through direct inhibition of sak expression, highlighting the multifaceted roles and multiple networks regulated by CcpA

Bioinspired Coordination Micelles Integrating High Stability, Triggered Cargo Release, and Magnetic Resonance Imaging

Catechol-Fe³⁺ coordinated micelles show the potential for achieving on-demand drug delivery and magnetic resonance imaging in a single nanoplatform. Herein, we developed bioinspired coordination-cross-linked amphiphilic polymeric micelles loaded with a model anticancer agent, doxorubicin (Dox). The nanoscale micelles could tolerate substantial dilution to a condition below the critical micelle concentration (9.4 ± 0.3 μg/mL) without sacrificing the nanocarrier integrity due to the catechol-Fe³⁺ coordinated core cross-linking. Under acidic conditions (pH 5.0), the release rate of Dox was significantly faster compared to that at pH 7.4 as a consequence of coordination collapse and particle de-cross-linking. The cell viability study in 4T1 cells showed no toxicity regarding placebo cross-linked micelles. The micelles with improved stability showed a dramatically increased Dox accumulation in tumors and hence the enhanced suppression of tumor growth in a 4T1 tumor-bearing mouse model. The presence of Fe³⁺ endowed the micelles <i>T</i>₁-weighted MRI capability both in vitro and in vivo without the incorporation of traditional toxic paramagnetic contrast agents. The current work presented a simple “three birds with one stone” approach to engineer the robust theranostic nanomedicine platform