Dialog Action-Aware Transformer for Dialog Policy Learning

Recent works usually address Dialog policy learning DPL by training a reinforcement learning (RL) agent to determine the best dialog action. However, existing works on deep RL require a large volume of agent-user interactions to achieve acceptable performance. In this paper, we propose to make full use of the plain text knowledge from the pre-trained language model to accelerate the RL agent's learning speed. Specifically, we design a dialog action-aware transformer encoder (DaTrans), which integrates a new fine-tuning procedure named masked last action task to encourage DaTrans to be dialog-aware and distils action-specific features. Then, DaTrans is further optimized in an RL setting with ongoing interactions and evolves through exploration in the dialog action space toward maximizing long-term accumulated rewards. The effectiveness and efficiency of the proposed model are demonstrated with both simulator evaluation and human evaluation.Comment: To be appeared in SIGdial 202

MCML: A Novel Memory-based Contrastive Meta-Learning Method for Few Shot Slot Tagging

Meta-learning is widely used for few-shot slot tagging in task of few-shot learning. The performance of existing methods is, however, seriously affected by \textit{sample forgetting issue}, where the model forgets the historically learned meta-training tasks while solely relying on support sets when adapting to new tasks. To overcome this predicament, we propose the \textbf{M}emory-based \textbf{C}ontrastive \textbf{M}eta-\textbf{L}earning (aka, MCML) method, including \textit{learn-from-the-memory} and \textit{adaption-from-the-memory} modules, which bridge the distribution gap between training episodes and between training and testing respectively. Specifically, the former uses an explicit memory bank to keep track of the label representations of previously trained episodes, with a contrastive constraint between the label representations in the current episode with the historical ones stored in the memory. In addition, the \emph{adaption-from-memory} mechanism is introduced to learn more accurate and robust representations based on the shift between the same labels embedded in the testing episodes and memory. Experimental results show that the MCML outperforms several state-of-the-art methods on both SNIPS and NER datasets and demonstrates strong scalability with consistent improvement when the number of shots gets greater

JoTR: A Joint Transformer and Reinforcement Learning Framework for Dialog Policy Learning

Dialogue policy learning (DPL) is a crucial component of dialogue modelling. Its primary role is to determine the appropriate abstract response, commonly referred to as the "dialogue action". Traditional DPL methodologies have treated this as a sequential decision problem, using pre-defined action candidates extracted from a corpus. However, these incomplete candidates can significantly limit the diversity of responses and pose challenges when dealing with edge cases, which are scenarios that occur only at extreme operating parameters. To address these limitations, we introduce a novel framework, JoTR. This framework is unique as it leverages a text-to-text Transformer-based model to generate flexible dialogue actions. Unlike traditional methods, JoTR formulates a word-level policy that allows for a more dynamic and adaptable dialogue action generation, without the need for any action templates. This setting enhances the diversity of responses and improves the system's ability to handle edge cases effectively. In addition, JoTR employs reinforcement learning with a reward-shaping mechanism to efficiently finetune the word-level dialogue policy, which allows the model to learn from its interactions, improving its performance over time. We conducted an extensive evaluation of JoTR to assess its effectiveness. Our extensive evaluation shows that JoTR achieves state-of-the-art performance on two benchmark dialogue modelling tasks, as assessed by both user simulators and human evaluators.Comment: Our code, models and other related resources are publicly available at https://github.com/KwanWaiChung/JoT

Lycium barbarum Extracts Protect the Brain from Blood-Brain Barrier Disruption and Cerebral Edema in Experimental Stroke

BACKGROUND AND PURPOSE: Ischemic stroke is a destructive cerebrovascular disease and a leading cause of death. Yet, no ideal neuroprotective agents are available, leaving prevention an attractive alternative. The extracts from the fruits of Lycium barbarum (LBP), a Chinese anti-aging medicine and food supplement, showed neuroprotective function in the retina when given prophylactically. We aim to evaluate the protective effects of LBP pre-treatment in an experimental stroke model. METHODS: C57BL/6N male mice were first fed with either vehicle (PBS) or LBP (1 or 10 mg/kg) daily for 7 days. Mice were then subjected to 2-hour transient middle cerebral artery occlusion (MCAO) by the intraluminal method followed by 22-hour reperfusion upon filament removal. Mice were evaluated for neurological deficits just before sacrifice. Brains were harvested for infarct size estimation, water content measurement, immunohistochemical analysis, and Western blot experiments. Evans blue (EB) extravasation was determined to assess blood-brain barrier (BBB) disruption after MCAO. RESULTS: LBP pre-treatment significantly improved neurological deficits as well as decreased infarct size, hemispheric swelling, and water content. Fewer apoptotic cells were identified in LBP-treated brains by TUNEL assay. Reduced EB extravasation, fewer IgG-leaky vessels, and up-regulation of occludin expression were also observed in LBP-treated brains. Moreover, immunoreactivity for aquaporin-4 and glial fibrillary acidic protein were significantly decreased in LBP-treated brains. CONCLUSIONS: Seven-day oral LBP pre-treatment effectively improved neurological deficits, decreased infarct size and cerebral edema as well as protected the brain from BBB disruption, aquaporin-4 up-regulation, and glial activation. The present study suggests that LBP may be used as a prophylactic neuroprotectant in patients at high risk for ischemic stroke.published_or_final_versio

M4LE: A Multi-Ability Multi-Range Multi-Task Multi-Domain Long-Context Evaluation Benchmark for Large Language Models

Managing long sequences has become an important and necessary feature for large language models (LLMs). However, it is still an open question of how to comprehensively and systematically evaluate the long-sequence capability of LLMs. One of the reasons is that conventional and widely-used benchmarks mainly consist of short sequences. In this paper, we propose M4LE, a Multi-ability, Multi-range, Multi-task, Multi-domain benchmark for Long-context Evaluation. M4LE is based on a diverse NLP task pool comprising 36 NLP datasets, 11 task types and 12 domains. To alleviate the scarcity of tasks with naturally long sequences and incorporate multiple-ability assessment, we propose an automatic approach (but with negligible human annotations) to convert short-sequence tasks into a unified long-sequence scenario where LLMs have to identify single or multiple relevant spans in long contexts based on explicit or semantic hints. Specifically, the scenario includes five different types of abilities: (1) explicit single-span; (2) semantic single-span; (3) explicit multiple-span; (4) semantic multiple-span; and (5) global context understanding. The resulting samples in M4LE are evenly distributed from 1k to 8k input length. We conducted a systematic evaluation on 11 well-established LLMs, especially those optimized for long-sequence inputs. Our results reveal that: 1) Current LLMs struggle to understand long context, particularly when tasks require multiple-span attention. 2) Semantic retrieval task is more difficult for competent LLMs. 3) Models fine-tuned on longer text with position interpolation have comparable performance to those using Neural Tangent Kernel (NTK) aware scaling methods without fine-tuning. We make our benchmark publicly available to encourage future research in this challenging area.Comment: Code and data are available at https://github.com/KwanWaiChung/M4L

Large Language Models as Source Planner for Personalized Knowledge-grounded Dialogue

Open-domain dialogue system usually requires different sources of knowledge to generate more informative and evidential responses. However, existing knowledge-grounded dialogue systems either focus on a single knowledge source or overlook the dependency between multiple sources of knowledge, which may result in generating inconsistent or even paradoxical responses. To incorporate multiple knowledge sources and dependencies between them, we propose SAFARI, a novel framework that leverages the exceptional capabilities of large language models (LLMs) in planning, understanding, and incorporating under both supervised and unsupervised settings. Specifically, SAFARI decouples the knowledge grounding into multiple sources and response generation, which allows easy extension to various knowledge sources including the possibility of not using any sources. To study the problem, we construct a personalized knowledge-grounded dialogue dataset \textit{\textbf{K}nowledge \textbf{B}ehind \textbf{P}ersona}~(\textbf{KBP}), which is the first to consider the dependency between persona and implicit knowledge. Experimental results on the KBP dataset demonstrate that the SAFARI framework can effectively produce persona-consistent and knowledge-enhanced responses

Dense Cranial Electroacupuncture Stimulation for Major Depressive Disorder—A Single-Blind, Randomized, Controlled Study

BACKGROUND: Previous studies suggest that electroacupuncture possesses therapeutic benefits for depressive disorders. The purpose of this study was to determine whether dense cranial electroacupuncture stimulation (DCEAS) could enhance the antidepressant efficacy in the early phase of selective serotonin reuptake inhibitor (SSRI) treatment of major depressive disorder (MDD). METHODS: In this single-blind, randomized, controlled study, patients with MDD were randomly assigned to 9-session DCEAS or noninvasive electroacupuncture (n-EA) control procedure in combination with fluoxetine (FLX) for 3 weeks. Clinical outcomes were measured using the 17-item Hamilton Depression Rating Scale (HAMD-17), Clinical Global Impression-severity (CGI-S), and Self-rating Depression Scale (SDS) as well as the response and remission rates. RESULTS: Seventy-three patients were randomly assigned to n-EA (n = 35) and DCEAS (n = 38), of whom 34 in n-EA and 36 in DCEAS group were analyzed. DCEAS-treated patients displayed a significantly greater reduction from baseline in HAMD-17 scores at Day 3 through Day 21 and in SDS scores at Day 3 and Day 21 compared to patients receiving n-EA. DCEAS intervention also produced a higher rate of clinically significant response compared to n-EA procedure (19.4% (7/36) vs. 8.8% (3/34)). The incidence of adverse events was similar in the two groups. CONCLUSIONS: DCEAS is a safe and effective intervention that augments the antidepressant efficacy. It can be considered as an additional therapy in the early phase of SSRI treatment of depressed patients. TRIAL REGISTRATION: Controlled-Trials.com ISRCTN88008690

Association of the SULT1A1 R213H polymorphism with colorectal cancer

1. Sulphotransferases are a superfamily of enzymes involved in both detoxification and bioactivation of endogenous and exogenous compounds. The arylsulphotransferase SULT1A1 has been implicated in a decreased activity and thermostability when the wild-type arginine at position 213 of the coding sequence is substituted by a histidine. SULT1A1 is the isoform primarily associated with the conversion of dietary N -OH arylamines to DNA binding adducts and is therefore of interest to determine whether this polymorphism is linked to colorectal cancer. 2. Genotyping, using polymerase chain reaction-restriction fragment length polymorphism (PCR-RFLP) analysis, was performed using DNA samples of healthy control subjects (n = 402) and patients with histologically proven colorectal cancer (n = 383). Both control and test populations possessed similar frequencies for the mutant allele (32.1 and 31%, respectively; P = 0.935). Results were not altered when age and gender were considered as potential confounders in a logistic regression analysis. 3. Examination of the sulphonating ability of the two allozymes with respect to the substrates p -nitrophenol and paracetamol showed that the affinity and rate of sulphonation was unaffected by substitution of arginine to histidine at position 213 of the amino acid sequence. 4. From this study, we conclude that the SULT1A1 R213H polymorphism is not linked with colorectal cancer in this elderly Australian population