Semantic-based Pre-training for Dialogue Understanding

Pre-trained language models have made great progress on dialogue tasks. However, these models are typically trained on surface dialogue text, thus are proven to be weak in understanding the main semantic meaning of a dialogue context. We investigate Abstract Meaning Representation (AMR) as explicit semantic knowledge for pre-training models to capture the core semantic information in dialogues during pre-training. In particular, we propose a semantic-based pre-training framework that extends the standard pre-training framework (Devlin et al., 2019) by three tasks for learning 1) core semantic units, 2) semantic relations and 3) the overall semantic representation according to AMR graphs. Experiments on the understanding of both chit-chats and task-oriented dialogues show the superiority of our model. To our knowledge, we are the first to leverage a deep semantic representation for dialogue pre-training.Comment: Accepted as oral in COLING202

Graph Pre-training for AMR Parsing and Generation

Abstract meaning representation (AMR) highlights the core semantic information of text in a graph structure. Recently, pre-trained language models (PLMs) have advanced tasks of AMR parsing and AMR-to-text generation, respectively. However, PLMs are typically pre-trained on textual data, thus are sub-optimal for modeling structural knowledge. To this end, we investigate graph self-supervised training to improve the structure awareness of PLMs over AMR graphs. In particular, we introduce two graph auto-encoding strategies for graph-to-graph pre-training and four tasks to integrate text and graph information during pre-training. We further design a unified framework to bridge the gap between pre-training and fine-tuning tasks. Experiments on both AMR parsing and AMR-to-text generation show the superiority of our model. To our knowledge, we are the first to consider pre-training on semantic graphs.Comment: ACL2022 camera-ready final versio

Duality Regularization for Unsupervised Bilingual Lexicon Induction

Unsupervised bilingual lexicon induction naturally exhibits duality, which results from symmetry in back-translation. For example, EN-IT and IT-EN induction can be mutually primal and dual problems. Current state-of-the-art methods, however, consider the two tasks independently. In this paper, we propose to train primal and dual models jointly, using regularizers to encourage consistency in back translation cycles. Experiments across 6 language pairs show that the proposed method significantly outperforms competitive baselines, obtaining the best-published results on a standard benchmark

Constituency Parsing using LLMs

Constituency parsing is a fundamental yet unsolved natural language processing task. In this paper, we explore the potential of recent large language models (LLMs) that have exhibited remarkable performance across various domains and tasks to tackle this task. We employ three linearization strategies to transform output trees into symbol sequences, such that LLMs can solve constituency parsing by generating linearized trees. We conduct experiments using a diverse range of LLMs, including ChatGPT, GPT-4, OPT, LLaMA, and Alpaca, comparing their performance against the state-of-the-art constituency parsers. Our experiments encompass zero-shot, few-shot, and full-training learning settings, and we evaluate the models on one in-domain and five out-of-domain test datasets. Our findings reveal insights into LLMs' performance, generalization abilities, and challenges in constituency parsing

Traffic safety analysis of inter-tunnel weaving section with conflict prediction models

With increasing traffic demand in urban areas of metropolises, many tunnels have been constructed to improve road capacity and traffic mobility. The distance between two consecutive tunnels is relatively short which usually forms a weaving section, leading to considerable traffic conflicts. The objective of this study is to evaluate the safety performance of such inter-tunnel sections. Conflict prediction models based on negative binomial regression were developed to identify influential factors. Field data were collected at ten selected sites in Nanjing, China, and used for calibrating and validating the proposed models. Two types of inter-tunnel weaving sections (type 1 and type 2) were found in the field with distinct lane markings and operation rules. The unique lane markings in type 1 weaving sections are designed to isolate weaving traffic flows and thus reduce conflicts, but in practice, contradictory to its design intention, lead to more traffic conflicts compared with type 2 weaving sections. In addition, the length of the diverging section, merging section, and whole weaving section are found to be significant influencing factors on the conflict occurrence. The findings in the present study are expected to help engineers better design inter-tunnel sections

Dynamic network rating for low carbon distribution network operation:a U.K. application

Dynamic asset rating (DAR) is one of the number of techniques that could be used to facilitate low carbon electricity network operation. Previous work has looked at this technique from an asset perspective. This paper focuses, instead, from a network perspective by proposing a dynamic network rating (DNR) approach. The models available for use with DAR are discussed and compared using measured load and weather data from a trial network area within Milton Keynes in the central area of the U.K. This paper then uses the most appropriate model to investigate, through a network case study, the potential gains in dynamic rating compared to static rating for the different network assets - transformers, overhead lines, and cables. This will inform the network operator of the potential DNR gains on an 11-kV network with all assets present and highlight the limiting assets within each season

Distribution network reconfiguration validation with uncertain loads - network configuration determination and application

Automatic load transfer (ALT) on the 11 kV network is the process by which circuit breakers on the network are switched to form open points in order to feed load from different primary substations. Some of the potential benefits that may be gained from dynamically using ALT include maximising utilisation of existing assets, voltage regulation and reduced losses. One of the key issues, that has yet to be properly addressed in published research, is how to validate that the modelled benefits really exist. On an 11 kV distribution network where the load is continually changing and the load on each distribution substation is unlikely to be monitored - reduction in losses from moving the normally open point is particularly difficult to prove. This study proposes a method to overcome this problem and uses measured primary feeder data from two parts of the Western Power Distribution 11 kV Network under different configurations. The process of choosing the different configurations is based on a heuristic modelling method of locating minimum voltages to help reduce losses