355 research outputs found
Semantic-based Pre-training for Dialogue Understanding
Pre-trained language models have made great progress on dialogue tasks.
However, these models are typically trained on surface dialogue text, thus are
proven to be weak in understanding the main semantic meaning of a dialogue
context. We investigate Abstract Meaning Representation (AMR) as explicit
semantic knowledge for pre-training models to capture the core semantic
information in dialogues during pre-training. In particular, we propose a
semantic-based pre-training framework that extends the standard pre-training
framework (Devlin et al., 2019) by three tasks for learning 1) core semantic
units, 2) semantic relations and 3) the overall semantic representation
according to AMR graphs. Experiments on the understanding of both chit-chats
and task-oriented dialogues show the superiority of our model. To our
knowledge, we are the first to leverage a deep semantic representation for
dialogue pre-training.Comment: Accepted as oral in COLING202
Graph Pre-training for AMR Parsing and Generation
Abstract meaning representation (AMR) highlights the core semantic
information of text in a graph structure. Recently, pre-trained language models
(PLMs) have advanced tasks of AMR parsing and AMR-to-text generation,
respectively. However, PLMs are typically pre-trained on textual data, thus are
sub-optimal for modeling structural knowledge. To this end, we investigate
graph self-supervised training to improve the structure awareness of PLMs over
AMR graphs. In particular, we introduce two graph auto-encoding strategies for
graph-to-graph pre-training and four tasks to integrate text and graph
information during pre-training. We further design a unified framework to
bridge the gap between pre-training and fine-tuning tasks. Experiments on both
AMR parsing and AMR-to-text generation show the superiority of our model. To
our knowledge, we are the first to consider pre-training on semantic graphs.Comment: ACL2022 camera-ready final versio
Duality Regularization for Unsupervised Bilingual Lexicon Induction
Unsupervised bilingual lexicon induction naturally exhibits duality, which
results from symmetry in back-translation. For example, EN-IT and IT-EN
induction can be mutually primal and dual problems. Current state-of-the-art
methods, however, consider the two tasks independently. In this paper, we
propose to train primal and dual models jointly, using regularizers to
encourage consistency in back translation cycles. Experiments across 6 language
pairs show that the proposed method significantly outperforms competitive
baselines, obtaining the best-published results on a standard benchmark
Constituency Parsing using LLMs
Constituency parsing is a fundamental yet unsolved natural language
processing task. In this paper, we explore the potential of recent large
language models (LLMs) that have exhibited remarkable performance across
various domains and tasks to tackle this task. We employ three linearization
strategies to transform output trees into symbol sequences, such that LLMs can
solve constituency parsing by generating linearized trees. We conduct
experiments using a diverse range of LLMs, including ChatGPT, GPT-4, OPT,
LLaMA, and Alpaca, comparing their performance against the state-of-the-art
constituency parsers. Our experiments encompass zero-shot, few-shot, and
full-training learning settings, and we evaluate the models on one in-domain
and five out-of-domain test datasets. Our findings reveal insights into LLMs'
performance, generalization abilities, and challenges in constituency parsing
Traffic safety analysis of inter-tunnel weaving section with conflict prediction models
With increasing traffic demand in urban areas of metropolises, many tunnels have been constructed to improve road capacity and traffic mobility. The distance between two consecutive tunnels is relatively short which usually forms a weaving section, leading to considerable traffic conflicts. The objective of this study is to evaluate the safety performance of such inter-tunnel sections. Conflict prediction models based on negative binomial regression were developed to identify influential factors. Field data were collected at ten selected sites in Nanjing, China, and used for calibrating and validating the proposed models. Two types of inter-tunnel weaving sections (type 1 and type 2) were found in the field with distinct lane markings and operation rules. The unique lane markings in type 1 weaving sections are designed to isolate weaving traffic flows and thus reduce conflicts, but in practice, contradictory to its design intention, lead to more traffic conflicts compared with type 2 weaving sections. In addition, the length of the diverging section, merging section, and whole weaving section are found to be significant influencing factors on the conflict occurrence. The findings in the present study are expected to help engineers better design inter-tunnel sections
Dynamic network rating for low carbon distribution network operation:a U.K. application
Dynamic asset rating (DAR) is one of the number of techniques that could be used to facilitate low carbon electricity network operation. Previous work has looked at this technique from an asset perspective. This paper focuses, instead, from a network perspective by proposing a dynamic network rating (DNR) approach. The models available for use with DAR are discussed and compared using measured load and weather data from a trial network area within Milton Keynes in the central area of the U.K. This paper then uses the most appropriate model to investigate, through a network case study, the potential gains in dynamic rating compared to static rating for the different network assets - transformers, overhead lines, and cables. This will inform the network operator of the potential DNR gains on an 11-kV network with all assets present and highlight the limiting assets within each season
Distribution network reconfiguration validation with uncertain loads - network configuration determination and application
Automatic load transfer (ALT) on the 11 kV network is the process by which circuit breakers on the network are switched to form open points in order to feed load from different primary substations. Some of the potential benefits that may be gained from dynamically using ALT include maximising utilisation of existing assets, voltage regulation and reduced losses. One of the key issues, that has yet to be properly addressed in published research, is how to validate that the modelled benefits really exist. On an 11 kV distribution network where the load is continually changing and the load on each distribution substation is unlikely to be monitored - reduction in losses from moving the normally open point is particularly difficult to prove. This study proposes a method to overcome this problem and uses measured primary feeder data from two parts of the Western Power Distribution 11 kV Network under different configurations. The process of choosing the different configurations is based on a heuristic modelling method of locating minimum voltages to help reduce losses
- …