A Recurrent Neural Model with Attention for the Recognition of Chinese Implicit Discourse Relations

We introduce an attention-based Bi-LSTM for Chinese implicit discourse relations and demonstrate that modeling argument pairs as a joint sequence can outperform word order-agnostic approaches. Our model benefits from a partial sampling scheme and is conceptually simple, yet achieves state-of-the-art performance on the Chinese Discourse Treebank. We also visualize its attention activity to illustrate the model's ability to selectively focus on the relevant parts of an input sequence.Comment: To appear at ACL2017, code available at https://github.com/sronnqvist/discourse-ablst

Argument Labeling of Discourse Relations using LSTM Neural Networks

A discourse relation can be described as a linguistic unit that is composed of sub-units that, when combined, present more information than the sum of its parts. A discourse relation is usually comprised of two arguments that relate to each other in a given form. A discourse relation may have another optional sub-unit called the discourse connective that connects the two arguments and describes the relationship between the two more explicitly. This is called Explicit Discourse relation. Extracting or labeling arguments present in an explicit discourse relations is a challenging task. In recent years, due to the CoNLL competitions, feature engineering has been applied to allow various machine learning models to achieve an F-measure value of about 55%. However, feature engineering is brittle and hand-crafted, requiring advanced knowledge of linguistics as well as the dataset in question. In this thesis, we propose an approach for segmenting (or identifying the boundaries of) Arg1 and Arg2 without feature engineering. We introduce a Bidirectional Long Short-Term Memory (LSTM) based model for argument labeling. We experimented with multiple configurations of our model. Using the Penn Discourse Treebank (PDTB) dataset, our best model achieved an F1 measure of 23.05% without any feature engineering. This is significantly higher than the 20.52% achieved by the state of the art Recurrent Neural Network (RNN) approach, but significantly lower than the feature based state of the art systems. On the other hand, because our approach learns only from the raw dataset, it is more widely applicable to multiple textual genres and languages

Neural Network Approaches to Implicit Discourse Relation Recognition

In order to understand a coherent text, humans infer semantic or logical relations between textual units. For example, in ``I am hungry. I did not have lunch today.'' the reader infers a ``causality'' relation even if it is not explicitly stated via a term such as ``because''. The linguistic device used to link textual units without the use of such explicit terms is called an ``implicit discourse relation''. Recognising implicit relations automatically is a much more challenging task than in the explicit case. Previous methods to address this problem relied heavily on conventional machine learning techniques such as CRFs and SVMs which require many hand-engineered features. In this thesis, we investigate the use of various convolutional neural networks and sequence-to-sequence models to address the automatic recognition of implicit discourse relations. We demonstrate how our sequence-to-sequence model can achieve state-of-the-art performance with the use of an attention mechanism. In addition, we investigate the automatic representation learning of discourse relations in high capacity neural networks and show that for certain discourse relations such a network does learn discourse relations in only a few neurons