Infusing Hierarchical Guidance into Prompt Tuning: A Parameter-Efficient Framework for Multi-level Implicit Discourse Relation Recognition

Multi-level implicit discourse relation recognition (MIDRR) aims at identifying hierarchical discourse relations among arguments. Previous methods achieve the promotion through fine-tuning PLMs. However, due to the data scarcity and the task gap, the pre-trained feature space cannot be accurately tuned to the task-specific space, which even aggravates the collapse of the vanilla space. Besides, the comprehension of hierarchical semantics for MIDRR makes the conversion much harder. In this paper, we propose a prompt-based Parameter-Efficient Multi-level IDRR (PEMI) framework to solve the above problems. First, we leverage parameter-efficient prompt tuning to drive the inputted arguments to match the pre-trained space and realize the approximation with few parameters. Furthermore, we propose a hierarchical label refining (HLR) method for the prompt verbalizer to deeply integrate hierarchical guidance into the prompt tuning. Finally, our model achieves comparable results on PDTB 2.0 and 3.0 using about 0.1% trainable parameters compared with baselines and the visualization demonstrates the effectiveness of our HLR method.Comment: accepted to ACL 202

A Side-by-side Comparison of Transformers for English Implicit Discourse Relation Classification

Though discourse parsing can help multiple NLP fields, there has been no wide language model search done on implicit discourse relation classification. This hinders researchers from fully utilizing public-available models in discourse analysis. This work is a straightforward, fine-tuned discourse performance comparison of seven pre-trained language models. We use PDTB-3, a popular discourse relation annotated dataset. Through our model search, we raise SOTA to 0.671 ACC and obtain novel observations. Some are contrary to what has been reported before (Shi and Demberg, 2019b), that sentence-level pre-training objectives (NSP, SBO, SOP) generally fail to produce the best performing model for implicit discourse relation classification. Counterintuitively, similar-sized PLMs with MLM and full attention led to better performance.Comment: TrustNLP @ ACL 202

Less is More: A Lightweight and Robust Neural Architecture for Discourse Parsing

Complex feature extractors are widely employed for text representation building. However, these complex feature extractors make the NLP systems prone to overfitting especially when the downstream training datasets are relatively small, which is the case for several discourse parsing tasks. Thus, we propose an alternative lightweight neural architecture that removes multiple complex feature extractors and only utilizes learnable self-attention modules to indirectly exploit pretrained neural language models, in order to maximally preserve the generalizability of pre-trained language models. Experiments on three common discourse parsing tasks show that powered by recent pretrained language models, the lightweight architecture consisting of only two self-attention layers obtains much better generalizability and robustness. Meanwhile, it achieves comparable or even better system performance with fewer learnable parameters and less processing time

Addressing the data bottleneck in implicit discourse relation classification

When humans comprehend language, their interpretation consists of more than just the sum of the content of the sentences. Additional logic and semantic links (known as coherence relations or discourse relations) are inferred between sentences/clauses in the text. The identification of discourse relations is beneficial for various NLP applications such as question-answering, summarization, machine translation, information extraction, etc. Discourse relations are categorized into implicit and explicit discourse relations depending on whether there is an explicit discourse marker between the arguments. In this thesis, we mainly focus on the implicit discourse relation classification, given that with the explicit markers acting as informative cues, the explicit relations are relatively easier to identify for machines. The recent neural network-based approaches in particular suffer from insufficient training (and test) data. As shown in Chapter 3 of this thesis, we start out by showing to what extent the limited data size is a problem in implicit discourse relation classification and propose data augmentation methods with the help of cross-lingual data. And then we propose several approaches for better exploiting and encoding various types of existing data in the discourse relation classification task. Most of the existing machine learning methods train on sections 2-21 of the PDTB and test on section 23, which only includes a total of less than 800 implicit discourse relation instances. With the help of cross validation, we argue that the standard test section of the PDTB is too small to draw conclusions upon. With more test samples in the cross validation, we would come to very different conclusions about whether a feature is generally useful. Second, we propose a simple approach to automatically extract samples of implicit discourse relations from multilingual parallel corpus via back-translation. After back-translating from target languages, it is easy for the discourse parser to identify those examples that are originally implicit but explicit in the back-translations. Having those additional data in the training set, the experiments show significant improvements on different settings. Finally, having better encoding ability is also of crucial importance in terms of improving classification performance. We propose different methods including a sequence-to-sequence neural network and a memory component to help have a better representation of the arguments. We also show that having the correct next sentence is beneficial for the task within and across domains, with the help of the BERT (Devlin et al., 2019) model. When it comes to a new domain, it is beneficial to integrate external domain-specific knowledge. In Chapter 8, we show that with the entity-enhancement, the performance on BioDRB is improved significantly, comparing with other BERT-based methods. In sum, the studies reported in this dissertation contribute to addressing the data bottleneck problem in implicit discourse relation classification and propose corresponding approaches that achieve 54.82% and 69.57% on PDTB and BioDRB respectively.Wenn Menschen Sprache verstehen, besteht ihre Interpretation aus mehr als nur der Summe des Inhalts der Sätze. Zwischen Sätzen im Text werden zusätzliche logische und semantische Verknüpfungen (sogenannte Kohärenzrelationen oder Diskursrelationen) hergeleitet. Die Identifizierung von Diskursrelationen ist für verschiedene NLP-Anwendungen wie Frage- Antwort, Zusammenfassung, maschinelle Übersetzung, Informationsextraktion usw. von Vorteil. Diskursrelationen werden in implizite und explizite Diskursrelationen unterteilt, je nachdem, ob es eine explizite Diskursrelationen zwischen den Argumenten gibt. In dieser Arbeit konzentrieren wir uns hauptsächlich auf die Klassifizierung der impliziten Diskursrelationen, da die expliziten Marker als hilfreiche Hinweise dienen und die expliziten Beziehungen für Maschinen relativ leicht zu identifizieren sind. Es wurden verschiedene Ansätze vorgeschlagen, die bei der impliziten Diskursrelationsklassifikation beeindruckende Ergebnisse erzielt haben. Die meisten von ihnen leiden jedoch darunter, dass die Daten für auf neuronalen Netzen basierende Methoden unzureichend sind. In dieser Arbeit gehen wir zunächst auf das Problem begrenzter Daten bei dieser Aufgabe ein und schlagen dann Methoden zur Datenanreicherung mit Hilfe von sprachübergreifenden Daten vor. Zuletzt schlagen wir mehrere Methoden vor, um die Argumente aus verschiedenen Aspekten besser kodieren zu können. Die meisten der existierenden Methoden des maschinellen Lernens werden auf den Abschnitten 2-21 der PDTB trainiert und auf dem Abschnitt 23 getestet, der insgesamt nur weniger als 800 implizite Diskursrelationsinstanzen enthält. Mit Hilfe der Kreuzvalidierung argumentieren wir, dass der Standardtestausschnitt der PDTB zu klein ist um daraus Schlussfolgerungen zu ziehen. Mit mehr Teststichproben in der Kreuzvalidierung würden wir zu anderen Schlussfolgerungen darüber kommen, ob ein Merkmal für diese Aufgabe generell vorteilhaft ist oder nicht, insbesondere wenn wir einen relativ großen Labelsatz verwenden. Wenn wir nur unseren kleinen Standardtestsatz herausstellen, laufen wir Gefahr, falsche Schlüsse darüber zu ziehen, welche Merkmale hilfreich sind. Zweitens schlagen wir einen einfachen Ansatz zur automatischen Extraktion von Samples impliziter Diskursrelationen aus mehrsprachigen Parallelkorpora durch Rückübersetzung vor. Er ist durch den Explikationsprozess motiviert, wenn Menschen einen Text übersetzen. Nach der Rückübersetzung aus den Zielsprachen ist es für den Diskursparser leicht, diejenigen Beispiele zu identifizieren, die ursprünglich implizit, in den Rückübersetzungen aber explizit enthalten sind. Da diese zusätzlichen Daten im Trainingsset enthalten sind, zeigen die Experimente signifikante Verbesserungen in verschiedenen Situationen. Wir verwenden zunächst nur französisch-englische Paare und haben keine Kontrolle über die Qualität und konzentrieren uns meist auf die satzinternen Relationen. Um diese Fragen in Angriff zu nehmen, erweitern wir die Idee später mit mehr Vorverarbeitungsschritten und mehr Sprachpaaren. Mit den Mehrheitsentscheidungen aus verschiedenen Sprachpaaren sind die gemappten impliziten Labels zuverlässiger. Schließlich ist auch eine bessere Kodierfähigkeit von entscheidender Bedeutung für die Verbesserung der Klassifizierungsleistung. Wir schlagen ein neues Modell vor, das aus einem Klassifikator und einem Sequenz-zu-Sequenz-Modell besteht. Neben der korrekten Vorhersage des Labels werden sie auch darauf trainiert, eine Repräsentation der Diskursrelationsargumente zu erzeugen, indem sie versuchen, die Argumente einschließlich eines geeigneten impliziten Konnektivs vorherzusagen. Die neuartige sekundäre Aufgabe zwingt die interne Repräsentation dazu, die Semantik der Relationsargumente vollständiger zu kodieren und eine feinkörnigere Klassifikation vorzunehmen. Um das allgemeine Wissen in Kontexten weiter zu erfassen, setzen wir auch ein Gedächtnisnetzwerk ein, um eine explizite Kontextrepräsentation von Trainingsbeispielen für Kontexte zu erhalten. Für jede Testinstanz erzeugen wir durch gewichtetes Lesen des Gedächtnisses einen Wissensvektor. Wir evaluieren das vorgeschlagene Modell unter verschiedenen Bedingungen und die Ergebnisse zeigen, dass das Modell mit dem Speichernetzwerk die Vorhersage von Diskursrelationen erleichtern kann, indem es Beispiele auswählt, die eine ähnliche semantische Repräsentation und Diskursrelationen aufweisen. Auch wenn ein besseres Verständnis, eine Kodierung und semantische Interpretation für die Aufgabe der impliziten Diskursrelationsklassifikation unerlässlich und nützlich sind, so leistet sie doch nur einen Teil der Arbeit. Ein guter impliziter Diskursrelationsklassifikator sollte sich auch der bevorstehenden Ereignisse, Ursachen, Folgen usw. bewusst sein, um die Diskurserwartung in die Satzdarstellungen zu kodieren. Mit Hilfe des kürzlich vorgeschlagenen BERT-Modells versuchen wir herauszufinden, ob es für die Aufgabe vorteilhaft ist, den richtigen nächsten Satz zu haben oder nicht. Die experimentellen Ergebnisse zeigen, dass das Entfernen der Aufgabe zur Vorhersage des nächsten Satzes die Leistung sowohl innerhalb der Domäne als auch domänenübergreifend stark beeinträchtigt. Die begrenzte Fähigkeit von BioBERT, domänenspezifisches Wissen, d.h. Entitätsinformationen, Entitätsbeziehungen etc. zu erlernen, motiviert uns, externes Wissen in die vortrainierten Sprachmodelle zu integrieren. Wir schlagen eine unüberwachte Methode vor, bei der Information-Retrieval-System und Wissensgraphen-Techniken verwendet werden, mit der Annahme, dass, wenn zwei Instanzen ähnliche Entitäten in beiden relationalen Argumenten teilen, die Wahrscheinlichkeit groß ist, dass sie die gleiche oder eine ähnliche Diskursrelation haben. Der Ansatz erzielt vergleichbare Ergebnisse auf BioDRB, verglichen mit Baselinemodellen. Anschließend verwenden wir die extrahierten relevanten Entitäten zur Verbesserung des vortrainierten Modells K-BERT, um die Bedeutung der Argumente besser zu kodieren und das ursprüngliche BERT und BioBERT mit einer Genauigkeit von 6,5% bzw. 2% zu übertreffen. Zusammenfassend trägt diese Dissertation dazu bei, das Problem des Datenengpasses bei der impliziten Diskursrelationsklassifikation anzugehen, und schlägt entsprechende Ansätze in verschiedenen Aspekten vor, u.a. die Darstellung des begrenzten Datenproblems und der Risiken bei der Schlussfolgerung daraus; die Erfassung automatisch annotierter Daten durch den Explikationsprozess während der manuellen Übersetzung zwischen Englisch und anderen Sprachen; eine bessere Repräsentation von Diskursrelationsargumenten; Entity-Enhancement mit einer unüberwachten Methode und einem vortrainierten Sprachmodell

Toward Multi-modal Multi-aspect Deep Alignment and Integration

Multi-modal/-aspect data contains complementary information about the same thing of interest that has the promising potential of leading to improved model robustness and thus gaining an increasing research focus. There are two typical categories of multi-modal/-aspect problems that require crossmodal/- aspect alignment and integration: 1) heterogeneous multi-modal problems that deal with data from multiple media forms, such as text, image etc., and 2) homogeneous multi-aspect problems that handle data with different aspects represented by the same media form, such as the syntactic and semantic aspects of a textual sentence etc. However, most of the existing approaches for multimodal/- aspect simply tackle the cross-modal/-aspect alignment and integration through various deep learning neural networks in an implicit manner and optimize based on the final task goals, leaving the potential strategies for improving the cross-modal/-aspect alignment and integration under-explored. This thesis aims to initiate an exploration of strategies and approaches towards multi-modal/-aspect deep alignment and integration. By looking into the limitations of existing approaches for both heterogeneous multi-modal problems and homogeneous multi-aspect problems, it proposes novel strategies and approaches for improving the cross-modal/-aspect alignment and integration and evaluates on the most essential representative tasks. For the heterogeneous setting, a cross-modal information captured graph-structured representation learning approach is proposed to enforce better cross-modal alignment and evaluated on the Language-to-Vision and Vision-and-Language scenarios. On the other hand, for the homogeneous setting, a bi-directional and deep crossintegration mechanism is explored to synthesise the multi-level semantics for comprehensive text understanding, which is validated in the joint multi-aspect natural language understanding context and its generalised text understanding setting

Christian Literature in Chinese Contexts

Christianity in China has a history dating back to the Tang Dynasty (618–907 CE), when Allopen—the first Nestorian missionary—arrived there in 635. In the late sixteenth century, Matteo Ricci together with other Jesuit missionaries commenced the Catholic missions to China. Protestant Christianity in China began with Robert Morrison, of London Missionary Society, who first set foot in Canton in 1807. Over the centuries, the Western missionaries and Chinese believers were engaged in the enterprise of the translation, publication, and distribution of a large corpus of Christian literature in Chinese. While the extensive distribution of Chinese publications facilitated the propagation of Christianity, the Christian messages have been subtly re-presented, re-appropriated, and transformed by these works of Chinese Christian literature. This Special Issue entitled “Christian Literature in Chinese Contexts” examines the multifarious dimensions of the production, translation, circulation, and reception of Christian literature (with “Christian” and “literature” in their broadest sense) against the cultural and sociopolitical contexts from the Tang period to modern China. The eight articles in this volume cover a variety of intriguing topics, including the literary/translation endeavors of Western missionaries in Chinese, the indigenous works of the Chinese Christians, the interaction between the Christian and Chinese literary traditions, Chinese reception of the Bible, and numerous other relevant concepts

Poems to Open Palms: Praise Performance and the State in the Sultanate of Oman

This dissertation traces the musical constitution of moral, economic, material, and social relations between rural communities and the state in the Sultanate of Oman. I argue that communities embedded within the authoritarian state hegemony of the Sultanate form and affirm social relations with the state through its embodied proxy, Sultan Qābūs bin Ṣa‘īd Āl Bū Ṣa‘īd, via the reciprocal exchange of state-directed giving and praise poetry responses. The circuit of exchange catalyzes the social production of political legitimacy and ensures continued generous distribution by mythopoetically presenting such cyclicity as resulting from elite and non-elite mutuality. This praise poetry is rendered within two song and dance complexes: al-razḥa, a collective war dance with drumming and antiphonal choral singing, and al-‘āzī, a choral ode with a solo singer, tight poetic structure, and a chorus of responders. Through a close analysis of the content and context of praise poems sung by Arab men’s performance troupes experienced over a year of participant observation fieldwork, I argue that praise poetry is an overlooked site for the construction and negotiation of state political legitimacy. Drawing on heterodox and Gramscian political economy, I show how musical performance operates within broader circuits of exchange by functioning as a site wherein non-market economic logics are fused with moral, performative, and political norms. Instead of simply tracing a circuit of utilitarian exchange (praise for gifts for praise), I focus on the how gifts and their responses reciprocally negotiate social relations between state elites and non-elites. By focusing on the words and actions of non elites as they integrate the various proffered benefits of a distributive state into their own communities, I attempt to complicate standard explanations of Arabian Gulf politics and statecraft. I posit two social mechanisms—one which relates generosity and political legitimacy and one that relates performance with the construction of a moral political community—and then follow them through their operation in social space. By singing praise poetry at celebrations of state distribution, praisers rhetorically render such state gifting as “generosity,” which is deeply tied to good leadership in the Omani context. In addition, praisers simultaneously mythopoetically generate a political community of generous givers and grateful receivers who are linked by relations of history, homeland, religion, and kinship. In this way, praise “opens palms” and induces continued elite distributions. However, unequal gifting is fraught with social hazard and threatens to trap communities in dependency relations with the state. By attending to the pragmatics of performance, however, I argue that razḥa and ‘āzī tacitly address this threat of dependency by performing strength and dignity while simultaneously seeking to redraw the relations of unequal gifting from ones of dependency to ones of mutual obligation—a “moral economy.” This ethnomusicological study is an attempt to show how musical and linguistic performers draw on a wide variety of tacit and explicit economic, moral, political, and communal factors in order to take social action in a context of authoritarian state hegemony

Re-Writing Composers' Lives: Critical Historiography and Musical Biography

Recent musicological discourse, while frequently considering issues of historiography and canonicity, has seldom critically engaged with biography as a genre of documentary significance to reception history for its attempts to shape public opinion of its subjects. In consequence, modern musicology has often taken for granted many tendencies and preoccupations that accumulated in musical biography in the late nineteenth and early twentieth centuries. This thesis presents a historiographical examination of the precedents for and accretions of these assumptions, in terms of the role played by biography both in the establishment and maintenance of ideological canons and in the resultant ‘top-down’ conception of music history as dominated by an elite handful of exalted composers. Exploration of the ways in which biographies constructed their subjects as ‘great’ and ‘exemplary’ – insofar as these concepts were idealized within the communities of readers for whom they were originally written – is conducted through two major studies of the published texts to c.1950 on canonical composers including J. S. Bach, Handel, Haydn, Mozart, Beethoven, Schubert, Mendelssohn, Chopin, Schumann, Wagner, Brahms, and Tchaikovsky. The first investigates the elaboration and distortion of a set of some twenty-five of the most famous myths of musical biography, from their origins in late eighteenth- and early nineteenth-century Continental European texts to their fullest development (and, in many cases, their refutation) in English-language biographies up to the mid-twentieth century. In contrast, the second critically analyzes the twelve volumes of the original ‘Master Musicians’ series (1899-1906) as exemplars of the biographical and musical paradigms of composer life-writing, and as late Victorian period pieces of significance to canon formation for their conception as a closed set of monographs of historically-important subjects appropriated to English ends. The conclusion provides a preliminary assessment of the implications to modern musicology of the findings of this thesis through re-evaluation of elements of recent biographical and hermeneutical scholarship, and proposes that the discipline might usefully adopt a more inclusive, self-reflexive approach to the study of musical biography in the future

Xenophobia in seventeenth-century India

=========ABSTRACT=========It is tempting to think of precolonial India as a harmonious society, but was it? This study brings evidence from new and unexpected sources to take position in the sensitive debate over that question. From the investigation of six conflicts in the Deccan region it draws conclusions about group behaviour that put modern clashes in context. Some of the conflicts under investigation appear odd today but were very real to the involved, as the antagonism between Left and Right Hand castes was for about a thousand years. Other conflicts continue to the present day: the seventeenth century saw lasting changes in the relationship between Hindus and Muslims as well as the rise of patriotism and early nationalism in both India and Europe. This book carefully brings to life the famous and obscure people who made the era, from the Dutch painter Heda to queen Khadija and from maharaja Shivaji to the English rebel Keigwin=========NOTES=========First Leiden University Press edition, 2009. Entirely revised from the author’s dissertation Xenophobia and Consciousness in Seventeenth-Century India: Six Cases from the Deccan, 12-Mar-2008.LEI Universiteit LeidenAsian Studie

Religion and Folk Belief in Chinese Literature and Theatre

This edited volume offers a historical, textual and ethnoanthropological exploration of the meaning and value of religion and ritual and their form and function in relation to Chinese literature and theatre. The term ‘theatre’ is used here to refer broadly to various types of live performances—theatrical and non-theatrical; sacred and profane— presented in a religious setting, thus including ritual performance and oral performance. Likewise, literature in this volume broadly encompasses both written and oral literatures, including drama, poetry, hagiography, legend, mythology and prosimetric narrative or chantefable for telling and singing. The contributors to the issue draw on a wide range of materials from historical, philosophical and literary texts to field reports and archaeological finds to archived documents and local gazetteers to personal interviews and participant observations. While all the essays are collected under the theme of ‘Religion and Folk Belief in Chinese Literature and Theatre’, they differ from each other in subject matter, source material and research approach. Rich and varied as they are, these essays fall into two main categories, namely, a historical approach to religion and ritual recorded in (written and visual) texts and an integrated approach that combines historical inquiries into written and visual texts with ethnoanthropological fieldwork on religious rituals and associated performances