Coreference Resolution for Software Architecture Documentation

In der Softwareentwicklung spielt die Softwarearchitekturdokumentation eine wichtige Rolle. Sie enthält viele wichtige Informationen über Gründe und Entwurfsentscheidungen. Daher gibt es viele Aktivitäten, die sich aus verschiedenen Gründen mit der Dokumentation befassen, z. B. um Informationen zu extrahieren oder verschiedene Formen der Dokumentation konsistent zu halten. Diese Aktivitäten beinhalten oft eine automatische Verarbeitung der Dokumentation, z. B. Traceability Link Recovery (TLR). Bei der automatischen Verarbeitung kann es jedoch zu Problemen kommen, wenn in der Dokumentation Koreferenzen vorhanden sind. Eine Koreferenz liegt vor, wenn sich zwei oder mehr Erwähnungen auf dieselbe Entität beziehen. Diese Erwähnungen können unterschiedlich sein und zu Mehrdeutigkeiten führen, z. B. wenn es sich um Pronomen handelt. Um dieses Problem zu lösen, werden in dieser Arbeit zwei Beiträge zur Koreferenzauflösung in der Softwarearchitekturdokumentation vorgeschlagen. Der erste Beitrag besteht darin, die Leistungsfähigkeit bestehender Modelle zur Koreferenzauflösung in der Softwarearchitekturdokumentation zu untersuchen. Der zweite Beitrag besteht darin, die Koreferenzauflösung in viele spezifischere Arten von Auflösungen zu unterteilen, wie die Pronomenauflösung, Abkürzungenauflösung usw. Für jede Kombination von spezifischen Auflösungen haben wir einen spezifischen Ansatz. Um die Arbeit dieser Abschlussarbeit zu evaluieren, werden wir uns zunächst ansehen, wie die Ansätze für die Koreferenzauflösung in der Softwarearchitekturdokumentation abschneiden. Hier erreicht Hobbs+Naive, eine Kombination aus Hobbs’ Algorithmus und naiver Nicht-Pronomen-Auflösung, einen F1-Score von 63%. StanfordCoreNLP_Deterministic, ein deterministisches System zur Koreferenzauflösung von Stanford CoreNLP, erreicht dagegen 59%. Dann wollen wir sehen, wie gut die Ansätze die Koreferenzen für eine bestimmte Aktivität, nämlich TLR, auflösen. StanfordCoreNLP_Deterministic erreicht einen F1-Score von 63%, während Hobbs+Naive 59% für diesen Aspekt erreicht. Da Koreferenzen von Pronomen eines der größten Probleme bei TLR sind, bewerten wir schließlich auch, wie die Ansätze bei der Pronomenauflösung abschneiden. In diesem Fall erreicht die Kombination mit Hobbs’ Algorithmus als Pronomenauflösungsmodell einen F1-Score von 74%, während StanfordCoreNLP_Neural nur 71% erreicht. Zusammenfassend lässt sich sagen, dass die Kombinationsansätze eine bessere Leistung bei der Koreferenzauflösung in der Softwarearchitekturdokumentation erbringen. Außerdem schneiden sie bei der Pronomenauflösung für TLR besser ab als die bestehenden Modellansätze. Nichtsdestotrotz sind die bestehenden Modellansätze bei der Koreferenzauflösung für TLR überlegen

Improving Coreference Resolution by Leveraging Entity-Centric Features with Graph Neural Networks and Second-order Inference

One of the major challenges in coreference resolution is how to make use of entity-level features defined over clusters of mentions rather than mention pairs. However, coreferent mentions usually spread far apart in an entire text, which makes it extremely difficult to incorporate entity-level features. We propose a graph neural network-based coreference resolution method that can capture the entity-centric information by encouraging the sharing of features across all mentions that probably refer to the same real-world entity. Mentions are linked to each other via the edges modeling how likely two linked mentions point to the same entity. Modeling by such graphs, the features between mentions can be shared by message passing operations in an entity-centric manner. A global inference algorithm up to second-order features is also presented to optimally cluster mentions into consistent groups. Experimental results show our graph neural network-based method combing with the second-order decoding algorithm (named GNNCR) achieved close to state-of-the-art performance on the English CoNLL-2012 Shared Task dataset

Review of coreference resolution in English and Persian

Coreference resolution (CR) is one of the most challenging areas of natural language processing. This task seeks to identify all textual references to the same real-world entity. Research in this field is divided into coreference resolution and anaphora resolution. Due to its application in textual comprehension and its utility in other tasks such as information extraction systems, document summarization, and machine translation, this field has attracted considerable interest. Consequently, it has a significant effect on the quality of these systems. This article reviews the existing corpora and evaluation metrics in this field. Then, an overview of the coreference algorithms, from rule-based methods to the latest deep learning techniques, is provided. Finally, coreference resolution and pronoun resolution systems in Persian are investigated.Comment: 44 pages, 11 figures, 5 table

Parallel Data Helps Neural Entity Coreference Resolution

Coreference resolution is the task of finding expressions that refer to the same entity in a text. Coreference models are generally trained on monolingual annotated data but annotating coreference is expensive and challenging. Hardmeier et al.(2013) have shown that parallel data contains latent anaphoric knowledge, but it has not been explored in end-to-end neural models yet. In this paper, we propose a simple yet effective model to exploit coreference knowledge from parallel data. In addition to the conventional modules learning coreference from annotations, we introduce an unsupervised module to capture cross-lingual coreference knowledge. Our proposed cross-lingual model achieves consistent improvements, up to 1.74 percentage points, on the OntoNotes 5.0 English dataset using 9 different synthetic parallel datasets. These experimental results confirm that parallel data can provide additional coreference knowledge which is beneficial to coreference resolution tasks.Comment: camera-ready version; to appear in the Findings of ACL 202

CAW-coref: Conjunction-Aware Word-level Coreference Resolution

State-of-the-art coreference resolutions systems depend on multiple LLM calls per document and are thus prohibitively expensive for many use cases (e.g., information extraction with large corpora). The leading word-level coreference system (WL-coref) attains 96.6% of these SOTA systems' performance while being much more efficient. In this work, we identify a routine yet important failure case of WL-coref: dealing with conjoined mentions such as 'Tom and Mary'. We offer a simple yet effective solution that improves the performance on the OntoNotes test set by 0.9% F1, shrinking the gap between efficient word-level coreference resolution and expensive SOTA approaches by 34.6%. Our Conjunction-Aware Word-level coreference model (CAW-coref) and code is available at https://github.com/KarelDO/wl-coref.Comment: Accepted at CRAC 202

A Pairwise Probe for Understanding BERT Fine-Tuning on Machine Reading Comprehension

Pre-trained models have brought significant improvements to many NLP tasks and have been extensively analyzed. But little is known about the effect of fine-tuning on specific tasks. Intuitively, people may agree that a pre-trained model already learns semantic representations of words (e.g. synonyms are closer to each other) and fine-tuning further improves its capabilities which require more complicated reasoning (e.g. coreference resolution, entity boundary detection, etc). However, how to verify these arguments analytically and quantitatively is a challenging task and there are few works focus on this topic. In this paper, inspired by the observation that most probing tasks involve identifying matched pairs of phrases (e.g. coreference requires matching an entity and a pronoun), we propose a pairwise probe to understand BERT fine-tuning on the machine reading comprehension (MRC) task. Specifically, we identify five phenomena in MRC. According to pairwise probing tasks, we compare the performance of each layer's hidden representation of pre-trained and fine-tuned BERT. The proposed pairwise probe alleviates the problem of distraction from inaccurate model training and makes a robust and quantitative comparison. Our experimental analysis leads to highly confident conclusions: (1) Fine-tuning has little effect on the fundamental and low-level information and general semantic tasks. (2) For specific abilities required for downstream tasks, fine-tuned BERT is better than pre-trained BERT and such gaps are obvious after the fifth layer.Comment: e.g.: 4 pages, 1 figur