1,319 research outputs found
Bridging the Gap in Multilingual Semantic Role Labeling: A Language-Agnostic Approach
Recent research indicates that taking advantage of complex syntactic features leads to favorable results in Semantic Role Labeling. Nonetheless, an analysis of the latest state-of-the-art multilingual systems reveals the difficulty of bridging the wide gap in performance between high-resource (e.g., English) and low-resource (e.g., German) settings. To overcome this issue, we propose a fully language-agnostic model that does away with morphological and syntactic features to achieve robustness across languages. Our approach outperforms the state of the art in all the languages of the CoNLL-2009 benchmark dataset, especially whenever a scarce amount of training data is available. Our objective is not to reject approaches that rely on syntax, rather to set a strong and consistent language-independent baseline for future innovations in Semantic Role Labeling. We release our model code and checkpoints at https://github.com/SapienzaNLP/multi-srl
Neural Unsupervised Domain Adaptation in NLP—A Survey
Deep neural networks excel at learning from labeled data and achieve
state-of-the-art results on a wide array of Natural Language Processing tasks.
In contrast, learning from unlabeled data, especially under domain shift,
remains a challenge. Motivated by the latest advances, in this survey we review
neural unsupervised domain adaptation techniques which do not require labeled
target domain data. This is a more challenging yet a more widely applicable
setup. We outline methods, from early approaches in traditional non-neural
methods to pre-trained model transfer. We also revisit the notion of domain,
and we uncover a bias in the type of Natural Language Processing tasks which
received most attention. Lastly, we outline future directions, particularly the
broader need for out-of-distribution generalization of future intelligent NLP
Persona-centric Metamorphic Relation guided Robustness Evaluation for Multi-turn Dialogue Modelling
Recently there has been significant progress in the field of dialogue system
thanks to the introduction of training paradigms such as fine-tune and prompt
learning. Persona can function as the prior knowledge for maintaining the
personality consistency of dialogue systems, which makes it perform well on
accuracy. Nonetheless, the conventional reference-based evaluation method falls
short in capturing the genuine text comprehension prowess of the model,
significantly relying on the quality of data annotation. In contrast, the
application of metamorphic testing offers a more profound insight into the
model's distinct capabilities without necessitating supplementary annotation
labels. This approach furnishes a more comprehensive portrayal of the model's
intricacies and exposes intricacies concealed within reference-based validation
techniques. Consequently, we introduce a persona-centric metamorphic relation
construction for metamorphic testing, aimed at evaluating both the persona
consistency and robustness of personalized dialogue models. For that reason,
this work evaluates several widely used training paradigms including learning
from scratch, pretrain + fine-tune and prompt learning in personalized dialogue
retrieval to know if they are more robust or if they have the same flaws as
their predecessor. Under three kinds of designed metamorphic relations with
consistent outputs, our experimental results reveal that prompt learning shows
stronger robustness compared to training from scratch and fine-tune. Although
tested retrieval models gain competitively high retrieval accuracy according to
the traditional reference-based validation, they are still fragile and
demonstrate various unexpected behaviors, thus there is still room for future
improvement in personalized dialogue retrieval
- …