3 research outputs found
Filtered Semi-Markov CRF
Semi-Markov CRF has been proposed as an alternative to the traditional Linear
Chain CRF for text segmentation tasks such as Named Entity Recognition (NER).
Unlike CRF, which treats text segmentation as token-level prediction, Semi-CRF
considers segments as the basic unit, making it more expressive. However,
Semi-CRF suffers from two major drawbacks: (1) quadratic complexity over
sequence length, as it operates on every span of the input sequence, and (2)
inferior performance compared to CRF for sequence labeling tasks like NER. In
this paper, we introduce Filtered Semi-Markov CRF, a variant of Semi-CRF that
addresses these issues by incorporating a filtering step to eliminate
irrelevant segments, reducing complexity and search space. Our approach is
evaluated on several NER benchmarks, where it outperforms both CRF and Semi-CRF
while being significantly faster. The implementation of our method is available
on \href{https://github.com/urchade/Filtered-Semi-Markov-CRF}{Github}.Comment: EMNLP 2023 (Findings
DyREx: Dynamic Query Representation for Extractive Question Answering
Extractive question answering (ExQA) is an essential task for Natural
Language Processing. The dominant approach to ExQA is one that represents the
input sequence tokens (question and passage) with a pre-trained transformer,
then uses two learned query vectors to compute distributions over the start and
end answer span positions. These query vectors lack the context of the inputs,
which can be a bottleneck for the model performance. To address this problem,
we propose \textit{DyREx}, a generalization of the \textit{vanilla} approach
where we dynamically compute query vectors given the input, using an attention
mechanism through transformer layers. Empirical observations demonstrate that
our approach consistently improves the performance over the standard one. The
code and accompanying files for running the experiments are available at
\url{https://github.com/urchade/DyReX}.Comment: Accepted at "2nd Workshop on Efficient Natural Language and Speech
Processing (ENLSP-II)" @ NeurIPS 202
Sélection globale de segments pour la reconnaissance d'entités nommées
International audienceNamed Entity Recognition is an important task in Natural Language Processing with applications in many domains. In this paper, we describe a novel approach to named entity recognition, in which we output a set of spans (i.e., segmentations) by maximizing a global score. During training, we optimize our model by maximizing the probability of the gold segmentation. During inference, we use dynamic programming to select the best segmentation under a linear time complexity. We prove that our approach outperforms CRF and semi-CRF models for Named Entity RecognitionLa reconnaissance d'entités nommées est une tâche importante en traitement automatique du langage naturel avec des applications dans de nombreux domaines. Dans cet article, nous décrivons une nouvelle approche pour la reconnaissance d'entités nommées, dans laquelle nous produisons un ensemble de segmentations en maximisant un score global. Pendant l'entraînement, nous optimisons notre modèle en maximisant la probabilité de la segmentation correcte. Pendant l'inférence, nous utilisons la programmation dynamique pour sélectionner la meilleure segmentation avec une complexité linéaire. Nous prouvons que notre approche est supérieure aux modèles champs de Markov conditionnels et semi-CMC pour la reconnaissance d'entités nommées