Multi-Span Extractive Question Answering for Named Entity Recognition

openNamed Entity Recognition (NER) is a Natural Language Processing (NLP) task that involves detecting and categorizing named entities in a text. Named entities can be names of people, organizations, locations and dates or can be specifically defined for the domain in which the NER task is adopted. NER proves helpful in a variety of applications, the most notable of which is Information Extraction, where NER allows the extraction of structured information from unstructured text. Plenty of approaches have been exploited, ranging from rule-based methods to machine-learning algorithms such as Conditional Random Fields and Hidden Markov Models, to deep-learning systems based on Recurrent Neural Networks and Transformer-based architectures. Lately, with the advent of Large Language Models (LLMs) such as GPT and in-context learning techniques, a new approach to NER has emerged: extracting named entities by posing questions and having the LLM fill in the answer with the named entities (one or more) to extract. These LLMs, employed as Generative Question Answering (GQA) models, still require careful prompt-engineering, both for dealing with inputs not fitting into the context-window, and for producing well-formatted output. If a traditional NER system detects several text spans associated with a named entity, the generative QA model should be prompted to extract the same number of text spans. Furthermore, if a named entity does not appear, the generative LLM used as QA should not produce non-existent content. Based on the above considerations, NER approached as a QA task might be a viable option, although an Extractive QA method may be more appropriate over a generative one. This thesis delves into the Extractive QA approach to NER, providing a comprehensive exploration of its core concepts and methodologies. In particular, this work will first present the operating principle of single-span EQA models, which can only extract a single span of text for each question, before moving on to the design and development of a new transformer-based model that enables Multi-Span Extractive QA, implemented according to specific guidelines so that it is a natural extension of the single-span operating principle. The proposed model is then evaluated on a NER dataset named BUSTER, where specific named entities from business transaction documents have to be extracted. Experiments and comparisons with other transformer-based NER systems, which constitute the baselines for the classical NER approach, allow to show the equal power of this Multi-span EQA approach to NER. Further investigative experiments reveal that the QA approach to NER through the proposed model implementation achieves better results in reduced training set scenarios. Future work will focus on trying to develop a technique to make the model perform Continual Learning on a sequence of NER datasets while retaining its capability to correctly respond to questions from all previously encountered datasets.Named Entity Recognition (NER) is a Natural Language Processing (NLP) task that involves detecting and categorizing named entities in a text. Named entities can be names of people, organizations, locations and dates or can be specifically defined for the domain in which the NER task is adopted. NER proves helpful in a variety of applications, the most notable of which is Information Extraction, where NER allows the extraction of structured information from unstructured text. Plenty of approaches have been exploited, ranging from rule-based methods to machine-learning algorithms such as Conditional Random Fields and Hidden Markov Models, to deep-learning systems based on Recurrent Neural Networks and Transformer-based architectures. Lately, with the advent of Large Language Models (LLMs) such as GPT and in-context learning techniques, a new approach to NER has emerged: extracting named entities by posing questions and having the LLM fill in the answer with the named entities (one or more) to extract. These LLMs, employed as Generative Question Answering (GQA) models, still require careful prompt-engineering, both for dealing with inputs not fitting into the context-window, and for producing well-formatted output. If a traditional NER system detects several text spans associated with a named entity, the generative QA model should be prompted to extract the same number of text spans. Furthermore, if a named entity does not appear, the generative LLM used as QA should not produce non-existent content. Based on the above considerations, NER approached as a QA task might be a viable option, although an Extractive QA method may be more appropriate over a generative one. This thesis delves into the Extractive QA approach to NER, providing a comprehensive exploration of its core concepts and methodologies. In particular, this work will first present the operating principle of single-span EQA models, which can only extract a single span of text for each question, before moving on to the design and development of a new transformer-based model that enables Multi-Span Extractive QA, implemented according to specific guidelines so that it is a natural extension of the single-span operating principle. The proposed model is then evaluated on a NER dataset named BUSTER, where specific named entities from business transaction documents have to be extracted. Experiments and comparisons with other transformer-based NER systems, which constitute the baselines for the classical NER approach, allow to show the equal power of this Multi-span EQA approach to NER. Further investigative experiments reveal that the QA approach to NER through the proposed model implementation achieves better results in reduced training set scenarios. Future work will focus on trying to develop a technique to make the model perform Continual Learning on a sequence of NER datasets while retaining its capability to correctly respond to questions from all previously encountered datasets

IGF2 acts as a mediator of FGF22-induced differentiation of excitatory synapses in the hippocampus

Honors (Bachelor's)BiochemistryUniversity of Michiganhttp://deepblue.lib.umich.edu/bitstream/2027.42/112076/4/blbulloc2.pd

On Representational Redundancy, Surplus Structure, and the Hole Argument

We address a recent proposal concerning 'surplus structure' due to Nguyen et al. ['Why Surplus Structure is Not Superfluous.' Br. J. Phi. Sci. Forthcoming.] We argue that the sense of 'surplus structure' captured by their formal criterion is importantly different from---and in a sense, opposite to---another sense of 'surplus structure' used by philosophers. We argue that minimizing structure in one sense is generally incompatible with minimizing structure in the other sense. We then show how these distinctions bear on Nguyen et al.'s arguments about Yang-Mills theory and on the hole argument

Semantic knowledge integration for learning from semantically imprecise data

Low availability of labeled training data often poses a fundamental limit to the accuracy of computer vision applications using machine learning methods. While these methods are improved continuously, e.g., through better neural network architectures, there cannot be a single methodical change that increases the accuracy on all possible tasks. This statement, known as the no free lunch theorem, suggests that we should consider aspects of machine learning other than learning algorithms for opportunities to escape the limits set by the available training data. In this thesis, we focus on two main aspects, namely the nature of the training data, where we introduce structure into the label set using concept hierarchies, and the learning paradigm, which we change in accordance with requirements of real-world applications as opposed to more academic setups.Concept hierarchies represent semantic relations, which are sets of statements such as "a bird is an animal." We propose a hierarchical classifier to integrate this domain knowledge in a pre-existing task, thereby increasing the information the classifier has access to. While the hierarchy's leaf nodes correspond to the original set of classes, the inner nodes are "new" concepts that do not exist in the original training data. However, we pose that such "imprecise" labels are valuable and should occur naturally, e.g., as an annotator's way of expressing their uncertainty. Furthermore, the increased number of concepts leads to more possible search terms when assembling a web-crawled dataset or using an image search. We propose CHILLAX, a method that learns from semantically imprecise training data, while still offering precise predictions to integrate seamlessly into a pre-existing application

Truth and forgetting in Guatemala : an examination of memoria del silencio and nunca mas

This thesis examines the topic of memory in Guatemala in reference to the two Reports published in an effort to make the truth about the nation's decades-long war known

Process Mining Handbook

This is an open access book. This book comprises all the single courses given as part of the First Summer School on Process Mining, PMSS 2022, which was held in Aachen, Germany, during July 4-8, 2022. This volume contains 17 chapters organized into the following topical sections: Introduction; process discovery; conformance checking; data preprocessing; process enhancement and monitoring; assorted process mining topics; industrial perspective and applications; and closing

Proceedings of the 2022 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory

In August 2022, Fraunhofer IOSB and IES of KIT held a joint workshop in a Schwarzwaldhaus near Triberg. Doctoral students presented research reports and discussed various topics like computer vision, optical metrology, network security, usage control, and machine learning. This book compiles the workshop\u27s results and ideas, offering a comprehensive overview of the research program of IES and Fraunhofer IOSB

Desiring Devastated Landscapes: Love After Ecological Collapse

This dissertation examines arguments within religion and ecology, particularly within the ecospiritual movement and methodology called the new cosmology, that humans should cultivate and sustain emotional relationships with nature by caring for nonhuman others as our evolutionary kin. Focusing on the U.S. Gulf Coast after Hurricane Katrina, Hurricane Rita, and the British Petroleum oil spill, I argue that new cosmology affords few opportunities to think about intimacies with severely damaged and toxic environments. I consider how to rethink common themes in religion and ecology, like sacrality, kinship, and hope, within the context of encounters with toxic creatures and damaged ecosystems. I argue that cultivating affinity and attachment with/in ecological destruction requires thinking through how so-called “negative” affects like fear, disgust, revulsion, melancholy, shame, and despair can be an important part of ecological theory and activism. Furthermore, I contend there are other avenues for theorizing desire and kinship at the theoretical intersections of social marginalization and environmental decline that are more helpful for speaking to intimacies with and in damaged environments