58 research outputs found
Analysis of syntactic and semantic features for fine-grained event-spatial understanding in outbreak news reports
<p>Abstract</p> <p>Background</p> <p>Previous studies have suggested that epidemiological reasoning needs a fine-grained modelling of events, especially their spatial and temporal attributes. While the temporal analysis of events has been intensively studied, far less attention has been paid to their spatial analysis. This article aims at filling the gap concerning automatic event-spatial attribute analysis in order to support health surveillance and epidemiological reasoning.</p> <p>Results</p> <p>In this work, we propose a methodology that provides a detailed analysis on each event reported in news articles to recover the most specific locations where it occurs. Various features for recognizing spatial attributes of the events were studied and incorporated into the models which were trained by several machine learning techniques. The best performance for spatial attribute recognition is very promising; 85.9% F-score (86.75% precision/85.1% recall).</p> <p>Conclusions</p> <p>We extended our work on event-spatial attribute recognition by focusing on machine learning techniques, which are CRF, SVM, and Decision tree. Our approach avoided the costly development of an external knowledge base by employing the feature sources that can be acquired locally from the analyzed document. The results showed that the CRF model performed the best. Our study indicated that the nearest location and previous event location are the most important features for the CRF and SVM model, while the location extracted from the verb's subject is the most important to the Decision tree model.</p
Identifying Conspiracy Theories News based on Event Relation Graph
Conspiracy theories, as a type of misinformation, are narratives that
explains an event or situation in an irrational or malicious manner. While most
previous work examined conspiracy theory in social media short texts, limited
attention was put on such misinformation in long news documents. In this paper,
we aim to identify whether a news article contains conspiracy theories. We
observe that a conspiracy story can be made up by mixing uncorrelated events
together, or by presenting an unusual distribution of relations between events.
Achieving a contextualized understanding of events in a story is essential for
detecting conspiracy theories. Thus, we propose to incorporate an event
relation graph for each article, in which events are nodes, and four common
types of event relations, coreference, temporal, causal, and subevent
relations, are considered as edges. Then, we integrate the event relation graph
into conspiracy theory identification in two ways: an event-aware language
model is developed to augment the basic language model with the knowledge of
events and event relations via soft labels; further, a heterogeneous graph
attention network is designed to derive a graph embedding based on hard labels.
Experiments on a large benchmark dataset show that our approach based on event
relation graph improves both precision and recall of conspiracy theory
identification, and generalizes well for new unseen media sources.Comment: Accepted to EMNLP 2023 Finding
Grounding event references in news
Events are frequently discussed in natural language, and their accurate identification is central to language understanding. Yet they are diverse and complex in ontology and reference; computational processing hence proves challenging. News provides a shared basis for communication by reporting events. We perform several studies into news event reference. One annotation study characterises each news report in terms of its update and topic events, but finds that topic is better consider through explicit references to background events. In this context, we propose the event linking task which—analogous to named entity linking or disambiguation—models the grounding of references to notable events. It defines the disambiguation of an event reference as a link to the archival article that first reports it. When two references are linked to the same article, they need not be references to the same event. Event linking hopes to provide an intuitive approximation to coreference, erring on the side of over-generation in contrast with the literature. The task is also distinguished in considering event references from multiple perspectives over time. We diagnostically evaluate the task by first linking references to past, newsworthy events in news and opinion pieces to an archive of the Sydney Morning Herald. The intensive annotation results in only a small corpus of 229 distinct links. However, we observe that a number of hyperlinks targeting online news correspond to event links. We thus acquire two large corpora of hyperlinks at very low cost. From these we learn weights for temporal and term overlap features in a retrieval system. These noisy data lead to significant performance gains over a bag-of-words baseline. While our initial system can accurately predict many event links, most will require deep linguistic processing for their disambiguation
Grounding event references in news
Events are frequently discussed in natural language, and their accurate identification is central to language understanding. Yet they are diverse and complex in ontology and reference; computational processing hence proves challenging. News provides a shared basis for communication by reporting events. We perform several studies into news event reference. One annotation study characterises each news report in terms of its update and topic events, but finds that topic is better consider through explicit references to background events. In this context, we propose the event linking task which—analogous to named entity linking or disambiguation—models the grounding of references to notable events. It defines the disambiguation of an event reference as a link to the archival article that first reports it. When two references are linked to the same article, they need not be references to the same event. Event linking hopes to provide an intuitive approximation to coreference, erring on the side of over-generation in contrast with the literature. The task is also distinguished in considering event references from multiple perspectives over time. We diagnostically evaluate the task by first linking references to past, newsworthy events in news and opinion pieces to an archive of the Sydney Morning Herald. The intensive annotation results in only a small corpus of 229 distinct links. However, we observe that a number of hyperlinks targeting online news correspond to event links. We thus acquire two large corpora of hyperlinks at very low cost. From these we learn weights for temporal and term overlap features in a retrieval system. These noisy data lead to significant performance gains over a bag-of-words baseline. While our initial system can accurately predict many event links, most will require deep linguistic processing for their disambiguation
Standardizing New Diagnostic Tests to Facilitate Rapid Responses to The Covid-19 Pandemic
In order to enhance the data interoperability, an expeditious and accurate standardization solution is highly desirable for naming rapidly emerging novel lab tests, and thus diminishes confusion in early responses to pandemic outbreaks. This is a preliminary study to explore the roles and implementation of medical informatics technology, especially natural language processing and ontology methods, in standardizing information about emerging lab tests during a pandemic, thereby facilitating rapid responses to the pandemic. The ultimate goal of this study is to develop an informatics framework for rapid standardization of lab testing names during a pandemic to better prepare for future public health threats. We first constructed an information model for lab tests approved during the COVID-19 pandemic and built a named entity recognition tool that can automatically extract lab test information specified in the information model from the Emergency Use Authorization(EUA)documents of the U.S. Food and Drug Administration (FDA), thus creating a catalog of approved lab tests with detailed information. To facilitate the standardization of lab testing data in electronic health records, we further developed the COVID-19 TestNorm, a tool that normalizes the names of various COVID-19 lab testing used by different healthcare facilities into standard Logical Observation Identifiers Names and Codes (LOINC). The overall accuracy of COVID-19 TestNorm on the development set was 98.9%, and on the independent test set was 97.4%. Lastly, we conducted a clinical study on COVID-19 re-positivity to demonstrate the utility of standardized lab test information in supporting clinical research. We believe that the result of my study indicates great a potential of medical informatics technologies for facilitating rapid responses to both current and future pandemics
Zero-Shot On-the-Fly Event Schema Induction
What are the events involved in a pandemic outbreak? What steps should be
taken when planning a wedding? The answers to these questions can be found by
collecting many documents on the complex event of interest, extracting relevant
information, and analyzing it. We present a new approach in which large
language models are utilized to generate source documents that allow
predicting, given a high-level event definition, the specific events,
arguments, and relations between them to construct a schema that describes the
complex event in its entirety. Using our model, complete schemas on any topic
can be generated on-the-fly without any manual data collection, i.e., in a
zero-shot manner. Moreover, we develop efficient methods to extract pertinent
information from texts and demonstrate in a series of experiments that these
schemas are considered to be more complete than human-curated ones in the
majority of examined scenarios. Finally, we show that this framework is
comparable in performance with previous supervised schema induction methods
that rely on collecting real texts while being more general and flexible
without the need for a predefined ontology
Doctor of Philosophy
dissertationEvents are one important type of information throughout text. Event extraction is an information extraction (IE) task that involves identifying entities and objects (mainly noun phrases) that represent important roles in events of a particular type. However, the extraction performance of current event extraction systems is limited because they mainly consider local context (mostly isolated sentences) when making each extraction decision. My research aims to improve both coverage and accuracy of event extraction performance by explicitly identifying event contexts before extracting individual facts. First, I introduce new event extraction architectures that incorporate discourse information across a document to seek out and validate pieces of event descriptions within the document. TIER is a multilayered event extraction architecture that performs text analysis at multiple granularities to progressively \zoom in" on relevant event information. LINKER is a unied discourse-guided approach that includes a structured sentence classier to sequentially read a story and determine which sentences contain event information based on both the local and preceding contexts. Experimental results on two distinct event domains show that compared to previous event extraction systems, TIER can nd more event information while maintaining a good extraction accuracy, and LINKER can further improve extraction accuracy. Finding documents that describe a specic type of event is also highly challenging because of the wide variety and ambiguity of event expressions. In this dissertation, I present the multifaceted event recognition approach that uses event dening characteristics (facets), in addition to event expressions, to eectively resolve the complexity of event descriptions. I also present a novel bootstrapping algorithm to automatically learn event expressions as well as facets of events, which requires minimal human supervision. Experimental results show that the multifaceted event recognition approach can eectively identify documents that describe a particular type of event and make event extraction systems more precise
CLEVA: Chinese Language Models EVAluation Platform
With the continuous emergence of Chinese Large Language Models (LLMs), how to
evaluate a model's capabilities has become an increasingly significant issue.
The absence of a comprehensive Chinese benchmark that thoroughly assesses a
model's performance, the unstandardized and incomparable prompting procedure,
and the prevalent risk of contamination pose major challenges in the current
evaluation of Chinese LLMs. We present CLEVA, a user-friendly platform crafted
to holistically evaluate Chinese LLMs. Our platform employs a standardized
workflow to assess LLMs' performance across various dimensions, regularly
updating a competitive leaderboard. To alleviate contamination, CLEVA curates a
significant proportion of new data and develops a sampling strategy that
guarantees a unique subset for each leaderboard round. Empowered by an
easy-to-use interface that requires just a few mouse clicks and a model API,
users can conduct a thorough evaluation with minimal coding. Large-scale
experiments featuring 23 Chinese LLMs have validated CLEVA's efficacy.Comment: EMNLP 2023 System Demonstrations camera-read
- …