6 research outputs found
CrisisMatch: Semi-Supervised Few-Shot Learning for Fine-Grained Disaster Tweet Classification
The shared real-time information about natural disasters on social media
platforms like Twitter and Facebook plays a critical role in informing
volunteers, emergency managers, and response organizations. However, supervised
learning models for monitoring disaster events require large amounts of
annotated data, making them unrealistic for real-time use in disaster events.
To address this challenge, we present a fine-grained disaster tweet
classification model under the semi-supervised, few-shot learning setting where
only a small number of annotated data is required. Our model, CrisisMatch,
effectively classifies tweets into fine-grained classes of interest using few
labeled data and large amounts of unlabeled data, mimicking the early stage of
a disaster. Through integrating effective semi-supervised learning ideas and
incorporating TextMixUp, CrisisMatch achieves performance improvement on two
disaster datasets of 11.2\% on average. Further analyses are also provided for
the influence of the number of labeled data and out-of-domain results.Comment: Accepted by ISCRAM 202
EdinburghNLP at WNUT-2020 Task 2: Leveraging Transformers with Generalized Augmentation for Identifying Informativeness in COVID-19 Tweets
Twitter has become an important communication channel in times of emergency.
The ubiquitousness of smartphones enables people to announce an emergency
they're observing in real-time. Because of this, more agencies are interested
in programatically monitoring Twitter (disaster relief organizations and news
agencies) and therefore recognizing the informativeness of a tweet can help
filter noise from large volumes of data. In this paper, we present our
submission for WNUT-2020 Task 2: Identification of informative COVID-19 English
Tweets. Our most successful model is an ensemble of transformers including
RoBERTa, XLNet, and BERTweet trained in a semi-supervised experimental setting.
The proposed system achieves a F1 score of 0.9011 on the test set (ranking 7th
on the leaderboard), and shows significant gains in performance compared to a
baseline system using fasttext embeddings.Comment: 5 pages + 1 Appendix draft (after review
DeCrisisMB: Debiased Semi-Supervised Learning for Crisis Tweet Classification via Memory Bank
During crisis events, people often use social media platforms such as Twitter
to disseminate information about the situation, warnings, advice, and support.
Emergency relief organizations leverage such information to acquire timely
crisis circumstances and expedite rescue operations. While existing works
utilize such information to build models for crisis event analysis,
fully-supervised approaches require annotating vast amounts of data and are
impractical due to limited response time. On the other hand, semi-supervised
models can be biased, performing moderately well for certain classes while
performing extremely poorly for others, resulting in substantially negative
effects on disaster monitoring and rescue. In this paper, we first study two
recent debiasing methods on semi-supervised crisis tweet classification. Then
we propose a simple but effective debiasing method, DeCrisisMB, that utilizes a
Memory Bank to store and perform equal sampling for generated pseudo-labels
from each class at each training iteration. Extensive experiments are conducted
to compare different debiasing methods' performance and generalization ability
in both in-distribution and out-of-distribution settings. The results
demonstrate the superior performance of our proposed method. Our code is
available at https://github.com/HenryPengZou/DeCrisisMB.Comment: Accepted by EMNLP 2023 (Findings
Automated Assessment of the Aftermath of Typhoons Using Social Media Texts
Disasters are one of the major threats to economics and human societies, causing substantial losses of human lives, properties and infrastructures. It has been our persistent endeavors to understand, prevent and reduce such disasters, and the popularization of social media is offering new opportunities to enhance disaster management in a crowd-sourcing approach. However, social media data is also characterized by its undue brevity, intense noise, and informality of language. The existing literature has not completely addressed these disadvantages, otherwise vast manual efforts are devoted to tackling these problems.
The major focus of this research is on constructing a holistic framework to exploit social media data in typhoon damage assessment. The scope of this research covers data collection, relevance classification, location extraction and damage assessment while assorted approaches are utilized to overcome the disadvantages of social media data. Moreover, a semi-supervised or unsupervised approach is prioritized in forming the framework to minimize manual intervention.
In data collection, query expansion strategy is adopted to optimize the search recall of typhoon-relevant information retrieval. Multiple filtering strategies are developed to screen the keywords and maintain the relevance to search topics in the keyword updates. A classifier based on a convolutional neural network is presented for relevance classification, with hashtags and word clusters as extra input channels to augment the information. In location extraction, a model is constructed by integrating Bidirectional Long Short-Time Memory and Conditional Random Fields. Feature noise correction layers and label smoothing are leveraged to handle the noisy training data. Finally, a multi-instance multi-label classifier identifies the damage relations in four categories, and the damage categories of a message are integrated with the damage descriptions score to obtain damage severity score for the message.
A case study is conducted to verify the effectiveness of the framework. The outcomes indicate that the approaches and models developed in this study significantly improve in the classification of social media texts especially under the framework of semi-supervised or unsupervised learning. Moreover, the results of damage assessment from social media data are remarkably consistent with the official statistics, which demonstrates the practicality of the proposed damage scoring scheme
Recommended from our members
Data Scarcity in Event Analysis and Abusive Language Detection
Lack of data is almost always the cause of the suboptimal performance of neural networks. Even though data scarce scenarios can be simulated for any task by assuming limited access to training data, we study two problem areas where data scarcity is a practical challenge: event analysis and abusive content detection} Journalists, social scientists and political scientists need to retrieve and analyze event mentions in unstructured text to compute useful statistical information to understand society. We claim that it is hard to specify information need about events using keyword-based representation and propose a Query by Example (QBE) setting for event retrieval. In the QBE setting, we assume that there are a few example sentences mentioning the event class a user is interested in and we aim to retrieve relevant events using only the examples as a query. Traditional event detection approaches are not applicable in this setting as event detection datasets are constructed based on pre-defined schemas which limits them to a small set of event and event-argument types. Moreover, the amount of annotated data in event detection datasets is limited that only allows us to build a retrieval corpus for evaluation. Thus we assume that there are no relevance judgments to train an event retrieval model -- except for the few examples of a specific event type. We create three QBE evaluation settings from three event detection datasets: PoliceKilling, ACE, and IndiaPoliceEvents. For the PoliceKilling dataset, where a relevant sentence describes a police killing event, we show that a query model constructed from the NLP features extracted from the few given examples is effective compared to event detection baselines. For the ACE dataset, where there are thirty-three types of events, we construct a QBE setting for each type and show that a sentence embedding approach effectively transfers for event matching. Finally, we conducted a unified evaluation of all three datasets using the sentence-embedding-based model and showed that it outperforms strong baselines.
We further examine the effect of data scarcity in abusive language detection. We first study a specific type of abusive language -- hate speech. Neural hate speech detection models trained from one dataset poorly generalize to another dataset from a different domain. This is because characteristics of hate speech vary based on racial and cultural aspects. Our data scarcity scenario assumes that we have a hate speech dataset from a domain and it needs to generalize to a test set from another domain using the unlabeled data from the test domain only. Thus we assume zero target domain data in this scenario. To tackle the data scarcity, we propose an unsupervised domain adaptation approach to augment labeled data for hate speech detection. We evaluate the approach with three different models (character CNNs, BiLSTMs, and BERT) on three different collections. We show our approach improves Area under the Precision/Recall curve by as much as 42% and recall by as much as 278%, with no loss (and in some cases a significant gain) in precision.
Finally, we examine the cross-lingual abusive language detection problem. Abusive language is a superclass of hate speech that includes profanity, aggression, offensiveness, cyberbullying, toxicity, and hate speech itself. There is a large collection of abusive language detection datasets in English such as Jigsaw. For other languages there exist datasets for abusive language detection but with very limited data. We propose a cross-lingual transfer learning approach to learn an effective neural abusive language classifier for such low-resource languages with help from a dataset from a resource-rich language. The framework is based on a nearest-neighbor architecture and is thus interpretable by design. It is a modern instantiation of the classic k-nearest neighbor model, as we use transformer representations in all its components. Unlike prior work on neighborhood-based approaches, we encode the neighborhood information based on query-neighbor interactions. We propose two encoding schemes and show their effectiveness using both qualitative and quantitative analyses. Our evaluation results on eight languages from two different datasets for abusive language detection show sizable improvements in F1 over strong baselines
Systematic Approaches for Telemedicine and Data Coordination for COVID-19 in Baja California, Mexico
Conference proceedings info:
ICICT 2023: 2023 The 6th International Conference on Information and Computer Technologies
Raleigh, HI, United States, March 24-26, 2023
Pages 529-542We provide a model for systematic implementation of telemedicine within a large evaluation center for COVID-19 in the area of Baja California, Mexico. Our model is based on human-centric design factors and cross disciplinary collaborations for scalable data-driven enablement of smartphone, cellular, and video Teleconsul-tation technologies to link hospitals, clinics, and emergency medical services for point-of-care assessments of COVID testing, and for subsequent treatment and quar-antine decisions. A multidisciplinary team was rapidly created, in cooperation with different institutions, including: the Autonomous University of Baja California, the Ministry of Health, the Command, Communication and Computer Control Center
of the Ministry of the State of Baja California (C4), Colleges of Medicine, and the College of Psychologists. Our objective is to provide information to the public and to evaluate COVID-19 in real time and to track, regional, municipal, and state-wide data in real time that informs supply chains and resource allocation with the anticipation of a surge in COVID-19 cases. RESUMEN Proporcionamos un modelo para la implementaci贸n sistem谩tica de la telemedicina dentro de un gran centro de evaluaci贸n de COVID-19 en el 谩rea de Baja California, M茅xico. Nuestro modelo se basa en factores de dise帽o centrados en el ser humano y colaboraciones interdisciplinarias para la habilitaci贸n escalable basada en datos de tecnolog铆as de teleconsulta de tel茅fonos inteligentes, celulares y video para vincular hospitales, cl铆nicas y servicios m茅dicos de emergencia para evaluaciones de COVID en el punto de atenci贸n. pruebas, y para el tratamiento posterior y decisiones de cuarentena. R谩pidamente se cre贸 un equipo multidisciplinario, en cooperaci贸n con diferentes instituciones, entre ellas: la Universidad Aut贸noma de Baja California, la Secretar铆a de Salud, el Centro de Comando, Comunicaciones y Control Inform谩tico.
de la Secretar铆a del Estado de Baja California (C4), Facultades de Medicina y Colegio de Psic贸logos. Nuestro objetivo es proporcionar informaci贸n al p煤blico y evaluar COVID-19 en tiempo real y rastrear datos regionales, municipales y estatales en tiempo real que informan las cadenas de suministro y la asignaci贸n de recursos con la anticipaci贸n de un aumento de COVID-19. 19 casos.ICICT 2023: 2023 The 6th International Conference on Information and Computer Technologieshttps://doi.org/10.1007/978-981-99-3236-