A deep multi-modal neural network for informative Twitter content classification during emergencies

YesPeople start posting tweets containing texts, images, and videos as soon as a disaster hits an area. The analysis of these disaster-related tweet texts, images, and videos can help humanitarian response organizations in better decision-making and prioritizing their tasks. Finding the informative contents which can help in decision making out of the massive volume of Twitter content is a difficult task and require a system to filter out the informative contents. In this paper, we present a multi-modal approach to identify disaster-related informative content from the Twitter streams using text and images together. Our approach is based on long-short-term-memory (LSTM) and VGG-16 networks that show significant improvement in the performance, as evident from the validation result on seven different disaster-related datasets. The range of F1-score varied from 0.74 to 0.93 when tweet texts and images used together, whereas, in the case of only tweet text, it varies from 0.61 to 0.92. From this result, it is evident that the proposed multi-modal system is performing significantly well in identifying disaster-related informative social media contents

On Identifying Hashtags in Disaster Twitter Data

Tweet hashtags have the potential to improve the search for information during disaster events. However, there is a large number of disaster-related tweets that do not have any user-provided hashtags. Moreover, only a small number of tweets that contain actionable hashtags are useful for disaster response. To facilitate progress on automatic identification (or extraction) of disaster hashtags for Twitter data, we construct a unique dataset of disaster-related tweets annotated with hashtags useful for filtering actionable information. Using this dataset, we further investigate Long Short Term Memory-based models within a Multi-Task Learning framework. The best performing model achieves an F1-score as high as 92.22%. The dataset, code, and other resources are available on Github

Identifying and Processing Crisis Information from Social Media

Social media platforms play a crucial role in how people communicate, particularly during crisis situations such as natural disasters. People share and disseminate information on social media platforms that relates to updates, alerts, rescue and relief requests among other crisis relevant information. Hurricane Harvey and Hurricane Sandy saw over tens of millions of posts getting generated, on Twitter, in a short span of time. The ambit of such posts spreads across a wide range such as personal and official communications, and citizen sensing, to mention a few. This makes social media platforms a source of vital information to different stakeholders in crisis situations such as impacted communities, relief agencies, and civic authorities. However, the overwhelming volume of data generated during such times, makes it impossible to manually identify information relevant to crisis. Additionally, a large portion of posts in voluminous streams is not relevant or bears minimal relevance to crisis situations. This has steered much research towards exploring methods that can automatically identify crisis relevant information from voluminous streams of data during such scenarios. However, the problem of identifying crisis relevant information from social media platforms, such as Twitter, is not trivial given the nature of unstructured text such as short text length and syntactic variations among other challenges. A key objective, while creating automatic crisis relevancy classification systems, is to make them adaptable to a wide range of crisis types and languages. Many related approaches rely on statistical features which are quantifiable properties and linguistic properties of the text. A general approach is to train the classification model on labelled data acquired from crisis events and evaluate on other crisis events. A key aspect missing from explored literature is the validity of crisis relevancy classification models when applied to data from unseen types of crisis events and languages. For instance, how would the accuracy of a crisis relevancy classification model, trained on earthquake type of events, change when applied to flood type of events. Or, how would a model perform when trained on crisis data in English but applied to data in Italian. This thesis investigates these problems from a semantics perspective, where the challenges posed by diverse types of crisis and language variations are seen as the problems that can be tackled by enriching the data semantically. The use of knowledge bases such as DBpedia, BabelNet, and Wikipedia, for semantic enrichment of data in text classification problems has often been studied. Semantic enrichment of data through entity linking and expansion of context via knowledge bases can take advantage of connections between different concepts and thus enhance contextual coherency across crisis types and languages. Several previous works have focused on similar problems and proposed approaches using statistical features and/or non-semantic features. The use of semantics extracted through knowledge graphs has remained unexplored in building crisis relevancy classifiers that are adaptive to varying crisis types and multilingual data. Experiments conducted in this thesis consider data from Twitter, a micro-blogging social media platform, and analyse multiple aspects of crisis data classification. The results obtained through various analyses in this thesis demonstrate the value of semantic enrichment of text through knowledge graphs in improving the adaptability of crisis relevancy classifiers across crisis types and languages, in comparison to statistical features as often used in much of the related work

Pulling Information from Social Media in the Aftermath of Unpredictable Disasters

Social media have become a primary communication channel among people and are continuously overwhelmed by huge volumes of User Generated Content. This is especially true in the aftermath of unpredictable disasters, when users report facts, descriptions and photos of the unfolding event. This material contains actionable information that can greatly help rescuers to achieve a better response to crises, but its volume and variety render manual processing unfeasible. This paper reports the experience we gained from developing and using a web-enabled system for the online detection and monitoring of unpredictable events such as earthquakes and floods. The system captures selected message streams from Twitter and offers decision support functionalities for acquiring situational awareness from textual content and for quantifying the impact of disasters. The software architecture of the system is described and the approaches adopted for messages filtering, emergency detection and emergency monitoring are discussed. For each module, the results of real-world experiments are reported. The modular design makes the system easy configurable and allowed us to conduct experiments on different crises, including Emilia earthquake in 2012 and Genoa flood in 2014. Finally, some possible functionalities relying on the analysis of multimedia information are introduced

Analyzing Twitter Feeds to Facilitate Crises Informatics and Disaster Response During Mass Emergencies

It is a common practice these days for general public to use various micro-blogging platforms, predominantly Twitter, to share ideas, opinions and information about things and life. Twitter is also being increasingly used as a popular source of information sharing during natural disasters and mass emergencies to update and communicate the extent of the geographic phenomena, report the affected population and casualties, request or provide volunteering services and to share the status of disaster recovery process initiated by humanitarian-aid and disaster-management organizations. Recent research in this area has affirmed the potential use of such social media data for various disaster response tasks. Even though the availability of social media data is massive, open and free, there is a significant limitation in making sense of this data because of its high volume, variety, velocity, value, variability and veracity. The current work provides a comprehensive framework of text processing and analysis performed on several thousands of tweets shared on Twitter during natural disaster events. Specifically, this work em- ploys state-of-the-art machine learning techniques from natural language processing on tweet content to process the ginormous data generated at the time of disasters. This study shall serve as a basis to provide useful actionable information to the crises management and mitigation teams in planning and preparation of effective disaster response and to facilitate the development of future automated systems for handling crises situations

What roles do social media play in hurricane Ian, before, during and after the event

In recent years, natural disasters like wildfires, tsunamis, and floods have surged in both severity and frequency, causing widespread harm, including physical damage, loss of life, economic turmoil, and societal unrest. Among these disasters, hurricanes, defined by wind speeds surpassing 74 mph, pose a persistent threat, bringing hazards such as heavy rainfall and inland flooding. Hurricane Ian, one of the most significant in recent U.S. history, formed on September 23rd, hit Florida on September 28th, and dissipated on October 2nd, leaving widespread devastation. In the realm of disaster management, Location-Based Social Media (LBSM) has emerged as a crucial tool, aiding in early warnings, damage assessment, rescue coordination, and recovery evaluation. This thesis focuses on the analysis of English and Spanish tweets related to Hurricane Ian, covering the period from its formation to 50 days after its dissipation. The tweet datasets were divided into two categories: all tweets and the top 1% most shared tweets. Employing the Latent Dirichlet Allocation (LDA) model, the study unveiled prevalent themes within the tweets over different timeframes. Additionally, sentiment analysis was conducted on both English and Spanish tweet datasets, using the Valence Aware Dictionary and sEntimentReasoner (VADER) model for English tweets and Vader-multi for Spanish tweets. This aimed to capture the evolving sentiments of individuals and their emotional responses to various topics. The findings reveal Twitter's effectiveness as an early warning system and a valuable tool for risk assessment and recovery. Leading up to the hurricane's landfall, discussions mainly revolved around weather and disaster-related topics. During and after the hurricane, the focus shifted to disaster-related and situational topics. Sentiment analysis indicated a growing negativity as the storm approached, followed by a gradual return to less negative sentiments after the hurricane passed. This thesis emphasizes the significance of social media platforms as essential resources for rapid decision-making during crises, particularly when quick responses are imperative