MEDIC: A Multi-Task Learning Dataset for Disaster Image Classification

Recent research in disaster informatics demonstrates a practical and important use case of artificial intelligence to save human lives and suffering during natural disasters based on social media contents (text and images). While notable progress has been made using texts, research on exploiting the images remains relatively under-explored. To advance image-based approaches, we propose MEDIC (Available at: https://crisisnlp.qcri.org/medic/index.html), which is the largest social media image classification dataset for humanitarian response consisting of 71,198 images to address four different tasks in a multi-task learning setup. This is the first dataset of its kind: social media images, disaster response, and multi-task learning research. An important property of this dataset is its high potential to facilitate research on multi-task learning, which recently receives much interest from the machine learning community and has shown remarkable results in terms of memory, inference speed, performance, and generalization capability. Therefore, the proposed dataset is an important resource for advancing image-based disaster management and multi-task machine learning research. We experiment with different deep learning architectures and report promising results, which are above the majority baselines for all tasks. Along with the dataset, we also release all relevant scripts (https://github.com/firojalam/medic).Comment: Multi-task Learning, Social media images, Image Classification, Natural disasters, Crisis Informatics, Deep learning, Datase

Application of Image Processing and Convolutional Neural Networks for Flood Image Classification and Semantic Segmentation

Floods are among the most destructive natural hazards that affect millions of people across the world leading to severe loss of life and damage to property, critical infrastructure, and the environment. Deep learning algorithms are exceptionally valuable tools for collecting and analyzing the catastrophic readiness and countless actionable flood data. Convolutional neural networks (CNNs) are one form of deep learning algorithms widely used in computer vision which can be used to study flood images and assign learnable weights and biases to various objects in the image. Here, we leveraged and discussed how connected vision systems can be used to embed cameras, image processing, CNNs, and data connectivity capabilities for flood label detection. We built a training database service of \u3e9000 images (image annotation service) including the image geolocation information by streaming relevant images from social media platforms, South Carolina Department of Transportation (SCDOT) 511 traffic cameras, the US geological Survey (USGS) live river cameras, and images downloaded from search engines. All these images were manually annotated to train the different models and detect a total of eight different object categories. We then developed a new python package called “FloodImageClassifier” to classify and detect objects within the collected flood images. “FloodImageClassifier” includes various CNNs architectures such as YOLOv3 (You look only once version 3), Fast R-CNN (Region-based CNN), Mask R-CNN, SSD MobileNet (Single Shot MultiBox Detector MobileNet), and EfficientDet (efficient object detection) to perform both object detection and segmentation simultaneously. Canny edge detection and aspect ratio concepts are also included in the package for flood water level estimation and classification. The pipeline is smartly designed to train a large number of images and calculate flood water levels and inundation areas which can be used to identify flood depth, severity, and risk. “FloodImageClassifier” can be embedded to the USGS live river cameras or 511 traffic cameras to monitor river and road flooding conditions and provide early intelligence to decision makers and emergency response authorities in real-time

Application of Artificial Intelligence Approaches in the Flood Management Process for Assessing Blockage at Cross-Drainage Hydraulic Structures

Floods are the most recurrent, widespread and damaging natural disasters, and are ex-pected to become further devastating because of global warming. Blockage of cross-drainage hydraulic structures (e.g., culverts, bridges) by flood-borne debris is an influen-tial factor which usually results in reducing hydraulic capacity, diverting the flows, dam-aging structures and downstream scouring. Australia is among the countries adversely impacted by blockage issues (e.g., 1998 floods in Wollongong, 2007 floods in Newcas-tle). In this context, Wollongong City Council (WCC), under the Australian Rainfall and Runoff (ARR), investigated the impact of blockage on floods and proposed guidelines to consider blockage in the design process for the first time. However, existing WCC guide-lines are based on various assumptions (i.e., visual inspections as representative of hy-draulic behaviour, post-flood blockage as representative of peak floods, blockage remains constant during the whole flooding event), that are not supported by scientific research while also being criticised by hydraulic design engineers. This suggests the need to per-form detailed investigations of blockage from both visual and hydraulic perspectives, in order to develop quantifiable relationships and incorporate blockage into design guide-lines of hydraulic structures. However, because of the complex nature of blockage as a process and the lack of blockage-related data from actual floods, conventional numerical modelling-based approaches have not achieved much success. The research in this thesis applies artificial intelligence (AI) approaches to assess the blockage at cross-drainage hydraulic structures, motivated by recent success achieved by AI in addressing complex real-world problems (e.g., scour depth estimation and flood inundation monitoring). The research has been carried out in three phases: (a) litera-ture review, (b) hydraulic blockage assessment, and (c) visual blockage assessment. The first phase investigates the use of computer vision in the flood management domain and provides context for blockage. The second phase investigates hydraulic blockage using lab scale experiments and the implementation of multiple machine learning approaches on datasets collected from lab experiments (i.e., Hydraulics-Lab Dataset (HD), Visual Hydraulics-Lab Dataset (VHD)). The artificial neural network (ANN) and end-to-end deep learning approaches reported top performers among the implemented approaches and demonstrated the potential of learning-based approaches in addressing blockage is-sues. The third phase assesses visual blockage at culverts using deep learning classifi-cation, detection and segmentation approaches for two types of visual assessments (i.e., blockage status classification, percentage visual blockage estimation). Firstly, a range of existing convolutional neural network (CNN) image classification models are imple-mented and compared using visual datasets (i.e., Images of Culvert Openings and Block-age (ICOB), VHD, Synthetic Images of Culverts (SIC)), with the aim to automate the process of manual visual blockage classification of culverts. The Neural Architecture Search Network (NASNet) model achieved best classification results among those im-plemented. Furthermore, the study identified background noise and simplified labelling criteria as two contributing factors in degraded performance of existing CNN models for blockage classification. To address the background clutter issue, a detection-classification pipeline is proposed and achieved improved visual blockage classification performance. The proposed pipeline has been deployed using edge computing hardware for blockage monitoring of actual culverts. The role of synthetic data (i.e., SIC) on the performance of culvert opening detection is also investigated. Secondly, an automated segmentation-classification deep learning pipeline is proposed to estimate the percentage of visual blockage at circular culverts to better prioritise culvert maintenance. The AI solutions proposed in this thesis are integrated into a blockage assessment framework, designed to be deployed through edge computing to monitor, record and assess blockage at cross-drainage hydraulic structures

UMaine Today

UMaine Today magazine, published twice a year by the University of Maine Division of Marketing and Communications, showcases creativity and achievement at the University of Maine. The goal of the general-interest magazine is to demonstrate the university’s value and contributions to the state, and to advance institutional goals.https://digitalcommons.library.umaine.edu/umaine_today/1077/thumbnail.jp

D3.3 Sustainable support for citizen and DIY science

This deliverable addresses the overall question of how DITOs capacities have been used to establish sustainable ways of supporting citizen and DIY science activities in Europe after the end of the project. We have identified three mechanisms, each of which is discussed in one chapter: (1) DITOs partners acting as innovation hubs in their respective local and practice communities. With a reference to D3.2, we revisit the concept of innovation hubs and briefly discuss what follow-up activities were taking to improve it. Then we present vignettes on DITOs partners including a short overview of activities carried out and future plans along with key learnings and challenges that others can learn from. All partners have used DITOs as an opportunity to modify existing activities, develop new ones and extend their staff capacities. Beyond that, how partners used the project varies with the previous experiences with Citizen Science (CS) and Do-It-Yourself (DIY) science as well as the type of organisations and external factors. While UCL brought their work to a European scale, other partners financed core activities, established their CS activities firmer at the organisation, multiplied their impact or changed their approach. (2) Exchange and cooperations between DITOs partners are seen as a key aspect of DITOs mentioned by all partners. DITOs has made possible countless cooperation experiences between consortium members as well as with external partners. These experiences have established new relationships and continue to be available as reservoirs of resources for future activities. To make these experiences more tangible, we give an overview of past and desired future cooperations mapped at the project mid-term meeting along with five stories of different cooperation experiences identified by partners as important. (3) ECSA as an established network for citizen science in Europe backed by a more mature organisation to make DITOs legacy a basis for future work. Here we present an overview of how ECSA’s capacities have been extended, e.g. in terms of the number of members, diversified communication channels and number of EU projects. We argue that acting as a legacy organisation for DITOs translates into the question: How can ECSA continue to promote citizen science, understood in a pluralistic way, and strengthen European cooperation and cross-fertilisation between practitioners? Our answer is: By promoting openness - both by supporting CS practitioners, like the DITOs innovation hubs, to work more openly and by making ECSA a more open organisation itself. We give an overview of the events we used to explore this topic and share six dimensions of openness for CS in Europe: (1) Using pluralistic concepts, (2) Improving situated openness of data and projects, (3) Addressing questions of power, (4) Building more open organisations, (5) Promoting cross-boundary cooperation and cultural diversity and (6) Supporting fair working conditions, team support and self-care. Finally, we show exemplary measures developed at ECSA to put these principles into practice. The data for this deliverable has been gathered through workshops, interviews with DITOs partners, synthesis work on activities carried out and discussion with ECSA staff. D3.3 Sustainable support for citizen and DIY science is Deliverable 3.3 (D3.3) from the coordination and support action (CSA) Doing It Together science (DITOs), grant agreement 709443