Large Scale Organization and Inference of an Imagery Dataset for Public Safety

Video applications and analytics are routinely projected as a stressing and significant service of the Nationwide Public Safety Broadband Network. As part of a NIST PSCR funded effort, the New Jersey Office of Homeland Security and Preparedness and MIT Lincoln Laboratory have been developing a computer vision dataset of operational and representative public safety scenarios. The scale and scope of this dataset necessitates a hierarchical organization approach for efficient compute and storage. We overview architectural considerations using the Lincoln Laboratory Supercomputing Cluster as a test architecture. We then describe how we intelligently organized the dataset across LLSC and evaluated it with large scale imagery inference across terabytes of data.Comment: Accepted for publication IEEE HPEC 201

Detecting natural disasters, damage, and incidents in the wild

Responding to natural disasters, such as earthquakes, floods, and wildfires, is a laborious task performed by on-the-ground emergency responders and analysts. Social media has emerged as a low-latency data source to quickly understand disaster situations. While most studies on social media are limited to text, images offer more information for understanding disaster and incident scenes. However, no large-scale image datasets for incident detection exists. In this work, we present the Incidents Dataset, which contains 446,684 images annotated by humans that cover 43 incidents across a variety of scenes. We employ a baseline classification model that mitigates false-positive errors and we perform image filtering experiments on millions of social media images from Flickr and Twitter. Through these experiments, we show how the Incidents Dataset can be used to detect images with incidents in the wild. Code, data, and models are available online at http://incidentsdataset.csail.mit.edu.Comment: ECCV 202

Recent advances in intelligent-based structural health monitoring of civil structures

This survey paper deals with the structural health monitoring systems on the basis of methodologies involving intelligent techniques. The intelligent techniques are the most popular tools for damage identification in terms of high accuracy, reliable nature and the involvement of low cost. In this critical survey, a thorough analysis of various intelligent techniques is carried out considering the cases involved in civil structures. The importance and utilization of various intelligent tools to be mention as the concept of fuzzy logic, the technique of genetic algorithm, the methodology of neural network techniques, as well as the approaches of hybrid methods for the monitoring of the structural health of civil structures are illustrated in a sequential manner

Exploring Relationships Between Ground and Aerial Views by Synthesis and Matching

Cross-view images, referring to the images taken from aerial and street views, contain drastically differing representations of the same scene of a given location. Due to the differences in the camera viewpoints of ground and aerial images the same semantic concepts in the two viewpoints look very different. Therefore the problem of relating them is very challenging. Thus, it becomes crucial to explore the cross-view relations and learn appropriate representations such that images from these two domains can be associated. In this dissertation we explore the relationship between ground and aerial views by synthesis and matching. First, we explore supervised approaches for cross view image synthesis problem to generate realistic images from the target (eg. ground) view, given an image from a source (eg. aerial) view. We solve this problem by utilizing Generative Adversarial Networks (GANs) to synthesize the target images and an auxiliary output, the target view segmentation maps, from source view images. We do so by enforcing the networks to correctly align and orient the different semantics in the scene by jointly penalizing the networks on the quality of target view images and the se- mantic segmentation maps. Next, we explore the geometrical cues between the aerial and ground images and attempt to preserve the pixels from aerial images to synthesize the ground images. We use homography to transform the aerial images to the street-view and preserve the pixels from the overlapping field of view, followed by inpainting the remaining regions in the ground image. Geometrically transformed images as input ease the network\u27s burden in synthesizing the cross- view images. Following the cross-view image synthesis problem, we solve the cross-view image matching problem. We propose a novel framework that uses the synthesized images for bridging the domain gap between the images from the two (aerial and ground) viewpoints and helps to learn better features for the cross-view image matching. Finally, the last part of the dissertation addresses the problem of matching the frames of a video with geo-tagged reference images for purpose of geo-localization. We develop a novel method that learns coherent features for individual frames in the query video by attending to all the frames of the video. We conduct extensive evaluations to validate that the proposed approach performs better compared to methods that learn image features independently

Automatic Building Damage Assessment Using Deep Learning and Ground-Level Image Data

We propose a novel damage assessment deep model for buildings. Common damage assessment approaches require both pre-event and post-event data, which are not available in many cases, to classify damaged areas based on the severity of destruction. In this work, we focus on assessing damage to buildings using only post-disaster data in a continuous fashion. Our model utilizes three different neural networks, one network for pre-processing the input data and two networks for extracting deep features from the input source. Combinations of these networks are distributed among three separate feature streams. A regressor summarizes extracted features into a single continuous value denoting the destruction level. To evaluate the model, we collected a small dataset of ground-level image data of damaged buildings. Experimental results demonstrate that models taking advantage of hierarchical rich features outperform baseline methods