Cross-Geography Generalization of Machine Learning Methods for Classification of Flooded Regions in Aerial Images

Identification of regions affected by floods is a crucial piece of information required for better planning and management of post-disaster relief and rescue efforts. Traditionally, remote sensing images are analysed to identify the extent of damage caused by flooding. The data acquired from sensors onboard earth observation satellites are analyzed to detect the flooded regions, which can be affected by low spatial and temporal resolution. However, in recent years, the images acquired from Unmanned Aerial Vehicles (UAVs) have also been utilized to assess post-disaster damage. Indeed, a UAV based platform can be rapidly deployed with a customized flight plan and minimum dependence on the ground infrastructure. This work proposes two approaches for identifying flooded regions in UAV aerial images. The first approach utilizes texture-based unsupervised segmentation to detect flooded areas, while the second uses an artificial neural network on the texture features to classify images as flooded and non-flooded. Unlike the existing works where the models are trained and tested on images of the same geographical regions, this work studies the performance of the proposed model in identifying flooded regions across geographical regions. An F1-score of 0.89 is obtained using the proposed segmentation-based approach which is higher than existing classifiers. The robustness of the proposed approach demonstrates that it can be utilized to identify flooded regions of any region with minimum or no user intervention

Agregação de ranks baseada em grafos

Orientador: Ricardo da Silva TorresTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Neste trabalho, apresentamos uma abordagem robusta de agregação de listas baseada em grafos, capaz de combinar resultados de modelos de recuperação isolados. O método segue um esquema não supervisionado, que é independente de como as listas isoladas são geradas. Nossa abordagem é capaz de incorporar modelos heterogêneos, de diferentes critérios de recuperação, tal como baseados em conteúdo textual, de imagem ou híbridos. Reformulamos o problema de recuperação ad-hoc como uma recuperação baseada em fusion graphs, que propomos como um novo modelo de representação unificada capaz de mesclar várias listas e expressar automaticamente inter-relações de resultados de recuperação. Assim, mostramos que o sistema de recuperação se beneficia do aprendizado da estrutura intrínseca das coleções, levando a melhores resultados de busca. Nossa formulação de agregação baseada em grafos, diferentemente das abordagens existentes, permite encapsular informação contextual oriunda de múltiplas listas, que podem ser usadas diretamente para ranqueamento. Experimentos realizados demonstram que o método apresenta alto desempenho, produzindo melhores eficácias que métodos recentes da literatura e promovendo ganhos expressivos sobre os métodos de recuperação fundidos. Outra contribuição é a extensão da proposta de grafo de fusão visando consulta eficiente. Trabalhos anteriores são promissores quanto à eficácia, mas geralmente ignoram questões de eficiência. Propomos uma função inovadora de agregação de consulta, não supervisionada, intrinsecamente multimodal almejando recuperação eficiente e eficaz. Introduzimos os conceitos de projeção e indexação de modelos de representação de agregação de consulta com base em grafos, e a sua aplicação em tarefas de busca. Formulações de projeção são propostas para representações de consulta baseadas em grafos. Introduzimos os fusion vectors, uma representação de fusão tardia de objetos com base em listas, a partir da qual é definido um modelo de recuperação baseado intrinsecamente em agregação. A seguir, apresentamos uma abordagem para consulta rápida baseada nos vetores de fusão, promovendo agregação de consultas eficiente. O método apresentou alta eficácia quanto ao estado da arte, além de trazer uma perspectiva de eficiência pouco abordada. Ganhos consistentes de eficiência são alcançadas em relação aos trabalhos recentes. Também propomos modelos de representação baseados em consulta para problemas gerais de predição. Os conceitos de grafos de fusão e vetores de fusão são estendidos para cenários de predição, nos quais podem ser usados para construir um modelo de estimador para determinar se um objeto de avaliação (ainda que multimodal) se refere a uma classe ou não. Experimentos em tarefas de classificação multimodal, tal como detecção de inundação, mostraram que a solução é altamente eficaz para diferentes cenários de predição que envolvam dados textuais, visuais e multimodais, produzindo resultados melhores que vários métodos recentes. Por fim, investigamos a adoção de abordagens de aprendizagem para ajudar a otimizar a criação de modelos de representação baseados em consultas, a fim de maximizar seus aspectos de capacidade discriminativa e eficiência em tarefas de predição e de buscaAbstract: In this work, we introduce a robust graph-based rank aggregation approach, capable of combining results of isolated ranker models in retrieval tasks. The method follows an unsupervised scheme, which is independent of how the isolated ranks are formulated. Our approach is able to incorporate heterogeneous models, defined in terms of different ranking criteria, such as those based on textual, image, or hybrid content representations. We reformulate the ad-hoc retrieval problem as a graph-based retrieval based on {\em fusion graphs}, which we propose as a new unified representation model capable of merging multiple ranks and expressing inter-relationships of retrieval results automatically. By doing so, we show that the retrieval system can benefit from learning the manifold structure of datasets, thus leading to more effective results. Our graph-based aggregation formulation, unlike existing approaches, allows for encapsulating contextual information encoded from multiple ranks, which can be directly used for ranking. Performed experiments demonstrate that our method reaches top performance, yielding better effectiveness scores than state-of-the-art baseline methods and promoting large gains over the rankers being fused. Another contribution refers to the extension of the fusion graph solution for efficient rank aggregation. Although previous works are promising with respect to effectiveness, they usually overlook efficiency aspects. We propose an innovative rank aggregation function that it is unsupervised, intrinsically multimodal, and targeted for fast retrieval and top effectiveness performance. We introduce the concepts of embedding and indexing graph-based rank-aggregation representation models, and their application for search tasks. Embedding formulations are also proposed for graph-based rank representations. We introduce the concept of {\em fusion vectors}, a late-fusion representation of objects based on ranks, from which an intrinsically rank-aggregation retrieval model is defined. Next, we present an approach for fast retrieval based on fusion vectors, thus promoting an efficient rank aggregation system. Our method presents top effectiveness performance among state-of-the-art related work, while promoting an efficiency perspective not yet covered. Consistent speedups are achieved against the recent baselines in all datasets considered. Derived from the fusion graphs and fusion vectors, we propose rank-based representation models for general prediction problems. The concepts of fusion graphs and fusion vectors are extended to prediction scenarios, where they can be used to build an estimator model to determine whether an input (even multimodal) object refers to a class or not. Performed experiments in the context of multimodal classification tasks, such as flood detection, show that the proposed solution is highly effective for different detection scenarios involving textual, visual, and multimodal features, yielding better detection results than several state-of-the-art methods. Finally, we investigate the adoption of learning approaches to help optimize the creation of rank-based representation models, in order to maximize their discriminative power and efficiency aspects in prediction and search tasksDoutoradoCiência da ComputaçãoDoutor em Ciência da Computaçã

A Proof-of-Concept of Integrating Machine Learning, Remote Sensing, and Survey Data in Evaluations: The Measurement of Disaster Resilience in the Philippines

Disaster resilience is a topic of increasing importance for policy makers in the context of climate change. However, measuring disaster resilience remains a challenge as it requires information on both the physical environment and socio-economic dimensions. In this study we developed and tested a method to use remote sensing (RS) data to construct proxy indicators of socio-economic change. We employed machine-learning algorithms to generate land-cover and land-use classifications from very high-resolution satellite imagery to appraise disaster damage and recovery processes in the Philippines following the devastation of typhoon Haiyan in November 2013. We constructed RS-based proxy indicators for N=20 barangays (villages) in the region surrounding Tacloban City in the central east of the Philippines. We then combined the RS-based proxy indicators with detailed socio-economic information collected during a rigorous-impact evaluation by DEval in 2016. Results from a statistical analysis demonstrated that fastest post-disaster recovery occurred in urban barangays that received sufficient government support (subsidies), and which had no prior disaster experience. In general, socio-demographic factors had stronger effects on the early recovery phase (0-2 years) compared to the late recovery phase (2-3 years). German development support was related to recovery performance only to some extent. Rather than providing an in-depth statistical analysis, this study is intended as a proof-of-concept. We have been able to demonstrate that high-resolution RS data and machine-learning techniques can be used within a mixed-methods design as an effective tool to evaluate disaster impacts and recovery processes. While RS data have distinct limitations (e.g., cost, labour intensity), they offer unique opportunities to objectively measure physical, and by extension socio-economic, changes over large areas and long time-scales.Zunehmende Wetterextreme und Naturkatastrophen sind Folgen des Klimawandels. Aufgrund dieser steigenden Risiken rückt die Resilienz der Bevölkerung im Katastrophenfall als zentrales Thema in den Vordergrund und hat zunehmende Bedeutung für politische Entscheidungstragende. Dennoch bleibt die Messung des mehrdimensionalen Konzepts der Katastrophenresilienz eine Herausforderung, da sie Informationen sowohl über die physische Umgebung als auch sozioökonomische Faktoren erfordert. In dieser Studie wird eine Methode entwickelt, um aus Fernerkundungsdaten (RS-Daten) Indikatoren zu entwickeln, die Aspekte des sozioökonomischen Wandels approximieren und somit messbar machen (Proxy-Indikatoren). Zu diesem Zweck wurden Algorithmen des maschinellen Lernens eingesetzt. Mit Hilfe dieser Algorithmen wurden aus hochauflösenden Satellitenbildern Klassifizierungen für Landstruktur und Landnutzung konstruiert, um Katastrophenschäden und iederaufbauprozesse auf den Philippinen nach der Zerstörung durch den Taifun Haiyan im November 2013 zu messen. Aus den RS-Daten wurden die Indikatoren für N=20 Barangays (Dörfer) in der Region um die Stadt Tacloban im zentralen Osten der Philippinen berechnet. Diese auf RS-Daten basierenden Indikatoren wurden mit detaillierten sozioökonomischen Informationen kombiniert, die für eine DEval-Evaluierung im Jahr 2016 erhoben wurden. Die Ergebnisse der statistischen Analyse zeigen, dass der schnellste Wiederaufbau nach der Katastrophe in städtischen Barangays zu beobachten war, die ausreichend staatliche Unterstützung (Subventionen) erhielten und über keine Katastrophenerfahrung verfügten. Im Vergleich hatten soziodemografische Faktoren allgemein stärkere Auswirkungen auf die frühe (0-2 Jahre) als auf die spätere (2-3 Jahre) Wiederaufbauphase. Es konnte nur ein bedingter Bezug zwischen der deutschen Entwicklungszusammenarbeit und den Wiederaufbauerfolgen festgestellt werden. Diese Studie versteht sich als Nachweis der Machbarkeit, weniger als detaillierte statistische Analyse. Sie belegt, dass hochauflösende RS-Daten und Techniken des maschinellen Lernens innerhalb eines integrierten Methodendesigns als effektives Werkzeug zur Bewertung von Katastrophenauswirkungen und Wiederherstellungsprozessen eingesetzt werden können. Trotz spezifischer Einschränkungen (hohe Kosten, Arbeitsintensität etc.) bieten RS-Daten einzigartige Möglichkeiten sowohl Umweltbedingungen als auch sozioökonomische Veränderungen über große Gebiete und lange Zeiträume hinweg objektiv messen zu können

Visual and Camera Sensors

This book includes 13 papers published in Special Issue ("Visual and Camera Sensors") of the journal Sensors. The goal of this Special Issue was to invite high-quality, state-of-the-art research papers dealing with challenging issues in visual and camera sensors

Detection of bodies in maritime rescue operations using Unmanned Aerial Vehicles with multispectral cameras

In this study, we use unmanned aerial vehicles equipped with multispectral cameras to search for bodies in maritime rescue operations. A series of flights were performed in open‐water scenarios in the northwest of Spain, using a certified aquatic rescue dummy in dangerous areas and real people when the weather conditions allowed it. The multispectral images were aligned and used to train a convolutional neural network for body detection. An exhaustive evaluation was performed to assess the best combination of spectral channels for this task. Three approaches based on a MobileNet topology were evaluated, using (a) the full image, (b) a sliding window, and (c) a precise localization method. The first method classifies an input image as containing a body or not, the second uses a sliding window to yield a class for each subimage, and the third uses transposed convolutions returning a binary output in which the body pixels are marked. In all cases, the MobileNet architecture was modified by adding custom layers and preprocessing the input to align the multispectral camera channels. Evaluation shows that the proposed methods yield reliable results, obtaining the best classification performance when combining green, red‐edge, and near‐infrared channels. We conclude that the precise localization approach is the most suitable method, obtaining a similar accuracy as the sliding window but achieving a spatial localization close to 1 m. The presented system is about to be implemented for real maritime rescue operations carried out by Babcock Mission Critical Services Spain.This study was performed in collaboration with BabcockMCS Spain and funded by the Galicia Region Government through the Civil UAVs Initiative program, the Spanish Government’s Ministry of Economy, Industry, and Competitiveness through the RTC‐2014‐1863‐8 and INAER4‐14Y (IDI‐20141234) projects, and the grant number 730897 under the HPC‐EUROPA3 project supported by Horizon 2020

Real-time Aerial Detection and Reasoning on Embedded-UAVs

We present a unified pipeline architecture for a real-time detection system on an embedded system for UAVs. Neural architectures have been the industry standard for computer vision. However, most existing works focus solely on concatenating deeper layers to achieve higher accuracy with run-time performance as the trade-off. This pipeline of networks can exploit the domain-specific knowledge on aerial pedestrian detection and activity recognition for the emerging UAV applications of autonomous surveying and activity reporting. In particular, our pipeline architectures operate in a time-sensitive manner, have high accuracy in detecting pedestrians from various aerial orientations, use a novel attention map for multi-activities recognition, and jointly refine its detection with temporal information. Numerically, we demonstrate our model's accuracy and fast inference speed on embedded systems. We empirically deployed our prototype hardware with full live feeds in a real-world open-field environment.Comment: In TGR

Feature extraction and selection algorithm based on self adaptive ant colony system for sky image classification

Sky image classification is crucial in meteorology to forecast weather and climatic conditions. The fine-grained cloud detection and recognition (FGCDR) algorithm is use to extract colour, inside texture and neighbour texture features from multiview of superpixels sky images. However, the FGCDR produced a substantial amount of redundant and insignificant features. The ant colony optimisation (ACO) algorithm have been used to select feature subset. However, the ACO suffers from premature convergence which leads to poor feature subset. Therefore, an improved feature extraction and selection for sky image classification (FESSIC) algorithm is proposed. This algorithm consists of (i) Gaussian smoothness standard deviation method that formulates informative features within sky images; (ii) nearest-threshold based technique that converts feature map into a weighted directed graph to represent relationship between features; and (iii) an ant colony system with self-adaptive parameter technique for local pheromone update. The performance of FESSIC was evaluated against ten benchmark image classification algorithms and six classifiers on four ground-based sky image datasets. The Friedman test result is presented for the performance rank of six benchmark feature selection algorithms and FESSIC algorithm. The Man-Whitney U test is then performed to statistically evaluate the significance difference of the second rank and FESSIC algorithms. The experimental results for the proposed algorithm are superior to the benchmark image classification algorithms in terms of similarity value on Kiel, SWIMCAT and MGCD datasets. FESSIC outperforms other algorithms for average classification accuracy for the KSVM, MLP, RF and DT classifiers. The Friedman test has shown that the FESSIC has the first rank for all classifiers. Furthermore, the result of Man-Whitney U test indicates that FESSIC is significantly better than the second rank benchmark algorithm for all classifiers. In conclusion, the FESSIC can be utilised for image classification in various applications such as disaster management, medical diagnosis, industrial inspection, sports management, and content-based image retrieval