Automated location of active fire perimeters in aerial infrared imaging using unsupervised edge detectors

A variety of remote sensing techniques have been applied to forest fires. However, there is at present no system capable of monitoring an active fire precisely in a totally automated manner. Spaceborne sensors show too coarse spatio-temporal resolutions and all previous studies that extracted fire properties from infrared aerial imagery incorporated manual tasks within the image processing workflow. As a contribution to this topic, this paper presents an algorithm to automatically locate the fuel burning interface of an active wildfire in georeferenced aerial thermal infrared (TIR) imagery. An unsupervised edge detector, built upon the Canny method, was accompanied by the necessary modules for the extraction of line coordinates and the location of the total burned perimeter. The system was validated in different scenarios ranging from laboratory tests to large-scale experimental burns performed under extreme weather conditions. Output accuracy was computed through three common similarity indices and proved acceptable. Computing times were below 1¿s per image on average. The produced information was used to measure the temporal evolution of the fire perimeter and automatically generate rate of spread (ROS) fields. Information products were easily exported to standard Geographic Information Systems (GIS), such as GoogleEarth and QGIS. Therefore, this work contributes towards the development of an affordable and totally automated system for operational wildfire surveillance.Peer ReviewedPostprint (author's final draft

A novel image feature descriptor for SLM spattering pattern classification using a consumable camera

In selective laser melting (SLM), spattering is an important phenomenon that is highly related to the quality of the manufactured parts. Characterisation and monitoring of spattering behaviours are highly valuable in understanding the manufacturing process and improving the manufacturing quality of SLM. This paper introduces a method of automatic visual classification to distinguish spattering characteristics of SLM processes in different manufacturing conditions. A compact feature descriptor is proposed to represent spattering patterns and its effectiveness is evaluated using real images captured in different conditions. The feature descriptor of this work combines information of spatter trajectory morphology, spatial distributions, and temporal information. The classification is performed using support vector machine (SVM) and random forests for testing and shows highly promising classification accuracy of about 97%. The advantages of this work include compactness for representation and semantic interpretability with the feature description. In addition, the qualities of manufacturing parts are mapped with spattering characteristics under different laser energy densities. Such a map table can be then used to define the desired spatter features, providing a non-contact monitoring solution for online anomaly detection. This work will lead to a further integration of real-time vision monitoring system for an online closed-loop prognostic system for SLM systems, in order to improve the performance in terms of manufacturing quality, power consumption, and fault detection

GUARDIANS final report

Emergencies in industrial warehouses are a major concern for firefghters. The large dimensions together with the development of dense smoke that drastically reduces visibility, represent major challenges. The Guardians robot swarm is designed to assist fire fighters in searching a large warehouse. In this report we discuss the technology developed for a swarm of robots searching and assisting fire fighters. We explain the swarming algorithms which provide the functionality by which the robots react to and follow humans while no communication is required. Next we discuss the wireless communication system, which is a so-called mobile ad-hoc network. The communication network provides also one of the means to locate the robots and humans. Thus the robot swarm is able to locate itself and provide guidance information to the humans. Together with the re ghters we explored how the robot swarm should feed information back to the human fire fighter. We have designed and experimented with interfaces for presenting swarm based information to human beings

Cognitive privacy middleware for deep learning mashup in environmental IoT

Data mashup is a Web technology that combines information from multiple sources into a single Web application. Mashup applications support new services, such as environmental monitoring. The different organizations utilize data mashup services to merge data sets from the different Internet of Multimedia Things (IoMT) context-based services in order to leverage the performance of their data analytics. However, mashup, different data sets from multiple sources, is a privacy hazard as it might reveal citizens specific behaviors in different regions. In this paper, we present our efforts to build a cognitive-based middleware for private data mashup (CMPM) to serve a centralized environmental monitoring service. The proposed middleware is equipped with concealment mechanisms to preserve the privacy of the merged data sets from multiple IoMT networks involved in the mashup application. In addition, we presented an IoT-enabled data mashup service, where the multimedia data are collected from the various IoMT platforms, and then fed into an environmental deep learning service in order to detect interesting patterns in hazardous areas. The viable features within each region were extracted using a multiresolution wavelet transform, and then fed into a discriminative classifier to extract various patterns. We also provide a scenario for IoMT-enabled data mashup service and experimentation results

Digital Image Processing

This book presents several recent advances that are related or fall under the umbrella of 'digital image processing', with the purpose of providing an insight into the possibilities offered by digital image processing algorithms in various fields. The presented mathematical algorithms are accompanied by graphical representations and illustrative examples for an enhanced readability. The chapters are written in a manner that allows even a reader with basic experience and knowledge in the digital image processing field to properly understand the presented algorithms. Concurrently, the structure of the information in this book is such that fellow scientists will be able to use it to push the development of the presented subjects even further

Three-dimensional visualisation and quantitative characterisation of fossil fuel flames using tomography and digital imaging techniques

This thesis describes the design, implementation and experimental evaluation of a prototype instrumentation system for the three-dimensional (3-D) visualisation and quantitative characterisation of fossil fuel flames. A review of methodologies and technologies for the 3-D visualisation and characterisation of combustion flames is given, together with a discussion of main difficulties and technical requirements in their applications. A strategy incorporating optical sensing, digital image processing and tomographic reconstruction techniques is proposed. The strategy was directed towards the reconstruction of 3-D models of a flame and the subsequent quantification of its 3-D geometric, luminous and fluid dynamic parameters. Based on this strategy, a flame imaging system employing three identical synchronised RG B cameras has been developed. The three cameras, placed equidistantly and equiangular on a semicircle around the flame, captured six simultaneous images of the flame from six different directions. Dedicated computing algorithms, based on image processing and tomographic reconstruction techniques have been developed to reconstruct the 3-D models of a flame. A set of geometric, luminous and fluid dynamic parameters, including surface area, volume, length, circularity, luminosity and temperature are determined from the 3-D models generated. Systematic design and experimental evaluation of the system on a gas-fired combustion rig are reported. The accuracy, resolution and validation of the system were also evaluated using purpose-designed templates including a high precision laboratory ruler, a colour flat panel and a tungsten lamp. The results obtained from the experimental evaluation are presented and the relationship between the measured parameters and the corresponding operational conditions are quantified. Preliminary investigations were conducted on a coal-fired industry-scale combustion test facility. The multi-camera system was reconfigured to use only one camera due to the restrictions at the site facility. Therefore the property of rotational symmetry of the flame had to be assumed. Under such limited conditions, the imaging system proved to provide a good reconstruction of the internal structures and luminosity variations inside the This thesis describes the design, implementation and experimental evaluation of a prototype instrumentation system for the three-dimensional (3-D) visualisation and quantitative characterisation of fossil fuel flames. A review of methodologies and technologies for the 3-D visualisation and characterisation of combustion flames is given, together with a discussion of main difficulties and technical requirements in their applications. A strategy incorporating optical sensing, digital image processing and tomographic reconstruction techniques is proposed. The strategy was directed towards the reconstruction of 3-D models of a flame and the subsequent quantification of its 3-D geometric, luminous and fluid dynamic parameters. Based on this strategy, a flame imaging system employing three identical synchronised RG B cameras has been developed. The three cameras, placed equidistantly and equiangular on a semicircle around the flame, captured six simultaneous images of the flame from six different directions. Dedicated computing algorithms, based on image processing and tomographic reconstruction techniques have been developed to reconstruct the 3-D models of a flame. A set of geometric, luminous and fluid dynamic parameters, including surface area, volume, length, circularity, luminosity and temperature are determined from the 3-D models generated. Systematic design and experimental evaluation of the system on a gas-fired combustion rig are reported. The accuracy, resolution and validation of the system were also evaluated using purpose-designed templates including a high precision laboratory ruler, a colour flat panel and a tungsten lamp. The results obtained from the experimental evaluation are presented and the relationship between the measured parameters and the corresponding operational conditions are quantified. Preliminary investigations were conducted on a coal-fired industry-scale combustion test facility. The multi-camera system was reconfigured to use only one camera due to the restrictions at the site facility. Therefore the property of rotational symmetry of the flame had to be assumed. Under such limited conditions, the imaging system proved to provide a good reconstruction of the internal structures and luminosity variations inside the This thesis describes the design, implementation and experimental evaluation of a prototype instrumentation system for the three-dimensional (3-D) visualisation and quantitative characterisation of fossil fuel flames. A review of methodologies and technologies for the 3-D visualisation and characterisation of combustion flames is given, together with a discussion of main difficulties and technical requirements in their applications. A strategy incorporating optical sensing, digital image processing and tomographic reconstruction techniques is proposed. The strategy was directed towards the reconstruction of 3-D models of a flame and the subsequent quantification of its 3-D geometric, luminous and fluid dynamic parameters. Based on this strategy, a flame imaging system employing three identical synchronised RG B cameras has been developed. The three cameras, placed equidistantly and equiangular on a semicircle around the flame, captured six simultaneous images of the flame from six different directions. Dedicated computing algorithms, based on image processing and tomographic reconstruction techniques have been developed to reconstruct the 3-D models of a flame. A set of geometric, luminous and fluid dynamic parameters, including surface area, volume, length, circularity, luminosity and temperature are determined from the 3-D models generated. Systematic design and experimental evaluation of the system on a gas-fired combustion rig are reported. The accuracy, resolution and validation of the system were also evaluated using purpose-designed templates including a high precision laboratory ruler, a colour flat panel and a tungsten lamp. The results obtained from the experimental evaluation are presented and the relationship between the measured parameters and the corresponding operational conditions are quantified. Preliminary investigations were conducted on a coal-fired industry-scale combustion test facility. The multi-camera system was reconfigured to use only one camera due to the restrictions at the site facility. Therefore the property of rotational symmetry of the flame had to be assumed. Under such limited conditions, the imaging system proved to provide a good reconstruction of the internal structures and luminosity variations inside the flame. Suggestions for future development of the technology are also reported

Percepción basada en visión estereoscópica, planificación de trayectorias y estrategias de navegación para exploración robótica autónoma

Tesis inédita de la Universidad Complutense de Madrid, Facultad de Informática, Departamento de Ingeniería del Software e Inteligencia artificial, leída el 13-05-2015En esta tesis se trata el desarrollo de una estrategia de navegación autónoma basada en visión artificial para exploración robótica autónoma de superficies planetarias. Se han desarrollado una serie de subsistemas, módulos y software específicos para la investigación desarrollada en este trabajo, ya que la mayoría de las herramientas existentes para este dominio son propiedad de agencias espaciales nacionales, no accesibles a la comunidad científica. Se ha diseñado una arquitectura software modular multi-capa con varios niveles jerárquicos para albergar el conjunto de algoritmos que implementan la estrategia de navegación autónoma y garantizar la portabilidad del software, su reutilización e independencia del hardware. Se incluye también el diseño de un entorno de trabajo destinado a dar soporte al desarrollo de las estrategias de navegación. Éste se basa parcialmente en herramientas de código abierto al alcance de cualquier investigador o institución, con las necesarias adaptaciones y extensiones, e incluye capacidades de simulación 3D, modelos de vehículos robóticos, sensores, y entornos operacionales, emulando superficies planetarias como Marte, para el análisis y validación a nivel funcional de las estrategias de navegación desarrolladas. Este entorno también ofrece capacidades de depuración y monitorización.La presente tesis se compone de dos partes principales. En la primera se aborda el diseño y desarrollo de las capacidades de autonomía de alto nivel de un rover, centrándose en la navegación autónoma, con el soporte de las capacidades de simulación y monitorización del entorno de trabajo previo. Se han llevado a cabo un conjunto de experimentos de campo, con un robot y hardware real, detallándose resultados, tiempo de procesamiento de algoritmos, así como el comportamiento y rendimiento del sistema en general. Como resultado, se ha identificado al sistema de percepción como un componente crucial dentro de la estrategia de navegación y, por tanto, el foco principal de potenciales optimizaciones y mejoras del sistema. Como consecuencia, en la segunda parte de este trabajo, se afronta el problema de la correspondencia en imágenes estéreo y reconstrucción 3D de entornos naturales no estructurados. Se han analizado una serie de algoritmos de correspondencia, procesos de imagen y filtros. Generalmente se asume que las intensidades de puntos correspondientes en imágenes del mismo par estéreo es la misma. Sin embargo, se ha comprobado que esta suposición es a menudo falsa, a pesar de que ambas se adquieren con un sistema de visión compuesto de dos cámaras idénticas. En consecuencia, se propone un sistema experto para la corrección automática de intensidades en pares de imágenes estéreo y reconstrucción 3D del entorno basado en procesos de imagen no aplicados hasta ahora en el campo de la visión estéreo. Éstos son el filtrado homomórfico y la correspondencia de histogramas, que han sido diseñados para corregir intensidades coordinadamente, ajustando una imagen en función de la otra. Los resultados se han podido optimizar adicionalmente gracias al diseño de un proceso de agrupación basado en el principio de continuidad espacial para eliminar falsos positivos y correspondencias erróneas. Se han estudiado los efectos de la aplicación de dichos filtros, en etapas previas y posteriores al proceso de correspondencia, con eficiencia verificada favorablemente. Su aplicación ha permitido la obtención de un mayor número de correspondencias válidas en comparación con los resultados obtenidos sin la aplicación de los mismos, consiguiendo mejoras significativas en los mapas de disparidad y, por lo tanto, en los procesos globales de percepción y reconstrucción 3D.Depto. de Ingeniería de Software e Inteligencia Artificial (ISIA)Fac. de InformáticaTRUEunpu

NIR Imagery-based Grass Fire Detection and Metrics Measurement using Small UAS

This thesis focuses on the generation of a new grass fire aerial image dataset and development of novel methods for near-infrared (NIR) imagery-based fire front identification and fire depth estimation using small unmanned aircraft systems (sUAS). The procedure for collection and creation of the Grass Fire Front and near-Infrared (NIR) and Thermal Imagery (GRAFFITI) dataset is introduced first including two levels of data: synced raw thermal and red, green and near-infrared (RGNIR) image pairs and processed image pairs of the same overlapping field-of-view. A novel NIR imagery-based fire detection and fire front identification algorithm is then proposed and validated against manually labeled ground truth, using the GRAFFITI dataset. A comparative study is further performed on the problem of grass fire front location and flame depth estimation using thermal and NIR imagery. Finally, recommendations are made to future researchers who are interested in wildland fire sensing using thermal or NIR imagery

UAV-Enabled Surface and Subsurface Characterization for Post-Earthquake Geotechnical Reconnaissance

Major earthquakes continue to cause significant damage to infrastructure systems and the loss of life (e.g. 2016 Kaikoura, New Zealand; 2016 Muisne, Ecuador; 2015 Gorkha, Nepal). Following an earthquake, costly human-led reconnaissance studies are conducted to document structural or geotechnical damage and to collect perishable field data. Such efforts are faced with many daunting challenges including safety, resource limitations, and inaccessibility of sites. Unmanned Aerial Vehicles (UAV) represent a transformative tool for mitigating the effects of these challenges and generating spatially distributed and overall higher quality data compared to current manual approaches. UAVs enable multi-sensor data collection and offer a computational decision-making platform that could significantly influence post-earthquake reconnaissance approaches. As demonstrated in this research, UAVs can be used to document earthquake-affected geosystems by creating 3D geometric models of target sites, generate 2D and 3D imagery outputs to perform geomechanical assessments of exposed rock masses, and characterize subsurface field conditions using techniques such as in situ seismic surface wave testing. UAV-camera systems were used to collect images of geotechnical sites to model their 3D geometry using Structure-from-Motion (SfM). Key examples of lessons learned from applying UAV-based SfM to reconnaissance of earthquake-affected sites are presented. The results of 3D modeling and the input imagery were used to assess the mechanical properties of landslides and rock masses. An automatic and semi-automatic 2D fracture detection method was developed and integrated with a 3D, SfM, imaging framework. A UAV was then integrated with seismic surface wave testing to estimate the shear wave velocity of the subsurface materials, which is a critical input parameter in seismic response of geosystems. The UAV was outfitted with a payload release system to autonomously deliver an impulsive seismic source to the ground surface for multichannel analysis of surface waves (MASW) tests. The UAV was found to offer a mobile but higher-energy source than conventional seismic surface wave techniques and is the foundational component for developing the framework for fully-autonomous in situ shear wave velocity profiling.PHDCivil EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/145793/1/wwgreen_1.pd

Estimation par stéréovision multimodale de caractéristiques géométriques d'un feu de végétation en propagation

Les travaux menés dans cette thèse concernent le développement d’un dispositif de vision permettant l’estimation de caractéristiques géométriques d’un feu de végétation en propagation. Ce dispositif est composé de plusieurs systèmes de stéréovision multimodaux générant des paires d’images stéréoscopiques à partir desquelles des points tridimensionnels sont calculés et les caractéristiques géométriques de feu tels que sa position, vitesse, hauteur, profondeur, inclinaison, surface et volume sont estimées. La première contribution importante de cette thèse est la détection de pixels de feu de végétation. Tous les algorithmes de détection de pixels de feu de la littérature ainsi que ceux développés dans le cadre de cette thèse ont été évalués sur une base de 500 images de feux de végétation acquises dans le domaine du visible et caractérisées en fonction des propriétés du feu dans l’image (couleur, fumée, luminosité). Cinq algorithmes de détection de pixels de feu de végétation basés sur la fusion de données issues d’images acquises dans le domaine du visible et du proche-infrarouge ont également été développés et évalués sur une autre base de données composée de 100 images multimodales caractérisées. La deuxième contribution importante de cette thèse concerne l’utilisation de méthodes de fusion d’images pour l’optimisation des points appariés entre les images multimodales stéréoscopiques. La troisième contribution importante de cette thèse est l’estimation des caractéristiques géométriques de feu à partir de points tridimensionnels obtenus depuis plusieurs paires d’images stéréoscopiques et recalés à l’aide de relevés GPS et d’inclinaison de tous les dispositifs de vision. Le dispositif d’estimation de caractéristiques géométriques à partir de systèmes de stéréovision a été évalué sur des objets rigides de dimensions connues et a permis d’obtenir les informations souhaitées avec une bonne précision. Les résultats des données obtenues pour des feux de végétation en propagation sont aussi présentés. Mots clefs : Feux de forêt, stéréovision, traitement d’images, segmentation, multimodal.This thesis presents the geometrical characteristics measurement of spreading vegetation fires with multimodal stereovision systems. Image processing and 3D registration are used in order to obtain a three-dimensional modeling of the fire at each instant of image acquisition and then to compute fire front characteristics like its position, its rate of spread, its height, its width, its inclination, its surface and its volume. The first important contribution of this thesis is the fire pixel detection. A benchmark of fire pixel detection algorithms of the litterature and of those that are developed in this thesis have been on a database of 500 vegetation fire images of the visible spectra which have been characterized according to the fire properties in the image (color, smoke, luminosity). Five fire pixel detection algorithms based on fusion of data from visible and near-infrared spectra images have also been developed and tested on another database of 100 multimodal images. The second important contribution of this thesis is about the use of images fusion for the optimization of the matching point’s number between the multimodal stereo images. The second important contribution of this thesis is the registration method of 3D fire points obtained with stereovision systems. It uses information collected from a housing containing a GPS and an IMU card which is positioned on each stereovision systems. With this registration, a method have been developed to extract the geometrical characteristics when the fire is spreading. The geometrical characteristics estimation device have been evaluated on a car of known dimensions and the results obtained confirm the good accuracy of the device. The results obtained from vegetation fires are also presented. Key words: wildland fire, stereovision, image processing segmentation, multimodal