A framework based on Gaussian mixture models and Kalman filters for the segmentation and tracking of anomalous events in shipboard video

Anomalous indications in monitoring equipment on board U.S. Navy vessels must be handled in a timely manner to prevent catastrophic system failure. The development of sensor data analysis techniques to assist a ship\u27s crew in monitoring machinery and summon required ship-to-shore assistance is of considerable benefit to the Navy. In addition, the Navy has a large interest in the development of distance support technology in its ongoing efforts to reduce manning on ships. In this thesis, algorithms have been developed for the detection of anomalous events that can be identified from the analysis of monochromatic stationary ship surveillance video streams. The specific anomalies that we have focused on are the presence and growth of smoke and fire events inside the frames of the video stream. The algorithm consists of the following steps. First, a foreground segmentation algorithm based on adaptive Gaussian mixture models is employed to detect the presence of motion in a scene. The algorithm is adapted to emphasize gray-level characteristics related to smoke and fire events in the frame. Next, shape discriminant features in the foreground are enhanced using morphological operations. Following this step, the anomalous indication is tracked between frames using Kalman filtering. Finally, gray level shape and motion features corresponding to the anomaly are subjected to principal component analysis and classified using a multilayer perceptron neural network. The algorithm is exercised on 68 video streams that include the presence of anomalous events (such as fire and smoke) and benign/nuisance events (such as humans walking the field of view). Initial results show that the algorithm is successful in detecting anomalies in video streams, and is suitable for application in shipboard environments

EARLY FOREST FIRE DETECTION USING TEXTURE, BLOB THRESHOLD, AND MOTION ANALYSIS OF PRINCIPAL COMPONENTS

Forest fires constantly threaten ecological systems, infrastructure and human lives. The purpose behind this study is minimizing the devastating damage caused by forest fires. Since it is impossible to completely avoid their occurrences, it is essential to accomplish a fast and appropriate intervention to minimize their destructive consequences. The most traditional method for detecting forest fires is human based surveillance through lookout towers. However, this study presents a more modern technique. It utilizes land-based real-time multispectral video processing to identify and determine the possibility of fire occurring within the camera’s field of view. The temporal, spectral, and spatial signatures of the fire are exploited. The methods discussed include: (1) Range filtering followed by entropy filtering of the infrared (IR) video data, and (2) Principal Component Analysis of visible spectrum video data followed by motion analysis and adaptive intensity threshold. The two schemes presented are tailored to detect the fire core, and the smoke plume, respectively. Cooled Midwave Infrared (IR) camera is used to capture the heat distribution within the field of view. The fire core is then isolated using texture analysis techniques: first, range filtering applied on two consecutive IR frames, and then followed by entropy filtering of their absolute difference. Since smoke represents the earliest sign of fire, this study also explores multiple techniques for detecting smoke plumes in a given scene. The spatial and temporal variance of smoke plume is captured using temporal Principal Component Analysis, PCA. The results show that a smoke plume is readily segmented via PCA applied on the visible Blue band over 2 seconds sampled every 0.2 seconds. The smoke plume exists in the 2nd principal component, and is finally identified, segmented, and isolated, using either motion analysis or adaptive intensity threshold. Experimental results, obtained in this study, show that the proposed system can detect smoke effectively at a distance of approximately 832 meters with a low false-alarm rate and short reaction time. Applied, such system would achieve early forest fire detection minimizing fire damage. Keywords: Image Processing, Principal Component Analysis, PCA, Principal Component, PC, Texture Analysis, Motion Analysis, Multispectral, Visible, Cooled Midwave Infrared, Smoke Signature, Gaussian Mixture Model

Dynamic texture analysis in video with application to flame, smoke and volatile organic compound vapor detection

Ankara : The Department of Electrical and Electronics Engineering and the Institute of Engineering and Science of Bilkent University, 2009.Thesis (Master's) -- Bilkent University, 2009.Includes bibliographical references leaves 74-82.Dynamic textures are moving image sequences that exhibit stationary characteristics in time such as fire, smoke, volatile organic compound (VOC) plumes, waves, etc. Most surveillance applications already have motion detection and recognition capability, but dynamic texture detection algorithms are not integral part of these applications. In this thesis, image processing based algorithms for detection of specific dynamic textures are developed. Our methods can be developed in practical surveillance applications to detect VOC leaks, fire and smoke. The method developed for VOC emission detection in infrared videos uses a change detection algorithm to find the rising VOC plume. The rising characteristic of the plume is detected using a hidden Markov model (HMM). The dark regions that are formed on the leaking equipment are found using a background subtraction algorithm. Another method is developed based on an active learning algorithm that is used to detect wild fires at night and close range flames. The active learning algorithm is based on the Least-Mean-Square (LMS) method. Decisions from the sub-algorithms, each of which characterize a certain property of the texture to be detected, are combined using the LMS algorithm to reach a final decision. Another image processing method is developed to detect fire and smoke from moving camera video sequences. The global motion of the camera is compensated by finding an affine transformation between the frames using optical flow and RANSAC. Three frame change detection methods with motion compensation are used for fire detection with a moving camera. A background subtraction algorithm with global motion estimation is developed for smoke detection.Günay, OsmanM.S

Smoke plume segmentation of wildfire images

Aquest treball s'emmarca dins del camp d'estudi de les xarxes neuronals en Aprenentatge profund. L'objectiu del projecte és analitzar i aplicar les xarxes neuronals que hi ha avui dia en el mercat per resoldre un problema en específic. Aquest és tracta de la segmentació de plomalls de fum en incendis forestals. S'ha desenvolupat un estudi de les xarxes neuronals utilitzades per resoldre problemes de segmentació d'imatges i també una reconstrucció posterior en 3D d'aquests plomalls de fum. L'algorisme finalment escollit és tracta del model UNet, una xarxa neuronal convolucional basada en l'estructura d'autoencoders amb connexions de pas, que desenvolupa tasques d'autoaprenentatge per finalment obtenir una predicció de la classe a segmentar entrenada, en aquest cas plomalls. de fum. Posteriorment, una comparativa entre algoritmes tradicionals i el model UNet aplicat fent servir aprenentatge profund s'ha realitzat, veient que tant quantitativament com qualitativament s'aconsegueix els millors resultats aplicant el model UNet, però a la vegada comporta més temps de computació. Tots aquests models s'han desenvolupat amb el llenguatge de programació Python utilitzant els llibres d'aprenentatge automàtic Tensorflow i Keras. Dins del model UNet s'han dut a terme múltiples experiments per obtenir els diferents valors dels hiperparàmetres més adequats per a l'aplicació del projecte, obtenint una precisió del 93.45 % en el model final per a la segmentació de fum en imatges d'incendis. forestals.Este trabajo se enmarca dentro del campo de estudio de las redes neuronales en aprendizaje profundo. El objetivo del proyecto es analizar y aplicar las redes neuronales que existen hoy en día en el mercado para resolver un problema en específico. Éste se trata de la segmentación de penachos de humo en incendios forestales. Se ha desarrollado un estudio de las redes neuronales utilizadas para resolver problemas de segmentación de imágenes y también una reconstrucción posterior en 3D de estos penachos de humo. El algoritmo finalmente escogido se trata del modelo UNet, una red neuronal convolucional basada en la estructura de autoencoders con conexiones de paso, que desarrolla tareas de autoaprendizaje para finalmente obtener una predicción de la clase a segmentar entrenada, en este caso penachos de humo. Posteriormente, una comparativa entre algoritmos tradicionales y el modelo UNet aplicado utilizando aprendizaje profundo se ha realizado, viendo que tanto cuantitativa como cualitativamente se consigue los mejores resultados aplicando el modelo UNet, pero a la vez conlleva más tiempo de computación. Todos estos modelos se han desarrollado con el lenguaje de programación Python utilizando libros de aprendizaje automático Tensorflow y Keras. Dentro del modelo UNet se han llevado a cabo múltiples experimentos para obtener los distintos valores de los hiperparámetros más adecuados para la aplicación del proyecto, obteniendo una precisión del 93.45 % en el modelo final para la segmentación de humo en imágenes de incendios forestales.This work is framed within the field of study of neural networks in Deep Learning. The aim of the project is to analyse and apply the neural networks that exist today in the market to solve a specific problem. This is about the segmentation of smoke plumes in forest fires. A study of the neural networks used to solve image segmentation problems and also a subsequent 3D reconstruction of these smoke plumes has been developed. The algorithm finally chosen is the UNet model, a convolutional neural network based on the structure of autoencoders with step connections, which develops self-learning tasks to finally obtain a prediction of the class to be trained, in this case smoke plumes. Also, a comparison between traditional algorithms and the UNet model applied using deep learning has been carried out, seeing that both quantitatively and qualitatively the best results are achieved by applying the UNet model, but at the same time it involves more computing time. All these models have been developed in the Python programming language using the Tensorflow and Keras machine learning books. Within the UNet model, multiple experiments have been carried out to obtain the different hyperparameter values most suitable for the project application, obtaining an accuracy of 93.45% in the final model for smoke segmentation in wildfire images

Orthorectification of helicopter-borne high resolution experimental burn observation from infra red handheld imagers

To pursue the development and validation of coupled fire-atmosphere models, the wildland fire modeling community needs validation data sets with scenarios where fire-induced winds influence fire front behavior, and with high temporal and spatial resolution. Helicopter-borne infrared thermal cameras have the potential to monitor landscape-scale wildland fires at a high resolution during experimental burns. To extract valuable information from those observations, three-step image processing is required: (a) Orthorectification to warp raw images on a fixed coordinate system grid, (b) segmentation to delineate the fire front location out of the orthorectified images, and (c) computation of fire behavior metrics such as the rate of spread from the time-evolving fire front location. This work is dedicated to the first orthorectification step, and presents a series of algorithms that are designed to process handheld helicopter-borne thermal images collected during savannah experimental burns. The novelty in the approach lies on its recursive design, which does not require the presence of fixed ground control points, hence relaxing the constraint on field of view coverage and helping the acquisition of high-frequency observations. For four burns ranging from four to eight hectares, long-wave and mid infra red images were collected at 1 and 3 Hz, respectively, and orthorectified at a high spatial resolution (<1 m) with an absolute accuracy estimated to be lower than 4 m. Subsequent computation of fire radiative power is discussed with comparison to concurrent space-borne measurementsPeer ReviewedPostprint (published version

Multi-modal video analysis for early fire detection

In dit proefschrift worden verschillende aspecten van een intelligent videogebaseerd branddetectiesysteem onderzocht. In een eerste luik ligt de nadruk op de multimodale verwerking van visuele, infrarood en time-of-flight videobeelden, die de louter visuele detectie verbetert. Om de verwerkingskost zo minimaal mogelijk te houden, met het oog op real-time detectie, is er voor elk van het type sensoren een set ’low-cost’ brandkarakteristieken geselecteerd die vuur en vlammen uniek beschrijven. Door het samenvoegen van de verschillende typen informatie kunnen het aantal gemiste detecties en valse alarmen worden gereduceerd, wat resulteert in een significante verbetering van videogebaseerde branddetectie. Om de multimodale detectieresultaten te kunnen combineren, dienen de multimodale beelden wel geregistreerd (~gealigneerd) te zijn. Het tweede luik van dit proefschrift focust zich hoofdzakelijk op dit samenvoegen van multimodale data en behandelt een nieuwe silhouet gebaseerde registratiemethode. In het derde en tevens laatste luik van dit proefschrift worden methodes voorgesteld om videogebaseerde brandanalyse, en in een latere fase ook brandmodellering, uit te voeren. Elk van de voorgestelde technieken voor multimodale detectie en multi-view lokalisatie zijn uitvoerig getest in de praktijk. Zo werden onder andere succesvolle testen uitgevoerd voor de vroegtijdige detectie van wagenbranden in ondergrondse parkeergarages

SMART ADDITIVE MANUFACTURING: IN-PROCESS SENSING AND DATA ANALYTICS FOR ONLINE DEFECT DETECTION IN METAL ADDITIVE MANUFACTURING PROCESSES

The goal of this dissertation is to detect the incipient flaws in metal parts made using additive manufacturing processes (3D printing). The key idea is to embed sensors inside a 3D printing machine and conclude whether there are defects in the part as it is being built by analyzing the sensor data using artificial intelligence (machine learning). This is an important area of research, because, despite their revolutionary potential, additive manufacturing processes are yet to find wider acceptance in safety-critical industries, such as aerospace and biomedical, given their propensity to form defects. The presence of defects, such as porosity, can afflict as much as 20% of additive manufactured parts. This poor process consistency necessitates an approach wherein flaws are not only detected but also promptly corrected inside the machine. This dissertation takes the critical step in addressing the first of the above, i.e., detection of flaws using in-process sensor signatures. Accordingly, the objective of this work is to develop and apply a new class of machine learning algorithms motivated from the domain of spectral graph theory to analyze the in-process sensor data, and subsequently, detect the formation of part defects. Defects in additive manufacturing originate due to four main reasons, namely, material, process parameters, part design, and machine kinematics. In this work, the efficacy of the graph theoretic approach is determined to detect defects that occur in all the above four contexts. As an example, in Chapter 4, flaws such as lack-of-fusion porosity due to poor choice of process parameters in additive manufacturing are identified with statistical accuracy exceeding 80%. As a comparison, the accuracy of existing conventional statistical methods is less than 65%. Advisor: Prahalada Ra

SMART ADDITIVE MANUFACTURING: IN-PROCESS SENSING AND DATA ANALYTICS FOR ONLINE DEFECT DETECTION IN METAL ADDITIVE MANUFACTURING PROCESSES

The goal of this dissertation is to detect the incipient flaws in metal parts made using additive manufacturing processes (3D printing). The key idea is to embed sensors inside a 3D printing machine and conclude whether there are defects in the part as it is being built by analyzing the sensor data using artificial intelligence (machine learning). This is an important area of research, because, despite their revolutionary potential, additive manufacturing processes are yet to find wider acceptance in safety-critical industries, such as aerospace and biomedical, given their propensity to form defects. The presence of defects, such as porosity, can afflict as much as 20% of additive manufactured parts. This poor process consistency necessitates an approach wherein flaws are not only detected but also promptly corrected inside the machine. This dissertation takes the critical step in addressing the first of the above, i.e., detection of flaws using in-process sensor signatures. Accordingly, the objective of this work is to develop and apply a new class of machine learning algorithms motivated from the domain of spectral graph theory to analyze the in-process sensor data, and subsequently, detect the formation of part defects. Defects in additive manufacturing originate due to four main reasons, namely, material, process parameters, part design, and machine kinematics. In this work, the efficacy of the graph theoretic approach is determined to detect defects that occur in all the above four contexts. As an example, in Chapter 4, flaws such as lack-of-fusion porosity due to poor choice of process parameters in additive manufacturing are identified with statistical accuracy exceeding 80%. As a comparison, the accuracy of existing conventional statistical methods is less than 65%. Advisor: Prahalada Ra