Harnessing Big Data and Machine Learning for Event Detection and Localization

Anomalous events are rare and significantly deviate from expected pattern and other data instances, making them hard to predict. Correctly and timely detecting anomalous severe events can help reduce risks and losses. Many anomalous event detection techniques are studied in the literature. Recently, big data and machine learning based techniques have shown a remarkable success in a wide range of fields. It is important to tailor big data and machine learning based techniques for each application; otherwise it may result in expensive computation, slow prediction, false alarms, and improper prediction granularity.First, we aim to address the above challenges by harnessing big data and machine learning techniques for fast and reliable prediction and localization of severe events. Firstly, to improve storage failure prediction, we develop a new lightweight and high performing tensor decomposition-based method, named SEFEE, for storage error forecasting in large-scale enterprise storage systems. SEFEE employs tensor decomposition technique to capture latent spatio-temporal information embedded in storage event logs. By utilizing the latent spatio-temporal information, we can make accurate storage error forecasting without training requirements of typical machine learning techniques. The training-free method allows for live prediction of storage errors and their locations in the storage system based on previous observations that had been used in tensor decomposition pipeline to extract meaningful latent correlations. Moreover, we propose an extension to include severity of the errors as contextual information to improve the accuracy of tensor decomposition which in turn improves the prediction accuracy. We further provide detailed characterization of NetApp dataset to provide additional insight into the dynamics of typical large-scale enterprise storage systems for the community.Next, we focus on another application -- AI-driven Wildfire prediction. Wildfires cause billions of dollars in property damages and loss of lives, with harmful health threats. We aim to correctly detect and localize wildfire events in the early stage and also classify wildfire smoke based on perceived pixel density of camera images. Due to the lack of publicly available dataset for early wildfire smoke detection, we first collect and process images from the AlertWildfire camera network. The images are annotated with bounding boxes and densities for deep learning methods to use. We then adapt a transformer-based end-to-end object detection model for wildfire detection using our dataset. The dataset and detection model together form as a benchmark named the Nevada smoke detection benchmark, or Nemo for short. Nemo is the first open-source benchmark for wildfire smoke detection with the focus of the early incipient stage. We further provide a weakly supervised Nemo version to enable wider support as a benchmark

Multi-teacher knowledge distillation as an effective method for compressing ensembles of neural networks

Deep learning has contributed greatly to many successes in artificial intelligence in recent years. Today, it is possible to train models that have thousands of layers and hundreds of billions of parameters. Large-scale deep models have achieved great success, but the enormous computational complexity and gigantic storage requirements make it extremely difficult to implement them in real-time applications. On the other hand, the size of the dataset is still a real problem in many domains. Data are often missing, too expensive, or impossible to obtain for other reasons. Ensemble learning is partially a solution to the problem of small datasets and overfitting. However, ensemble learning in its basic version is associated with a linear increase in computational complexity. We analyzed the impact of the ensemble decision-fusion mechanism and checked various methods of sharing the decisions including voting algorithms. We used the modified knowledge distillation framework as a decision-fusion mechanism which allows in addition compressing of the entire ensemble model into a weight space of a single model. We showed that knowledge distillation can aggregate knowledge from multiple teachers in only one student model and, with the same computational complexity, obtain a better-performing model compared to a model trained in the standard manner. We have developed our own method for mimicking the responses of all teachers at the same time, simultaneously. We tested these solutions on several benchmark datasets. In the end, we presented a wide application use of the efficient multi-teacher knowledge distillation framework. In the first example, we used knowledge distillation to develop models that could automate corrosion detection on aircraft fuselage. The second example describes detection of smoke on observation cameras in order to counteract wildfires in forests.Comment: Doctoral dissertation in the field of computer science, machine learning. Application of knowledge distillation as aggregation of ensemble models. Along with several uses. 140 pages, 67 figures, 13 table

A review of machine learning applications in wildfire science and management

Artificial intelligence has been applied in wildfire science and management since the 1990s, with early applications including neural networks and expert systems. Since then the field has rapidly progressed congruently with the wide adoption of machine learning (ML) in the environmental sciences. Here, we present a scoping review of ML in wildfire science and management. Our objective is to improve awareness of ML among wildfire scientists and managers, as well as illustrate the challenging range of problems in wildfire science available to data scientists. We first present an overview of popular ML approaches used in wildfire science to date, and then review their use in wildfire science within six problem domains: 1) fuels characterization, fire detection, and mapping; 2) fire weather and climate change; 3) fire occurrence, susceptibility, and risk; 4) fire behavior prediction; 5) fire effects; and 6) fire management. We also discuss the advantages and limitations of various ML approaches and identify opportunities for future advances in wildfire science and management within a data science context. We identified 298 relevant publications, where the most frequently used ML methods included random forests, MaxEnt, artificial neural networks, decision trees, support vector machines, and genetic algorithms. There exists opportunities to apply more current ML methods (e.g., deep learning and agent based learning) in wildfire science. However, despite the ability of ML models to learn on their own, expertise in wildfire science is necessary to ensure realistic modelling of fire processes across multiple scales, while the complexity of some ML methods requires sophisticated knowledge for their application. Finally, we stress that the wildfire research and management community plays an active role in providing relevant, high quality data for use by practitioners of ML methods.Comment: 83 pages, 4 figures, 3 table

Deep Learning Methods for Remote Sensing

Remote sensing is a field where important physical characteristics of an area are exacted using emitted radiation generally captured by satellite cameras, sensors onboard aerial vehicles, etc. Captured data help researchers develop solutions to sense and detect various characteristics such as forest fires, flooding, changes in urban areas, crop diseases, soil moisture, etc. The recent impressive progress in artificial intelligence (AI) and deep learning has sparked innovations in technologies, algorithms, and approaches and led to results that were unachievable until recently in multiple areas, among them remote sensing. This book consists of sixteen peer-reviewed papers covering new advances in the use of AI for remote sensing

Applicability of Multispectral Imagery for Detection of Prescribed Fires and Rekindling

Forest fires are an increasingly relevant problem nowadays with the worsening of global warming’s most severe consequences. These fire occurrences, that can cause immense damage to forest ecosystems and have a great negative impact in peoples lives,begin often with rekindles. These problems can be very difficult to tackle, needing to involve a lot of people to surveil the areas at risk. A system capable of executing this surveillance protocol and alerting the fire fighting authorities of fire and possible rekindle occurrences would be extremely beneficial in these scenarios.A system aiming to achieve this goal is being implemented, composed of an UAV equipped with a multispectral camera, capturing aerial images of these areas. This dissertation presents a fire detection model to be used in prescribed fires and rekindling situations, identifying fire instances within the captured images. It makes use of the camera’s various spectral bands to highlight the areas at greatest risk and of deep learning technology to autonomously recognise these areas.Incêndios florestais são um problema cada vez mais relevante nos dias de hoje com o agravamento das consequências mais graves do aquecimento global. Estas ocorrências,que podem causar imensos danos aos ecossistemas florestais e ter um grande impacto negativo na vida das pessoas, são muitas vezes iniciadas por reacendimentos. Estes problemas podem ser muito difíceis de combater, necessitando de envolver muitas pessoas para vigiar as áreas de risco. Um sistema capaz de executar este protocolo de vigilância e alertar as autoridades de combate a incêndio sobre fogos e possíveis reacendimentos seria extremamente benéfico nestes cenários.Para alcançar este objetivo, está a ser implementado um sistema composto por um UAV, equipado com uma câmera multiespectral, que irá capturar imagens aéreas dessas áreas. Esta dissertação apresenta um modelo de detecção de incêndios para ser utilizado em situações de fogos controlados e reacendimentos, identificando ocorrências de fogo nas imagens capturadas. Faz uso das várias bandas espectrais da câmera para destacar as áreas de maior risco e de tecnologia de aprendizagem automática para reconhecer essas áreas de forma autônoma

Location-aware Adaptive Normalization: A Deep Learning Approach For Wildfire Danger Forecasting

Climate change is expected to intensify and increase extreme events in the weather cycle. Since this has a significant impact on various sectors of our life, recent works are concerned with identifying and predicting such extreme events from Earth observations. With respect to wildfire danger forecasting, previous deep learning approaches duplicate static variables along the time dimension and neglect the intrinsic differences between static and dynamic variables. Furthermore, most existing multi-branch architectures lose the interconnections between the branches during the feature learning stage. To address these issues, this paper proposes a 2D/3D two-branch convolutional neural network (CNN) with a Location-aware Adaptive Normalization layer (LOAN). Using LOAN as a building block, we can modulate the dynamic features conditional on their geographical locations. Thus, our approach considers feature properties as a unified yet compound 2D/3D model. Besides, we propose using the sinusoidal-based encoding of the day of the year to provide the model with explicit temporal information about the target day within the year. Our experimental results show a better performance of our approach than other baselines on the challenging FireCube dataset. The results show that location-aware adaptive feature normalization is a promising technique to learn the relation between dynamic variables and their geographic locations, which is highly relevant for areas where remote sensing data builds the basis for analysis. The source code is available at https://github.com/HakamShams/LOAN

Building a Fused Deposition Modelling (FDM) 3D Printing Visual Defect Detection System, Part 1: Creation of a Dynamic Imaging System, a Pixel-wise Segmentation Dataset with a Hybrid Synthetic Data Creation Method, and a Semantic Segmentation Algorithm with the SegFormer Deep Learning Model

Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced Analytics, specialization in Business AnalyticsAs a part of an effort to develop a surface defect detection system for FDM 3D printed objects, this work project studies the application of the SegFormer network to semantically segment 3D printed objects. The project also showcases an affordable and accessible imaging system designed for the surface defect detection system, to support the decisions made during the segmentation task and to be used to evaluate the segmentation models. To achieve this, the first-ever pixel-wise annotation dataset of 3D-printed object images was created. Model-O1, a SegFormer MiT-B0 model trained on this dataset with minimal data augmentation resulted in an Intersection-over-Union score of 87.04%. A synthetic data creation method that caters to the nature of 3D printed objects was also proposed, which expands upon existing synthetic data creation methods. The model trained on this dataset, Model-A2, achieved an IoU score of 89.31%, the best performance achieved among the models developed in this project. During the evaluation of the model based on the inference results, Model-A2 was also identified to be the most practical model for building a surface defect detection system