Project RISE: Recognizing Industrial Smoke Emissions

Industrial smoke emissions pose a significant concern to human health. Prior works have shown that using Computer Vision (CV) techniques to identify smoke as visual evidence can influence the attitude of regulators and empower citizens to pursue environmental justice. However, existing datasets are not of sufficient quality nor quantity to train the robust CV models needed to support air quality advocacy. We introduce RISE, the first large-scale video dataset for Recognizing Industrial Smoke Emissions. We adopted a citizen science approach to collaborate with local community members to annotate whether a video clip has smoke emissions. Our dataset contains 12,567 clips from 19 distinct views from cameras that monitored three industrial facilities. These daytime clips span 30 days over two years, including all four seasons. We ran experiments using deep neural networks to establish a strong performance baseline and reveal smoke recognition challenges. Our survey study discussed community feedback, and our data analysis displayed opportunities for integrating citizen scientists and crowd workers into the application of Artificial Intelligence for social good.Comment: Technical repor

Técnicas de aprendizaje profundo aplicadas a la detección de entornos con humo/niebla

Ante el surgimiento en los últimos años de modelos cada vez más complejos de Deep Learning, se analizará la utilidad de estos ante un problema real. Este se trata la necesidad de detectar humo en centrales termosolares de canales parabólicos para evitar la contaminación que pueda llegar a causar los fluidos empleados en ellas. Para ello, se analizarán dos modelos de redes neuronales desarrollados recientemente y se comprobará su validez ante un dataset específico extraído de cámaras de ese tipo de centrales.Due to the development in recent years of complex Deep Learning models, the purpose of this thesis is to analyze their effectiveness in the context of a real problem. It is focused on the need to detect smoke in parabolic channel solar thermal power plants in order to avoid the contamination that may be caused by the fluids used in them. To do this, two recently developed neural network models will be analyzed, and their effectiveness will be tested against a specific dataset extracted from cameras of this type of power plant.Universidad de Sevilla. Grado en Ingeniería Electrónica, Robótica y Mecatrónica

Building a Fused Deposition Modelling (FDM) 3D Printing Visual Defect Detection System, Part 1: Creation of a Dynamic Imaging System, a Pixel-wise Segmentation Dataset with a Hybrid Synthetic Data Creation Method, and a Semantic Segmentation Algorithm with the SegFormer Deep Learning Model

Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced Analytics, specialization in Business AnalyticsAs a part of an effort to develop a surface defect detection system for FDM 3D printed objects, this work project studies the application of the SegFormer network to semantically segment 3D printed objects. The project also showcases an affordable and accessible imaging system designed for the surface defect detection system, to support the decisions made during the segmentation task and to be used to evaluate the segmentation models. To achieve this, the first-ever pixel-wise annotation dataset of 3D-printed object images was created. Model-O1, a SegFormer MiT-B0 model trained on this dataset with minimal data augmentation resulted in an Intersection-over-Union score of 87.04%. A synthetic data creation method that caters to the nature of 3D printed objects was also proposed, which expands upon existing synthetic data creation methods. The model trained on this dataset, Model-A2, achieved an IoU score of 89.31%, the best performance achieved among the models developed in this project. During the evaluation of the model based on the inference results, Model-A2 was also identified to be the most practical model for building a surface defect detection system

딥러닝을 이용한 화재 감지 및 화재 이미지 시맨틱 분할

학위논문 (박사) -- 서울대학교 대학원 : 자연과학대학 협동과정 계산과학전공, 2020. 8. 강명주.최근 딥 러닝은 다양한 분야에서 가장 중요하고 강력한 주제이다. 딥러닝은 이미지 분류에서 뛰어난 성능을 보였으며, 이후 컴퓨터 비전의 이지미에서 객체 감지 및 시맨틱 분할에도 적용되었다. 본 논문에서는 뛰어난 성능을 가진 딥 러닝을 사용하여 화재 이미지 감지 및 분할 작업에 적합한 네트워크를 제안한다. 또한 딥러닝 압축 기법을 사용하여 화재 이미지 분할 딥러닝 모델에 적용하여 소규모 네트워크를 제안하였고 이를 임베디드 장치에 적용했다. 여러가지 광범위한 실험을 통해 화재감지와 화재 이미지 분할에서 기존의 기법보다 좋다는 점을 보였다.Recently, deep learning has become the most important and powerful topic in various research fields. It has shown excellent performance in image classification and has been applied to the fields of object detection and semantic image segmentation of computer vision. In this thesis, we proposed deep neural networks suitable for fire image detection and segmentation tasks with excellent performance. In addition, we proposed a small-sized network for fire image segmentation based on squeezed deep-learning techniques and applied it to an embedded device. Several extensive experiments are presented to demonstrate its better performance compared with the existing methods for fire detection and image segmentation.1 Introduction 1 2 Preliminaries 4 2.1 Image classification 4 2.2 Object detection 9 2.3 Semantic image segmentation 12 2.4 Compressed deep learning 15 3 Fire Detection and Localization 17 3.1 Related work 17 3.2 Proposed method 19 3.3 Experiments 21 3.3.1 Fire area localization 28 3.3.2 Fire localization results 30 3.4 Conclusion 32 4 Semantic Segmentation using Deep Learning for Fire Images 34 4.1 Related work 34 4.2 Proposed architecture 38 4.2.1 Comparison with FusionNet 40 4.3 Experimental results 41 4.3.1 Experimental results using FiSmo dataset 42 4.3.2 Experimental results using Corsican Fire Database 45 4.4 Conclusion 48 5 Squeezed Semantic Segmentation for Fire Images 49 5.1 Related work 49 5.2 Squeezed Fire Binary Segmentation Networks(SFBSNet) 50 5.2.1 SFBSNet architecture 50 5.2.2 Implementation details 54 5.3 Experiments 56 5.3.1 Ablation study 58 5.3.2 Experiments on FiSmo dataset 58 5.3.3 Experiments on Corsican Fire Database 60 5.3.4 Additional experiments on Still dataset 60 5.4 Conclusion 62 6 Conclusion and Future Works 64 Abstract (in Korean) 73Docto

Visual localization in challenging environments

Visual localization, the method of self-localization based on camera images, has established as an additional, GNSS-free technology that is investigated in increasingly real and challenging applications. Particularly demanding is the self-localization of first responders in unstructured and unknown environments, for which visual localization can substantially contribute to increase the situational awareness and safety of first responders. Challenges arise from the operation under adverse conditions on computationally restricted platforms in the presence of dynamic objects. Current solutions are quickly pushed to their limits and the development of more robust approaches is of high demand. This thesis investigates the application of visual localization in dynamic, adverse environments to identify challenges and accordingly to increase the robustness, on the example of a dedicated visual-inertial navigation system. The methodical contributions of this work relate to the introduction of semantic understanding, improvements in error propagation and the development of a digital twin. The geometric visual odometry component is extended to a hybrid approach that includes a deep neural network for semantic segmentation to ignore distracting image areas of certain object classes. A Sensor-AI approach complements this method by directly training the network to segment image areas that are critical for the considered visual odometry system. Another improvement results from analyses and modifications of the existing error propagation in visual odometry. Furthermore, a digital twin is presented that closely replicates geometric and radiometric properties of the real sensor system in simulation in order to multiply experimental possibilities. The experiments are based on datasets from inspections that are used to motivate three first responder scenarios, namely indoor rescue, flood disaster and wildfire. The datasets were recorded in corridor, mall, coast, river and fumarole environments and aim to analyze the influence of the dynamic elements person, water and smoke. Each investigation starts with extensive in-depth analyses in simulation based on created synthetic video clones of the respective dynamic environments. Specifically, a combined sensitivity analysis allows to jointly consider environment, system design, sensor property and calibration error parameters to account for adverse conditions. All investigations are verified with experiments based on the real system. The results show the susceptibility of geometric approaches to dynamic objects in challenging scenarios. The introduction of the segmentation aid within the hybrid system contributes well in terms of robustness by preventing significant failures, but understandably it cannot compensate for a lack of visible static backgrounds. As a consequence, future visual localization systems require both the ability of semantic understanding and its integration into a complementary multi-sensor system