Lambert, Peter

Merci, Bart

Sette, Bart

Van de Walle, Rik

Verstockt, Steven

English

Ghent University Academic Bibliography

State of the art in vision-based fire and smoke detection Steven Verstockt, Peter Lambert, Rik Van de Walle Department of Electronics and Information Systems – Multimedia Lab –Ghent University - IBBT, Ghent, Belgium  Bart Merci Department of Flow Heat and Combustion Mechanics – Ghent University – Fund of Scientific Research (FWO-Vlaanderen), Ghent, Belgium  Bart Sette Warrington Fire Ghent, Ghent, Belgium   Abstract Video processing techniques for automatic fire and smoke detection havebecome a hot topic in computer vision during the last decade. The severalvision-based detection algorithms that have been proposed in literaturehave lead to a large amount of techniques that can be used to detect thepresence of fire at an early stage.  In this paper, we perform a thorough evaluation of the existing techniques and propose an improved smoke detection algorithm. The novel algorithm is based on a chromaticity-based background subtraction method with back-step correction, wavelet-based energy analysis, and boundary disorder analysis. These steps are chosen because they are mainlyindependent of the burning material and the fire circumstances. Experiments with a large number of fire and non-fire video sequences show that our method is fast and capable of accurately detecting smoke. Introduction Fire is one of the leading hazards affecting everyday life around the world. To avoid large scale fire and smoke damage, timely and accurate fire detection is essential. The majority of fire detection systems used today is based on particle sampling, temperature sampling, and air transparency testing. Unfortunately, these systems are generally limited to indoors, require a close proximity to the fire and cannot provide additional information about fire circumstances such as size, location, and propagation. A further drawback of those conventional detectors is the transport and threshold delay, i.e., the time for particles to reach and to activate a detector. To provide more reliable and faster information, visual-based approaches are becoming more and more interesting. Detecting fire from video images is a relatively new research subject. Current research shows that the vision-based detection of smoke and flames promises fast detection and can be a viable alternative or complement for the more traditional techniques. One of the major differences between video fire detection (VFD) and the conventional heat and smoke detection technologies is the potential of VFD to be useful in conditions in which conventional methods cannot be used, such as detection at a distance in large and open spaces. VFD does also not have the delay traditional sensors suffer from. As soon as smoke or flames occur in one of the camera views, fire can be detected. Further, with the increase of CCTV and the progress on video surveillance, VFD can be incorporated in existing surveillance systems at relatively low cost.  Depending on whether VFD focuses on flames or smoke characteristics, a distinction can be made between video image flame detection (VIFD) and video image smoke detection (VISD). Both are based on the use and analysis of color, motion, energy, and disorder information in video. Where in the beginning mainly flame detection was investigated, recently there is a tendency towards smoke detection. Since in most cases smoke occurs much faster in the field of view of the cameras, focusing on smoke offers a faster detection. More detailed information of existing VIFD and VISD systems and their pro and contras is given in Section 2.  Due to the variability of shape, motion, transparency, colors, and patterns of smoke and flames, many of the existing VFD approaches are still vulnerable to false alarms. The block-based video smoke detection algorithm proposed in this paper reduces many of these false-alarm problems by starting from a novel chromaticity-based background subtraction with back-step correction, which is fully described in Section 3. Next, Section 4 briefly presents the wavelet-based energy analysis and boundary disorder analysis, i.e., the smoke discriminating steps in our algorithm. Since these steps do not significantly differ from the existing work, their description is limited but references to the relevant literature are given. Section 5 ends this paper with conclusions and future work.  State of the art in video fire detection The number of papers about fire detection in the computer vision literature is rather limited. As is, this relatively new subject in vision research has still a long way to go. Nevertheless, the results from existing work already seem very promising. An overview of the state-of-the-art, i.e., a collection of frequently referenced papers, is presented in Table 1. For each of these papers we investigated the underlying algorithms and checked the appropriate techniques. In the following, we will discuss each of these detection techniques and analyze their use in the listed papers.  Table 1. State of the art VFD techniques Color detection was one of the first detection techniques used in VFD and is still by far the most popular. The majority of the color-based approaches in VFD makes use of RGB color space, sometimes in combination with HSI/HSV saturation [2,3]. The main reason for using RGB is the equality in RGB values of smoke pixels and the easily distinguishable red-yellow range of flames. The major rule-based techniques used to detect the fire-colored pixels are Gaussian-smoothed color histograms [1], statistically generated color models [6], and blending functions [17]. The test results of color-based fire detection in the referenced work seems promising at first, but the variability in color, density, lighting, and background do raise questions about its applicability in real world detection systems. A far more interesting color-based fire detection mechanism seems the detection of chrominance decrease [13], which is used in our novel background subtraction and is further discussed in the next sections. Moving object detection is the second technique that is frequently used as a first step in VFD to eliminate the disturbance of stationary non-smoke objects. In order to detect possible motion, which may be caused from fire, the moving part in the current video frame is detected by means of a motion segmentation algorithm. To determine if the motion is due to smoke or an ordinary moving object, further analysis of moving regions is necessary. The most effective algorithms to perform moving object detection are background subtraction [4,5,7,13-15,17], temporal differencing [16], and optical flow [10,18].  Other frequently used fire detection techniques are flicker detection [3,7,8,14] and energy analysis [5,13,17]. Both focus on the temporal behaviour of flames and smoke. Flickering is the temporal periodicity with which pixels appear and disappear at the edges of turbulent flames. For turbulent flames it has been shown experimentally that the flicker frequency is around 10Hz and that it is not greatly affected by the burning material and the burner [4,7]. For smoke however, the flicker frequency is time-varying. As such, smoke flicker detection does not seem a very reliable technique. More interesting for detecting the temporal behaviour of smoke is energy analysis, as is further described in Section 4.  Fire also has the unique characteristic that it does not remain a steady color, i.e., the flames are composed of several varying colors within a small area. Spatial difference analysis [3,14] focuses on this characteristic. Using range filters [3] or spatial wavelet analysis [14] the spatial color variations in pixel values are analyzed to eliminate ordinary fire-colored objects with a solid flame color. This technique works well for flame detection, but is not always applicable for smoke detection. Nevertheless, in very light regions, where chromaticity-based background subtraction fails, it is proven to be an effective alternative.  Also interesting for smoke detection is the disorder analysis of smoke regions over time. Some examples of frequently used metrics are randomness of area size [11], boundary roughness [13], and turbulence variance [7]. Although those metrics differ in definition, the outcome of each of them is almost identical. In our work, turbulence variance is used, which is determined by relating the perimeter of the region to the square root of the area, as will be further covered in Section 4.  Although not directly related to fire characteristics, sub-blocking, training, and clean-up post-processing are three modules which are commonly used in VFD systems to simplify and improve the detection process. Sub-blocking [5,9,17] reduces measurement disturbances, i.e., filters out errors and measurements inaccuracies. Input images are subdivided in ‘n x n’ size blocks, mostly 16x16 pixels, and a block value is computed as the average of all the pixel values in the block. Thereafter, further analysis is performed on block level instead of on pixel level. Training is used to create background [5,15] and fire color models [1,15], which augment moving object detection and color-based fire detection. The creation of a background model of normal state is very effective for removal of fire-like objects, like lights, and detection of motion-sensitive regions, like windows. But also for energy analysis it is very useful, since a background model of normal edges simplifies the edge degradation analysis. As such, it is used in our approach. Clean-up post-processing, like median filtering, is mostly used as a final step to remove outliers and to group neighboring elements. Based on the analysis of the related work and on our own experiments, we propose a novel smoke detection algorithm which consists of 5 steps: (i) sub-blocking (ii) background subtraction, (iii) energy analysis, (iv) boundary disorder analysis, and (v) clean-up post-processing. In the following we present in more detail the novel background subtraction. Chromaticity-based background subtraction Since smoke regions gradually come into the scene, standard background estimation methods have difficulties to detect them. As such, smoke at an initial state is frequently faulty classified as background, making the background model unusable for further detection. For this reason, a novel background estimation method is created that is able to cope with the gradual characteristic of smoke.  A moving (smoke) block is determined by comparing the YUV chrominance values of the block b in the current frame Fn with the values of the corresponding block in the background model BGn. If for more than half of the pixels [i,j] in the block b, the difference Difn[i,j] of the absolute values of the chrominance values exceeds the chrominance decrease threshold tc, the block is labeled as foreground (Eq.1).  For blocks with very low chrominance values, e.g., blocks containing very light sky regions, chrominance difference analysis will always label the block as background. To overcome this problem, blocks with an average absolute chrominance value below 0.01 are further investigated by analyzing their spatial luminance difference. When the standard deviation σ of the spatial luminance difference FLn(b) of the current block is tl times higher than the one of the corresponding estimated background block BGLn(b), the block is also labeled foreground. Both thresholds, tc and tl, are dependent on the chrominance values of the block in the background model and their value is equal to the function values of two trained chrominance-based exponential functions.   ܦ݅ ௡݂ሾ݅, ݆ሿ ൌ  ቄ 1    ݂݅ ሺ|ܤܩ௡ሾ݅, ݆ሿ| െ |ܨ௡ሾ݅, ݆ሿ|ሻ ൐  ݐ௖ 0    ݋ݐ݄݁ݎݓ݅ݏ݁                                           ܨ௡ሺܾሻ ՜  ەۖ۔ۖۓ   ݂݋ݎ݁݃ݎ݋ݑ݊݀         ݂݅ ෍ ܦ݅ ௡݂ሾ݅, ݆ሿ  ൐  ܾ݈݋ܿ݇ݏ݅ݖ݁2   ሾ௜,௝ሿ ఢ ௕                   ሺ1ሻ                   ݂݅  |ܨ௡ሺܾሻ|തതതതതതതതത ൏ 0,01 ܣܰܦ ߪ൫ܨ௡௅ሺܾሻ൯ߪ൫ܤܩ௡௅ሺܾሻ൯  ൐ ݐ௟    ܾܽܿ݇݃ݎ݋ݑ݊݀         ݋ݐ݄݁ݎݓ݅ݏ݁                                                                       As soon as the current frame is labeled, a new background estimation BGn+1 is made (Eq.2), which is partly based on the method used by Toreyin et al. [14]. To overcome problems with gradually appearing smoke, a buffer of previous background estimations is kept in memory and is compared to the current block values of (Eq.1). If the values of the buffered background blocks exceed those of the current background block, the block Fn(b) is relabeled as back-step and the background estimation BGn+1(b) reuses the estimation of BGn-step(b). In this way, smoke detection results become much better, as is illustrated in Fig. 2.     Since so far the background estimation only uses the smoke detection results of the chromaticity analysis, non-smoke moving objects can become part of the background. For this reason, standard background estimation is used in conjunction with the chromaticity-based estimation when updating the background. In this way, presence of non-smoke moving objects in the background disappears.    Fig. 2. Influence of back-step on smoke detection results  Sequence from the ISYSS project - http://www.ibbt.be/nl/project/isyss - used with permission Energy and boundary disorder analysis Based on the results of the moving smoke detection, foreground blocks are grouped into connected regions, i.e., blobs. For each of the blobs the algorithm uses energy and boundary disorder analysis to filter out faulty classified foreground blocks of smoke-like objects and to make a final decision as whether the blob in frame Fn contains smoke or not.  The energy analysis is quasi identical to the one that is used in the work of Calderara and Piccini [5,17]. Using the discrete wavelet transform, the energy of the blob intensity values is compared with the energy of the intensity value of the blob region in the background estimated frame and the background buffer. If the blob energy drops and the energy variation exceeds the energy variation threshold tE, the blob is labeled as candidate smoke. Then, for each candidate smoke blob, the boundary disorder of the blob is analyzed over time using the turbulence metric, which is also used in the work of Xiong et al. [7]. This metric is determined by relating the perimeter of the candidate blob to the square root of the area of this blob. If the boundary variation between the blob, the blob region in the background estimated frame and the buffer exceeds the disorder threshold tD, the blob is labeled as smoke. Conclusions and future work  In this paper, a thorough evaluation of the state-of-the-art in video fire detection is presented and a novel chromaticity-based smoke detection algorithm with back-step correction is proposed. Starting from sub-blocked (16x16) video frames, background subtraction, energy analysis, and boundary disorder analysis are used to make a clear distinction between smoke and non-smoke regions.  Our work has been tested with different smoke and non-smoke CIF sequences at 25 fps and the results show that the algorithm can detect the presence of smoke in most cases in quasi real-time. Moreover, false alarms, one of the major problems of many other video fire detection techniques, are drastically reduced. A thorough description and evaluation of our algorithm is not covered in this paper. However, this is subject of our current work and will be reported soon. The proposed smoke detection algorithm is able to correctly generate fire alarms, but crucial information about location and propagation of the fire is still missing. This information is of great importance to get a better understanding of the fire. Some work [9] already concentrates on this matter, but the results of these approaches are still limited. As such, detection of smoke localization and smoke propagation is one of the next challenges in our video fire research.  Acknowledgement The research activities as described in this paper were funded by Ghent University, the Interdisciplinary Institute for Broadband Technology (IBBT), University College West Flanders, Warrington Fire Ghent, the Institute for the Promotion of Innovation by Science and Technology in Flanders (IWT), the Fund for Scientific Research-Flanders (FWO-Flanders), the Belgian Federal Science Policy Office (BFSPO), and the European Union. References [1] W. Phillips, M. Shah, N. da Vitoria Lobo, Flame recognition in video, Pattern recognition letters, 2002   [2] T.-H. Chen, P.-H. Wu, and Y.-C. Chiou, An early fire-detection method based on image processing, ICIP, 2004 [3] Jessica Ebert, Jennie Shipley, Computer vision based method for fire detection in color videos, International journal of imaging, 2009 [4] Z. Xu, J. Xu, Automatic fire smoke detection based on image visual features, CIS, 2007 [5] P. Piccinini, S. Calderara, R. Cucchiara, Reliable smoke detection system in the domains of image energy and color, ICIP, 2008 [6] T. Çelik, H. Özkaramanl, H. Demirel, Fire and smoke detection without sensors, EUSIPCO, 2007  [7] Z. Xiong, R. Caballero, H. Wang, A.M. Finn, M. A. Lelic, P.-Y. Peng,  Video-based smoke detection: possibilities, techniques, and challenges, SUPDET, 2007 [8] G. Marbach, M. Loepfe, T. Brupbacher, An image processing technique for fire detection in video images, Fire safety journal, 2006 [9] Rubayiat Yasmin, Detection of smoke propagation direction using color video sequences, IJSC, 2009 [10] F. Gomez-Rodriguez, B.C. Arrue, A. Ollero, Smoke monitoring and measurement using image processing - application to forest fires, Automatic target recognition conference, 2003 [11] P. V. K. Borges, J. Mayer, E. Izquierdo, Efficient visual fire detection applied for video retrieval, EUSIPCO, 2008 [12] C.B. Liu, N. Ahuja, Vision based fire detection, ICPR, 2004 [13] B.U. Toreyin, Y. Dedeoglu, A.E. Cetin, Contour based smoke detection in video using wavelets, EUSIPCO, 2006   [14] B.U. Toreyin, Y. Dedeoglu, U. Güdükbay, A.E. Çetin, Computer vision based method for real-time fire and flame detection, Pattern recognition letters, 2006 [15] T. Celik, H. Demirel, H. Ozkaramanli, M. Uyguroglu, Fire detection using statistical color model in video sequences, JVCIR, 2007 [16]  B. Lee, D.Han, Real-Time fire detection using camera sequence image in tunnel environment, ICIC, 2007 [17] S. Calderara, P. Piccinini, R. Cucchiara, Smoke detection in video surveillance: a MoG model in the wavelet domain, ICVS, 2008 [18] F. Gomez-Rodriguez, S. Pascual-Pena, B. Arrue, A. Ollero, Smoke detection using image processing, Forest fire research, 2002 

State of the art in vision-based fire and smoke dectection

https://biblio.ugent.be/publication/785789/file/785805

State of the art in vision-based fire and smoke dectection

Abstract

Similar works

Full text

Available Versions

Ghent University Academic Bibliography