CCTV is an important part of the surveillance systems. While a person observes the screens with the naked eye in existing CCTV monitoring systems, it is possible that the flame and events are not detected opportunely due to eye fatigue and human carelessness. In this paper, we propose a real-time flame detection method and design architecture of surveillance system to achieve early fire detection. When flame is detected, this system displays the video sequence from the corresponding camera channel on the whole screen with the alarm signal

ChangYong Ri

HakSong Ri

JinHyok Ju

KyongChol Rim

ICTACT Journal on Image and Video Processing

Directory of Open Access Journals

ISSN: 0976-9102 (ONLINE)                                                                                           ICTACT JOURNAL ON IMAGE AND VIDEO PROCESSING, AUGUST 2018, VOLUME: 09, ISSUE: 01 DOI: 10.21917/ijivp.2018.0262 1857 AN IMPROVED SURVEILLANCE SYSTEM ARCHITECTURE BASED ON FLAME DETECTION IN CCTVS  ChangYong Ri, JinHyok Ju, HakSong Ri and KyongChol Rim Institute of Information Science, Kim Il Sung University, Pyongyang, D.P.R. of Korea   Abstract CCTV is an important part of the surveillance systems. While a person observes the screens with the naked eye in existing CCTV monitoring systems, it is possible that the flame and events are not detected opportunely due to eye fatigue and human carelessness. In this paper, we propose a real-time flame detection method and design architecture of surveillance system to achieve early fire detection. When flame is detected, this system displays the video sequence from the corresponding camera channel on the whole screen with the alarm signal.  Keywords: Surveillance System, Flame Detection, CCTV 1. INTRODUCTION To develop an intelligent CCTV (closed-circuit television) surveillance system is a challenging task. Now, most CCTV surveillance systems are used in real-time monitoring and storing of the video sequences from cameras. They simply assist to investigate the case by playing back the stored videos after the events were occurred. However, this cannot prevent disasters in advance. Many researchers have studied for development of surveillance system based on flame and stork detection. In Ref [1] proposed fire flame and smoke detection architecture in video sequence. They used HSI (hue, saturation and intensity) color model for flame, and RGB (red, green and blue) for smoke detection. And they used the image accumulation to extract the motion attributes of flame and smoke regions, and used the optical flow method to recognize smoke region. However, flame and smoke colors depend on lighting condition, so the accuracy of fire detection by employing simple color model may be insufficient. In Ref [2] proposed the forest fire detection method based on pixel classification. They used RGB and YCbCr (luma, blue-difference and red-difference of chroma) color models. Using these two color models, they presented seven simple rules for separating the pixels of an image. Based on these rules, every pixel of an image accordingly classified to two classes, i.e. flame and non-flame. However, they used simple color models to detect the fire. In Ref [3], an adaptive median filter was used as preprocessing of image, and a directional color difference was calculated using an RGB color model. Then the edges of flame were obtained by applying a threshold value. By combining the spatio-temporal information of the images using these digital image processing techniques, they carried out the flame detection. However, in their study, since the flame region was obtained only by using the edge information of the flame, the accuracy of fire detection in complicated indoor environment could not be satisfied. In Ref [4] proposed fire detection and alarm system architecture. They used the frame differences to analyze and detect the fire flame in video sequences. They proposed six conditional expressions for dividing the flame region by using four color models such as HSV (hue, saturation, value), YCbCr, RGB and CIE L*a*b*. This is similar to the method presented in Ref [2]. Moreover, they used frame differences to distinguish fire from the spurious fire such as candle flame, match flame, gas range flame and other objects which are similar to fire. In Ref [5], various fire detection methods based on image processing techniques were compared. Under the comparison, they proposed a fire detection algorithm by using chromaticity based background subtraction, wavelet based energy analysis and edge disordered analysis. The method presented in their study has used to develop the real-time forest monitoring system. In Ref [6] proposed the fire detection method in surveillance devices such as CCTV, wireless camera. They used RGB color model to separate the fire candidate pixels in image, and used Sobel edge extraction to detect ground-truth fire regions. In Ref [7] proposed an algorithm for early detection of flames using IP Cameras. For the purpose of the smoke region detection, they used the color, motion and growth properties in video sequences obtained from IP camera. Recently, deep learning techniques have been widely applied to the field of computer vision, and some researchers have used CNN (convolutional neural network) to detect flame and smoke. In Ref [8] the faster R-CNN (regional CNN) was used to detect forest fire, and in ref [9] the CNN was used to early detect fire in varying indoor and outdoor environments for CCTV surveillance cameras. In Ref [10], the multi-channel CNN was used to fire recognition. These fire detection methodologies employed the deep learning techniques. However, deep learning requires the big training data and high performance training machine. Compared to deep learning based methodologies, traditional image processing based method is simple and does not require huge dataset, so it can be implemented simply and we introduce it to our surveillance system. It automatically detects fire and events in advance and informs the watcher, so it can prevent disasters in advance. Fire causes considerable losses of life and property. However, early detection of flam can prevent accident of fire and reduce its losses. Therefore, real-time flame detection for prevention of fire in CCTV monitoring system is important. In order to improve the efficiency of real-time fire detection, in this work we integrate the color based rule, motion based rule and boundary energy based rule in wavelet space. So we propose real-time flame detection method and surveillance system architecture on CCTVs using proposed method. 2. FLAME DETECTION METHOD In the case of fire detection, it is important to develop an efficient algorithm that automatically detects the flame in time by CHANGYONG RI et al.: AN IMPROVED SURVEILLANCE SYSTEM ARCHITECTURE BASED ON FLAME DETECTION IN CCTVS 1858 separating the dynamic and static elements in the video sequences to reduce the false alarm rate. Here, the dynamic elements refer to the motion and diffusion phenomenon of the flame in the event of fire, that is, the spatial and frequency dynamic characteristics of the flame. Static elements are sunlight reflections and artificial structures which are similar to flame in color. In this work, three flame detection methods are integrated to detect the flame region in time accurately. First, the color features are used to detect the initial candidate regions of flame. Then, we measure the flame motion characteristics, that is, the temporal changes of the candidate flame regions. And then, by analyzing the dynamic frequency characteristics from the wavelet transform in color space, we determine the final flame regions. Rule 1: Convert the RGB color space into the L*a*b* color space which is more similar to human visual perception:  2.7689 1.7518 1.13021.0000 4.5907 0.06010.0000 0.0565 5.5943X RY GZ B                          (1)  1/30 0*1/30 0116 16, 0.008856903.3 , 0.008856Y YfY YLY YfY Y             (2) *0 0500X Ya f fX Y               *0 0200Y Zb f fY Z               where,  1/3 0.008856( ) 7.787 16, 0.008856116t tf t tt    (3) The following rule (color based rule) is applied to detect initial candidate flame region in an image:  * * * * * *{( , ) | ( , ) , ( , ) , ( , ) }x y L x y L a x y a b x y b     (4) Rule 2: It is not enough to detect fire with color alone. Since there are many object regions that show the similar colors to flames but not. Therefore, we examine the motion of candidate flame regions to increase the accuracy of the flame detection. In order to reduce the calculation cost and to detect the corresponding flame regions, we apply motion detection only to the regions resulted by applying rule 1. The motion of the flame pixels in video sequence frames is detected by the following rule 2.  1{( , ) | ( , ) ( , ) }i ix y I x y I x y th    (5) Rule 3: Moreover, we consider the boundary of flame to be irregular. That is, the flame regions have no fixed boundary, and the boundary shapes are very disordered. From this, we measure the wavelet transform based energy of candidate flame regions to determine the final flame regions. In this step, we apply the energy based rule to the candidate flame regions that resulted by applying Rule 1 and Rule 2. For the two dimensional image, wavelet transform is performed in both the horizontal and vertical directions. Then the four sub-bands of LL (low-low), HL (high-low), LH (low-high) and HH (high-high) are obtained. Using the high frequency components, we obtain the wavelet transform based energy as follows.        2 2 21 , , ,LH HL HHE I k l I k l I k lM N   (6) where, I is the wavelet energy value, M and N is the horizontal and vertical size of motion region. Synthetically, when the rules 1 to 3 are satisfied, it is determined that there is a fire in the video sequence frames. Then the current frame is fully monitored and the alarm is activated. The Fig.1 shows the flow chart of flame detection process.  Fig.1. Flowchart of the flame detection process 3. IMPROVED SURVEILLANCE SYSTEM ARCHITECTURE In general, it is almost impossible for a person to monitor the 20 screens of CCTVs in real time and to prevent the disasters due to physical constraints. Our goal is to build the CCTV monitoring system based on fire detection using artificial intelligence techniques, so that it is capable of responding to disasters such as fire in advance. Currently, the smoke detection sensors are used to prevent the fire. We design an integrated intelligent surveillance system by embedding the flame detection module in the CCTV monitoring system without excluding the existing smoke detection sensor. This system is composed by adding the surveillance server to the existing CCTV media server of data center. The surveillance system consists of detectors and surveillance servers. Detectors take the appropriate frames (about 3-5 FPS) in video sequences and detect the flame. Then, detectors send the detected flame information to the surveillance server. Of course, detectors include the object detector and the face detector in addition to Video Sequence Color Transformation Color based Rule Motion based Rule Energy based Rule Fire Region Result based Rule Yes based Rule Yes based Rule Yes based Rule No based Rule No based Rule No based Rule ISSN: 0976-9102 (ONLINE)                                                                                           ICTACT JOURNAL ON IMAGE AND VIDEO PROCESSING, AUGUST 2018, VOLUME: 09, ISSUE: 01 1859 flame detector. Here, we only consider the flame detector proposed in this work. Surveillance server keeps surveillance events received from detectors and sends them to monitoring system. At this time, the priority of the CCTV channels to be monitored is determined, and a warning alarm is sounded if the situation is urgent. The Fig.2 shows the flow chart of the surveillance server integrated CCTV monitoring system. The video stream captured by the camera is transmitted to the media server. Then it is recorded on media storage, the other side it is displayed through the monitor. At this time, three frames of video sequences per second are extracted and transmitted to the flame detection engine. This engine detects the candidate fire regions employing the flame and smoke detection algorithms. Then the result is transmitted to the surveillance server. This surveillance server confirms the surveillance event data and alarm with monitor if the fire is detected. At this time, surveillance server also refers to the information of other fire detection sensors (thermal sensors and smoke sensors, etc.).  Fig.2. Flow chart of the surveillance server integrated CCTV monitoring system 4. RESULTS AND DISCUSSION In order to verify the effectiveness of the flame detection method proposed in this work, we carry out the related experiments. We use the video data of 4891 frames (15fps, 24fps and 30fps) which size is 480×320 pixels or 320×240 pixels. The videos are obtained from internet. (http://signal.ee.bilkent.edu.tr/visifire, https://zenodo.org/record/). Table.1. List of the video information used in the experiment Video Total frame Fire frame Explanation Movie 1 153 153 accident in highway Movie 2 220 220 forest fire Movie 3 752 650 outdoor fire pot Movie 4 352 352 nocturnal fire Movie 5 246 246 burning firewood Movie 6 403 0 non fire Movie 7 510 510 man behind fire Movie 8 368 368 burning tree Movie 9 213 213 fire in garden Movie 10 190 190 forest fire Movie 11 120 0 moving truck Movie 12 165 0 walking men on ground Movie 13 412 0 walking men in hallway Movie 14 394 0 dancing man Movie 15 393 393 accident in tunnel The Table.1 shows the video information used in the experiments. The videos include not only fire frames, but also non-fire objects to check the false fire alarms. Movie 11, Movie 12, Movie 13, and Movie 14 are the fire-colored moving people and truck. For performance evaluation, we use the number of true-detected fire frames (TP) and the number of false-detected fire frames (FP) there is no fire. In order to compare the flame detection accuracy of proposed method and previous methods, we use the following characteristics:  100 TPFDfire frames  (7)  100-  FPNFDnon fire frames  (8) The Table.2 shows the comparison of flame detection results of the proposed method with the method of Agrawal et al [4]. The experimental results show that the proposed method has better flame detection performance than the related method. Comparing with previous work, the average flame detection accuracy (FD) of the proposed method was improved to 4.9%. And the error detection rate (NFD) was fallen to 0.8%. This is because of integration the color based rule, motion based rule and boundary energy based rule in wavelet space in proposed flame detection method. Table 2. Comparison of flame detection results between proposed method and related method Video Fire frame FD (%) NFD (%) Proposed Agrawal et al. Proposed Agrawal et al. Movie 1 153 83.0 62.7 - - Movie 2 220 100 100 - - Movie 3 650 83.1 63.8 4.9 6.3 Movie 4 352 100 99.4 - - Movie 5 246 99.6 100 - - Movie 6 0 - - 7.9 8.5 Movie 7 510 91.3 90.7 - - Movie 8 368 97.5 89.2 - - Movie 9 213 100 100 - - Movie 10 190 98.6 100 - - Movie 11 0 - - 9.4 7.6 Movie 12 0 - - 6.7 5.9 Camera Media   Server Surveillance Server Flame   Detector Monitor Other Sensors If Event Media Storage Surveillance Events 3 FPS 30 FPS No Yes CHANGYONG RI et al.: AN IMPROVED SURVEILLANCE SYSTEM ARCHITECTURE BASED ON FLAME DETECTION IN CCTVS 1860 Movie 13 0 - - 3.4 4.6 Movie 14 0 - - 4.8 9.3 Movie 15 393 92.8 91.6 - - average  94.6 89.7 6.2 7.0 Depending on the architecture of the intelligence CCTV monitoring system suggested in this work, we build a real-time CCTV monitoring system consisting of 20 cameras. In this system, the number of frames per second, which is used to fire detection, set to 3. This system stores the results of fire detection to use them as a training data in the future. 5. CONCLUSION In this work, we proposed the rule based fire detection method and designed the intelligent CCTV surveillance system that integrates surveillance server. Recently, there are many object recognition methods based on deep learning techniques, which achieve high object recognition accuracy. The flame detection accuracy can also be improved by using deep learning techniques. However, using the deep learning techniques, a huge amount of training data is a key issue. In the future, we will study the flame detection methods by using deep learning techniques and apply it to the intelligence CCTV monitoring systems. REFERENCES [1] Chunyu Yu, Zhibin Mei and Xi Zhang, “A Real-Time Video Fire Flame and Smoke Detection Algorithm”, Procedia Engineering, Vol. 62, pp. 891-898, 2013. [2] V. Vipin, “Image Processing Based Forest Fire Detection”, International Journal of Emerging Technology and Advanced Engineering, Vol. 2, No. 2, pp. 87-95, 2012. [3] A.D. Patil, K.R. Kuwar, M.S. More, P.A. Pagar and S.D. Somawanshi, “Advance Algorithm for Fire Detection using Image Processing and Color Recognition”, Proceedings of International Conference on Wireless Communication and Android Apps, pp. 9-14, 2015. [4] R.A. Agrawal and S.T. Khandare, “Fire Detection Using Image Processing”, International Journal of Advanced Engineering and Global Technology, Vol. 3, No. 2, pp. 1499-1503, 2015. [5] S. Verstockt, B. Merci, P. Lambert, R. Van De Walle and B. Sette, “State of the Art in Vision-based Fire and Smoke Detection”, Proceedings of 14th International Conference on Automatic Fire Detection, pp. 285-292, 2009. [6] Kumarguru Poobalan and Siau-Chuin Liew, “Fire Detection Algorithm using Image Processing Techniques”, Proceedings of 3rd International Conference on Artificial Intelligence and Computer Science, pp. 160-168, 2015.  [7] Leonardo Millan-Garcia, Gabriel Sanchez-Perez, Mariko Nakano, Karina Toscano-Medina, Hector Perez-Meana and Luis Rojas-Cardenas, “An Early Fire Detection Algorithm Using IP Cameras”, Sensors, Vol. 12, pp. 5670-5686, 2012. [8] Qi-xing Zhang, Gao-hua Lin, Yong-ming Zhang, Gao Xu and Jin-jun Wang, “Wildland Forest Fire Smoke Detection Based on Faster R-CNN using Synthetic Smoke Images”, Procedia Engineering, Vol. 211, pp. 441-446, 2018. [9] Khan Muhammad, Jamil Ahmad and Sung Wook Baik, “Early Fire Detection using Convolutional Neural Networks during Surveillance for Effective Disaster Management”, Neurocomputing, Vol. 288, pp. 30-42, 2018. [10] Wentao Mao, Wenpeng Wang, Zhi Dou and Yuan Li, “Fire Recognition Based On Multi-Channel Convolutional Neural Network”, Fire Technology, Vol. 54, No. 2, pp. 531-554, 2018. 

AN IMPROVED SURVEILLANCE SYSTEM ARCHITECTURE BASED ON FLAME DETECTION IN CCTVS

http://ictactjournals.in/paper/IJIVP_Vol9_Iss1_Paper10_1857_1860.pdf

AN IMPROVED SURVEILLANCE SYSTEM ARCHITECTURE BASED ON FLAME DETECTION IN CCTVS

Abstract

Similar works

Full text

Available Versions

Directory of Open Access Journals