Kopilovic, I.

Szirányi, Tamás

Vágvölgyi, B.

SZTAKI Publication Repository

Application of panoramic annular lens for motion analysis tasks: surveillance and smoke detection

Kopilovic, I

Vagvolgyi, B

 Authors’ copy Application of Panoramic Annular Lens for Motion Analysis Tasks:  Surveillance and Smoke Detection   Abstract  In this paper some applications of motion analysis are investigated with respects to the compact panoramic optical system (Panoramic Annular Lens). Panoramic image acquisition makes multiple or mechanically controlled camera systems needless for many applications. Panoramic Annular Lens’s main advantage to other omnidirectional monitoring systems is that it is a cheap, small, compact device with no external hyperboloidal surface in a glass sphere as in other panoramic optical devices. Converting the annular image to a rectangular one, we get a low-resolution (cc. 1000x150) image. We have developed some algorithms, which can analyze this low-resolution image to get motion information for surveillance and smoke detection application.  1. Introduction  Surveillance with intelligent camera systems is becoming increasingly important in everyday life. It is widely used in traffic control, driving assistance and surgery. In these applications compactness is also essential. At the same time, it is well known that omnidirectional camera systems have many advantages compared to conventional optics.  Figure 1.Image formation in PAL optics (left), and the PAL (right).  With large field of view we can avoid the use of multiple camera systems or mechanically controlled cameras. This is very useful in applications such as surveillance systems, intelligent vehicle control [2], endoscopic measurements [3], etc. In our work we deal with a special lens called Panoramic Annular Lens (PAL) [1], which has different optical properties compared to other omnidirectional systems (Figure 1, 2). The main advantage of PAL is its small, compact size, sharp image mapping, although its vertical view angle is limited to about 50-70 degrees.    Figure 2. The annular image of the PAL.  2. Motion analysis using the PAL optics  Surveillance systems employing omnidirectional optics have a lot of advantages against traditional systems that use pan-tilt-zoom (PTZ) cameras with conventional optics. Since they have full 360 degrees view, they are able to monitor the whole panorama around the camera and track any number of moving objects simultaneously without moving. These systems can substitute three or even more tracking PTZ cameras with only a single panoramic device. Traditional controllable cameras are complex systems with several moving parts, thus they need heavy maintenance. On the other hand, omnidirectional optics are stationary devices. Furthermore, in case of moving or turning camera additional computations have to be carried out to identify the background in every camera position, and perspective distortion of the conventional optics has to be eliminated. Systems employing static omnidirectional devices do not have to deal with that difficulty. However, panoramic systems have special optical distortions that must be eliminated, and the annular input image must be transformed to a rectangular view before starting image processing. While systems with traditional optics project only a small portion of the panorama on the CCD array of the camera, omnidirectional optics projects the whole panorama on it. Thus, the resolution of objects is lower on the annular image than on the perspective image of the conventional optics. In our system, the PAL was used instead of omnidirectional optics with hyperbolical mirrors, because it is ideal for surveillance purposes in that it is a small and cheap device with smart image mapping.  2.1. Change detection  Our surveillance system uses statistical methods to find the background scene on the streaming video by extracting static regions from every frame and joining them together to the background image dynamically. Thereafter, foreground objects can be easily detected by simply subtracting the computed background from the actual image captured from the camera. A similar algorithm was developed on the University of Maryland [4] for mechanically controlled tracking cameras. In case of PAL, this task can be carried out easier, because the background scene is a still image, so we do not have to deal with the always-changing distortion of the background. In most cases, surveillance systems have to take care of the changing environment, especially when the system is installed outdoors. The method used for adaptation to the changing background is briefly described in Table 1.  Table 1. Adaptation to the changing background.  The variance of brightness changes is computed for every part of the image. Regions with low variance values are considered as background, since low variance indicates change in illumination. See [5] for a similar illumination invariant method for background extraction.  Maintenance of the changing statistics of regions and suppression of the influences of recurrent disturbances (e.g. shaking of the trees). These routines can learn disturbances locally, and ignore them in the detection process (Figure 3 and 4). It is simple and does not need special image processing hardware unlike most recent surveillance systems. We also developed an algorithm to differentiate between moving objects and disturbances by computing motion fields from the optical flow. This way it is possible to work more accurately, but it requires more computation power than power of standard PCs.   Figure 3. Motion detection on noisy input without filtering out disturbances. Checked regions represent detected movement.   Figure 4. Motion detection on noisy input with filtering out disturbances.  2.2. Tracking  The PAL makes tracking much easier in many cases. By the nature of the system it is possible to track any number of intruders simultaneously, without moving the camera. We developed two spatio-temporal segmentation methods to track moving objects in the field of view of the PAL. The first method compares certain properties of found objects (e.g. size, shape, motion direction) on every two succeeding frames to find matching pairs of objects. In this way, real-time tracking (20-30 fps) with all preprocessing is possible even on a low cost PC. The other routine computes motion fields from the optical flow and estimates trajectories of objects (Figure 6). In order to maintain higher frame rate a multiscale correlation algorithm was used with the SIMD instruction set of the Intel Pentium processors (MMX) to compute the optical flow. Although this routine is very fast, in some cases it is only able to process 2-3 frames per second.  2.3. Motion recognition  In intelligent surveillance applications it is important to recognize certain motion patterns or gestures, e.g. someone moves on a certain trajectory or sits down. We developed two kinds of motion recognition algorithms. One of them uses tracking information to find objects having special motion properties; the other computes the history of motion to find certain motion patterns. Finally, an advanced smoke recognition algorithm, detecting the special motion of smoke is presented in details.  2.3.1. Recognition based on tracking information. Using tracking information, several motion recognition tasks can be carried out. Our system uses them to filter out objects having certain properties, like certain size, shape, motion direction, relative velocity and trajectory.  2.3.2. Motion pattern recognition. History of the motion is computed for every moving object with a method similar to [6]. The obtained motion history patterns are trained to a neural network. In the detection phase the neural network is able to recognize trained patterns, like someone sits down or lifts up his hand.  2.3.3. Smoke Motion Detection. The panoramic optics can be used to build up a complex surveillance system, e.g. in large halls. As a building block of this system, a smoke detection algorithm is designed to recognize and alarm situations when smoke appears in the observation field of the camera. Smoke detection is usually performed with chemical or optical sensors, but little is known about methods using digital image processing, although it may have advantages in situation where conventional sensors are less effective. The problem is a special case of motion detection and recognition. Difficulties arise with the discrimination of motion caused by non-smoke events (e.g. filter out human, vehicle or machine motion). Motion analysis literature deals mainly with rigid or elastic motion and relatively few of it treats the so-called fluid motion. The latter articles are mainly concerned with particle image velocimetry, and the application of computational fluid dynamics to image processing [9]. Since numerical algorithms applied in these methods have relatively large computational cost, they are infeasible for real-time applications. Instead, a real-time smoke detection algorithm has been developed based on optical velocity field computation. The method detects irregularities in the optical velocity field due to smoke motion. As a first step, we have established model assumptions. Two distinguished characteristics of smoke motion were taken into account: 1. Mainly upward moving shape 2. Irregularities in motion due to non-rigidity: e.g. the distribution of velocity vector orientations is locally approximately uniform in the interval ]0,]. These assumptions were applied to analyse the optical velocity pattern. There is one fundamental problem regarding the optical flow computation. Video sequence recording smoke propagation may contain large temporal aliasing for frame rates less than about 10 frame/sec. These problems may be partially accounted for by applying multiscale optical flow computation with velocity warping [8]. The considerations above lead to the algorithm shown in Table 2. In the first step, the correlation-based multiscale optical flow algorithm of Anandan et al [8] was simplified to achieve real-time operation. After velocity computation, regions that show upward motion at larger scales are back-projected to regions at the initial scale, and the statistical distribution of velocity orientations is computed for pixel positions falling into them. The statistical evaluation of irregularities in the motion field was performed as follows. In case of n orientations the interval ]0,] is divided into n equal subintervals, and a distribution functions n]1,0[f  is obtained. The distance between distribution f and the uniform distribution was measured with the Kullback-Leibler distance formula, which simplifies to the classical normalized entropy formula in this case:  nn1in ]1,0[f]1,0[))i(fln()i(f)nln(1)f(e    For uniform distribution en(f)=1, and for singular distribution en(f)0 ( 0)f(elim nnn, if 0)f(lim nn). Statistically, the distribution of smoke velocity vector orientations is closer to uniform than that of rigid or elastic motion, i.e.:  0)t)f(e(P)t)f(e(Plimand0)t)f(e(P)t)f(e(Plimsmokenrigidn1trigidnsmoken0t This property allows for Bayesian threshold selection, which was performed by calculating the entropy for numerous sequences of smoke and non-smoke motion. For small number of samples k-nearest neighbour selection is also possible.  Table 2. The smoke detection algorithm. 1. Compute multiscale optical flow field. 2. Project back to the initial scale points that perform upward motion at the highest scale to obtain regions that contribute to that motion. 3. Determine the local distribution of the velocity vector orientations for these regions. 4. Compute irregularity measure (entropy) for the distributions. 5. Alarm if necessary with the help of a statistical (e.g. Bayesian) decision procedure  The system alarms if entropy exceeds the threshold obtained by Bayesian decision. A detection example is shown in Figures 4 and 5. Two different types of motion are present in the observation field. The program detects the motion of the smoke, and does not detect the waving of the arm, which represents elastic motion.   Figure 5. Snapshot of the detection alarm. The non-smoke motion is not detected.   Figure 6. Motion detection using optical flow.  3. Conclusion  In this paper we have shown that a simple and cheap panoramic system can be well applied for surveillance systems for different (e.g. motion detection and smoke alarm) purposes using only this simple-to-mount device and our robust motion-analysis algorithms. For the smoke-detection we continue further research including the improvement of the algorithm by temporal filtering and more adaptive classification.  Acknowledgements  The help of László Czúni and Tamás Greguss are greatly acknowledged. This work has been supported by the Tateyama Co., Japan.   References  [1] P. Greguss. Exoscope – a New Omnidirectional Imaging and Holographic Device for Life Science Studies. Optical Methods in Biomedical and Environmental Sciences, Elsevier BV, 1994, pp. 309 – 312. [2] Y. Yagi, Y. Nishizawa, M. Yachida. Map-Based Navigation for a Mobile Robot with Omnidirectional Image Sensor COPIS, IEEE Transactions on Robotics and Automation, Vol. 11. No. 5. October 1995. [3] D. R. Matthys, J. A. Gilbert, P. Greguss. Endoscopic measurement using radial metrology with digital correlation, Optical Engineering, Vol. 30, No.10, October 1991, pp. 1455-1460. [4] I. Haritaoglu, D. Harwood, L. S. Davis, Computer Vision Laboratory University of Maryland. Active Outdoor Surveillance. Proceedings of the 10th International Conference on Image Analysis and Processing, Venice, September 1999, pp. 1096 – 1099. [5] E. Durucan, J. Snoeckx, Y. Weilenmann. Illumination Invariant Background Extraction. Proceedings of the 10th International Conference on Image Analysis and Processing, Venice, September 1999, pp. 1136 – 1139. [6] J. W. Davis, A. F. Bobick. The Representation and Recognition of Action Using Temporal Templates. IEEE Conference on Computer Vision and Pattern Recognition, 1997. [7] M. Maurizot, P. Bouthemy, B. Delyon: 2D Fluid Motion Analysis From a Single Image: Applications to Experimental Visualisation of Fluid Flow, IRISA, Report No. 1155, 1998 [8] L. Barron, D.J. Fleet, S. Beauchemin. Performance of optical flow techniques, International Journal of Computer Vision. 12(1), 1994, pp 43-77. [9] A.R. Rao, R.C. Jain: Computerized Flow Field Analysis: Oriented Texture Fields, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 14,No. 7, 1992, pp. 693-709 

Application of panoramic annular lens for motion analysis tasks: surveillance and smoke detection

Abstract

Similar works

Full text

Available Versions

SZTAKI Publication Repository

SZTAKI Publication Repository