In this paper, we present a system framework for event detection in pedestrian and tracking applications. The system is built upon a robust computer vision approach to detecting and tracking pedestrians in unconstrained crowded scenes. Upon this framework we propose a pedestrian indexing scheme and suite of tools for detecting events or retrieving data from a given scenario

Kelly, Philip

O'Connor, Noel E.

Smeaton, Alan F.

English

M. Harville

Crossref

Event Detection in Pedestrian Detection and Tracking Applications

Abstract. In this paper, we present a system framework for event de-tection in pedestrian and tracking applications. The system is built upon a robust computer vision approach to detecting and tracking pedestri-ans in unconstrained crowded scenes. Upon this framework we propose a pedestrian indexing scheme and suite of tools for detecting events or retrieving data from a given scenario

Philip Kelly

Noel E. O’connor

Alan F. Smeaton

CiteSeerX

Event detection in pedestrian detection and tracking applications

Name not available

Event Detection in Pedestrian Detection andTracking ApplicationsPhilip Kelly, Noel E. O’Connor, and Alan F. SmeatonCentre for Digital Video Processing, Adaptive Information Cluster,Dublin City University, IrelandAbstract. In this paper, we present a system framework for event de-tection in pedestrian and tracking applications. The system is built upona robust computer vision approach to detecting and tracking pedestri-ans in unconstrained crowded scenes. Upon this framework we proposea pedestrian indexing scheme and suite of tools for detecting events orretrieving data from a given scenario.Key words: Pedestrian Detection, Tracking, Stereo, Event Detection1 IntroductionRobust detection and tracking of humans is a key enabling technology for manyapplications. It is key to knowing who is where in a scene and what their actionshave been. It potentially allows other layers in an application’s framework to inferbeliefs about those people. However, depending upon the end-user application,a variety of event detectors may need to be defined. In this paper, we proposea framework for detecting user-defined events at run-time which may then beused in a variety of surveillance applications.2 Pedestrian Detection and TrackingMany of the person detection and tracking techniques described in the liter-ature make assumptions about the environmental conditions, pedestrian andbackground colour intensity information [1], occlusions [2], the pedestrian flow,pedestrian movements, or that a person will exist within the scene for a signif-icant number of frames un-occluded[3]. We have developed a robust pedestriandetection and tracking system [5] for a single stereo camera that makes noneof the above assumptions and in addition requires no external training. In ad-dition, it is camera viewpoint invariant and is able to handle a large variety ofpedestrian appearances, making a single constraining assumption that a per-son is standing vertically with respect to the groundplane. During tracking, thesystem can obtain statistics about each pedestrian, such as colour appearancemodels, the duration of time spent in the camera’s field of view and the paththey have traversed through the scene. Since stereo information is used, both the3D position, velocity and the height of a pedestrian can be accurately obtained.2 Event Detection in Pedestrian Detection and Tracking Applications(a) (b) (c) (d) (e) (f)(g) (h) (i) (j) (k) (l)Fig. 1. Examples; (a)-(f) Pedestrian Detection; (g)-(l) Pedestrian Tracking.Evaluation of this system on 3,500 manually annotated ground truth pedestriansfrom sequences with varying camera height, camera orientation and environmen-tal conditions reveal extremely accurate performance of our proposed approach.Figure 1(a)-(f) presents some illustrative results where each detected pedestrianis enclosed by a bounding box of a certain colour. An example of a trackingsequence can be seen in figure 1(g)-(l), where the second row of images depictthe scene from a plan-view or birds-eye view orientation. In these plan-view im-ages, the white lines indicate the bounds of the scene, the position of detectedpedestrians in that frame are illustrated by a circle of the same colour as theirbounding box, and tracks are depicted as “tails” from the centre of the circle toprevious positions in the scene.3 Event DetectionBeyond detection and tracking, many applications require a further level of pro-cessing whereby the actions of pedestrians must be interpreted. For some appli-cations these event detectors can be hard-coded into the application framework,for example surveillance applications which determine pedestrian flow densitiesduring specific time periods. For many applications, however, the exact eventdetector cannot be hard-coded into the system as; (1) the event definition isdependent on an undefined scene; or (2) the event is itself undefined. The firstscenario is typical of many Ambient Intelligence (AmI) applications [6], as al-though the event required to be detected is known (e.g. detect pedestrians inEvent Detection in Pedestrian Detection and Tracking Applications 3a designated area waiting to cross the road), information about the scene (e.g.the exact designated area) is unknown. The second scenario is typical of gen-eral purpose surveillance applications where events are determined by a user fordetection either at run-time or after the event has occurred. Take for exam-ple a typical CCTV surveillance system, if the surveillance video is augmentedwith robust pedestrian tracking and statistical information it becomes possibleto quickly search the video for specific events via the augmented meta-data. Forexample, if a lost child’s appearance is known (e.g. the colour of their clothes andtheir height) then all possible detections of the child can be subsequently flaggedfrom every camera in the system. Other examples include adding “watched” ar-eas in a scene (e.g. to determine the number of pedestrians who entered a specificshop) or flagging “unusual” events such as lingering pedestrians (e.g. to reviewall footage of persons who were in the vicinity of an area that was vandalisedfor a significant amount of time).4 Proposed ApproachBuilding on our prior work, we have developed a pedestrian indexing scheme andsuite of tools for detecting events and retrieving data from a given scenario. Onesuch tool is the creation of 3D hotspot regions from 2D plan-view images - seethe 2D yellow coloured area in figure 2(a). In this figure, it should be noted thata background colour model has been projected onto the image-plane to allowthe gauging of distance and orientation within the plan-view image. Using thisplan-view image, a hotspot is simply created in the proposed system by circling aregion of interest within the plan-view image. The resultant hotspot can be seenas a 3D area of interest which can be incorporated into the definition of eventsfor some application scenarios. For example, in the framework of an automatedpedestrian traffic light system it can be used to define an area where pedestrianstend to wait before crossing the road – see figures 2(b)-(f). A detected event,which in turn leads to a changing of the traffic lights, could then be definedusing this hotspot region and an event (or query) syntax. An example of thesyntax is, Event(cross) = hpt(1), p(g5), t(10s), which declares an event calledcross to be detected if on hotspot number 1 (hpt(1)) there are greater than 5pedestrians (p(g5)) who have been waiting for more than 10 seconds (t(10s)).The proposed system can be easily used across scenes with differing underlyingstructure whilst the event syntax allows a number of user-defined events to becreated and searched for during run-time in content-based retrieval applications.5 Future WorkThese techniques will underpin a user-defined event detection system which canbe simply tailored for a suite of pedestrian detection and tracking applications.Currently the event syntax consists of a number of features including; hotspots,timings (such as the length of time an object is in the scene), and pedestriannumbers. However, the syntax should be extended to incorporate other features,4 Event Detection in Pedestrian Detection and Tracking Applications(a) (b) (c) (d) (e) (f)Fig. 2. Pedestrian crossing application; (a) Hotspot; (b)-(f) Waiting to cross the road.including; time of day, pedestrian interactions (such as pedestrians walking in agroup or on their own) and pedestrian statistics such as colour, height, velocityand position).AcknowledgementsThis material is based on works supported by Science Foundation Ireland underGrant No. 03/IN.3/I361. The authors wish to acknowledge the support of theEuropean Commission under the FP6-027026-K-SPACE contract. This paperwas awarded a best poster prize at the 2007 K-Space Jamboree Workshop forPhD students.References1. A.W. Senior, “Tracking with probabilistic appearance models,” in IEEE Interna-tional Workshop on Performance Evaluation of Tracking and Surveillance Systems,June 2002, pp. 48–55.2. M. Harville, “Stereo person tracking with adaptive plan-view templates of heightand occupancy statistics,” International Journal of Computer Vision, vol. 22, pp.127–142, 2004.3. A.E Elgammal and L.S. Davis, “Probabilistic framework for segmenting peopleunder occlusion,” in IEEE International Conference on Computer Vision, 2001,vol. 2, pp. 145–152.4. P. Kelly, E. Cooke, N. E. O’Connor, and A. F. Smeaton, “Pedestrian detection usingstereo and biometric information,” in International Conference on Image Analysisand Recognition, September 2006, pp. 802–813.5. P. Remagnino and G.L. Foresti, “Ambient intelligence: A new multidisciplinaryparadigm,” in IEEE Transactions on Systems, Man and Cybernetics, January 2005,vol. 35, pp. 1–6.

O\u27Connor, Noel E.

Irish Universities

DCU Online Research Access Service

Ambient intelligence: A new multidisciplinary paradigm,&quot;

Pedestrian detection using stereo and biometric information,&quot;

Probabilistic framework for segmenting people under occlusion,&quot;

Stereo person tracking with adaptive plan-view templates of height and occupancy statistics,&quot;

Tracking with probabilistic appearance models,&quot;

Event detection in pedestrian detection and tracking applications

Abstract

Similar works

Full text

Available Versions

Crossref

CiteSeerX

Name not available

Irish Universities

DCU Online Research Access Service