There has been a lot of research done towards both camera and Wi-Fi tracking respectively, both these techniques have their benefits and drawbacks. By combining these technologies it is possible to eliminate their respective weaknesses, to increase the possibilities of the system as a whole. This is accomplished by fusing the sensor data from Wi-Fi and camera before inserting it in a particle filter. This will result in a more accurate and robust localization system

Ledda, Alessandro

Spruyt, Vincent

Van den Berghe, Sam

Weyn, Maarten

English

Ghent University Academic Bibliography

2011 INTERNATIONAL CONFERENCE ON INDOOR POSITIONING AND INDOOR NAVIGATION (IPIN), 21-23 SEPTEMBER 2011, GUIMARA˜ES, PORTUGALCombining wireless and visual tracking for anindoor environmentSam Van den Berghe*, Maarten Weyn**, Vincent Spruyt***, Alessandro Ledda*****Artesis University College, Antwerp, Belgium. Email: sam.vandenberghe@student.artesis.be**Artesis University College, Antwerp, Belgium. Email: maarten.weyn@artesis.be***Artesis University College, Antwerp, Belgium. Email: v.spruyt@ieee.org****Artesis University College, Antwerp, Belgium. Email: Alessandro.ledda@artesis.beAbstract—There has been a lot of research done towards bothcamera and Wi-Fi tracking respectively, both these techniqueshave their benefits and drawbacks. By combining these tech-nologies it is possible to eliminate their respective weaknesses,to increase the possibilities of the system as a whole. This isaccomplished by fusing the sensor data from Wi-Fi and camerabefore inserting it in a particle filter. This will result in a moreaccurate and robust localization system.I. INTRODUCTIONThe need for localization is increasing and so is the range ofpossibilities. The increasing availability of mobile applicationsand social networking has made the hunger for localizationgreater, as well as the possible solutions. There are multipleways to track people in a building environment. Some arevery accurate like ultra-wide band [1], while others require noadditional infrastructure [2, p. 24]. But there is not one idealtechnology, there is always a drawback when using a certaintechnology [3, p. 72]. By combining these technologies we cantry to remove the negative aspects of each individual methodand augment its strengths. This paper proposes an algorithmthat combines Wi-Fi localization and static camera tracking.The main goal is by combining Wi-Fi fingerprint local-ization and camera tracking, to increase the accuracy andreliability of the overall system. A static camera is moreaccurate than Wi-Fi localization, but has blind spots, suffersfrom occlusion and it is difficult to perform identification. Wi-Fi localization is accurate up to room level [3], but requiresusers to carry a Wi-Fi capable device, this also means thatidentification is inherent in this form of localization. Thatmeans that Wi-Fi alone cannot locate anybody who does notwant to be tracked, i.e. does not enable his or hers Wi-Fidevice.The purpose of fusing Wi-Fi and video data is to havea smaller localization error in the rooms where there is acamera, in contrast to only Wi-Fi, but still offer room levellocalization where there are no cameras. This paper will ratherfocus on preparing the captured images and fusing that datawith the Wi-Fi data, than on the localization algorithm and Wi-Fi data. The Localization algorithm using Wi-Fi is the sameas described in [2].The preparation of the vision aspect is defined as isolatinghuman figures in the image, modeling those locations in theimage as a Gaussian mixture model [4] on a floor plan. Fusionof camera and Wi-Fi will encompass the way the probabilitiesof both methods are combined to get the most accurate yetstill robust tracking.II. METHODSA. Particle FilterA particle filter [5] is able to cope with the multi-modalnature of the problem of locating a person in a complexenvironment. An added benefit of using a particle filter is thatwe can alter the measurement model as desired, so we can havedifferent measurement models depending on what kind of datais available. To properly scan all the channels for Service SetIdentifier (SSID) and signal strength a certain amount of timeis needed, while a camera updates multiple times per second.This results in data arriving asynchronous. In case of a highserver load it is possible that several frames of image data areignored.The main components in a particle filter are the motionmodel, measurement model and resampling [2], [6]. Themotion model generally consists of rules that govern how theparticles can move, these rules are usually modeled to reflectthe real world. It is possible to move the particles randomlybut it is more efficient to move particles as the tracked objectwould.The measurement model describes how the measurementsfrom the world are used to assign a weight to particles. Thehigher the weight of a particle, the more ‘correct’ we estimatethat particle to be. The sum of all particle weights need to be1, so that the collection of particles can be called a posteriordensity function.The resampling step describes how particles are repositionedbetween frames. Particles with low weights are removed, whileparticles with high weights are duplicated. This results in ahigher particle density in areas with high probability, sincethose are the areas that are the most interesting to monitor.B. MeasurementBoth measurements are fundamentally different: where theWi-Fi measurement compares the signal strength of the client(a tag, smartphone, netbook, etc.) to a database of signalstrengths, camera tracking involves detecting an object as itmoves through the environment. This means that Wi-Fi doesnot have problems with identification, since only the objectthat is being tracked can transmit the data relevant to itslocalization and by doing so automatically identifies itself.Identification might be easy for Wi-Fi localization, it cannottrack an object that does not give its Wi-Fi signal strength.Camera tracking has much more difficulties to identify whatit is tracking, it is not inherent as with Wi-Fi. However it ispossible to detect all other objects in the viewplane, so thatit is possible to track the people who are not being trackedwith Wi-Fi or to increase the accuracy by combining the twomeasurements.C. Wi-FiThe measurement model of [2] is used. It uses patternmatching, here the difference feature vector of the receivedsignal strengths (RSS) from multiple Wi-Fi-access points. Thefingerprint data and the measurement taken by a client arecompared, using the kernel method. Penalties are added ifaccess points are missing from the measurement data or ifextra access points are found in the measurement data. If anaccess point is visible at the location of the tag or device butis not represented in the fingerprint of a certain location, thenwe assume that the fit between measurement and fingerprintis less accurate. The same logic applies when there is anaccess point missing in the measurement that is saved in thefingerprint. This is implemented by adding a penalty to theweight, respective to either the RSS of the extra signal or theexpected RSS value.Because fingerprint matching relies on a database with RSSvalues from the area wherein the tracking will occur, it isnecessary to measure those RSS values at certain intervals inspace. This is a drawback, because it requires some manuallabor, but is preferred to methods like time of flight, because itdoes not require that the location of access points are known.D. Vision systemThis section will describe the processing of the video framesundergo before the data is fused together which illustratedby Figure 1. First the foreground segmentation is described,followed by how human shapes are extracted and finallymapped to a floor plan.1) Background Subtraction: Because of the static cameraposition, a good point to start detecting people is backgroundsubtraction. In its most basic form background subtraction(BGS) takes an image of a room with only background objects,then it uses the absolute difference between the backgroundimage and the current video frame, this is called imagedifferencing. After thresholding this will result in a maskwhich segments the foreground objects from the background.However backgrounds are not static. Changes in lightingand objects being moved, like chairs and tables, can renderthe background image outdated and useless. To combat thisit is necessary to update the background image at a specificlearning rate. This results in a trade-off between copingwith fast changing environment factors, such as lighting, andpreventing temporarily stationary foreground objects to beabsorbed in the background.An approach that differs from the image differencing in theway that it does not use a single image as background model, isMixture of Gaussians, which is displayed in Figure 1(b). Herea pixel in the background model is represented by Gaussiankernels at a certain color vector, in this case the RGB colorvalue. Because a pixel can consist of multiple Gaussians, thismethod can accurately model regions where the backgroundimage changes over time between a couple of color vectors,such as a tree branch moving in the wind. a pixel from thecurrent frame is compared to that pixel in the backgroundmodel, which is a certain amount of Gaussian kernels. If itlies within a certain threshold of a Gaussian it is classified asbackground. If the pixel that is being compared falls outsideall Gaussians it is classified as foreground model and thebackground model is updated. [4]The resulting image is called a foreground mask, it isbasically a binary map of pixels which are deemed to be ofa foreground object. This mask will consist of all objects thatare not stationary. This also includes things like chairs thathave recently been moved. Since the goal is to track humanbeings we try to eliminate these false positives. Generallya person will appear as a tall blob in the foreground mask,thus by focusing on these shapes we can reduce the impactof objects like moved furniture. Figure 1(b) shows the resultfrom a mixture of Gaussians BGS.2) Human filtering: A person in three dimensional spacewill occupy a cuboid, when projected onto a two dimensionalplane, like an image, that person will occupy a rectangle in theimage. The image is filtered by a box-filter with the width andheight of the rectangle a person would occupy in the image.The difference is that the filter is not centered around its originpoint. The origin point is located at the bottom of the structureelement, this focuses the most intensity at the bottom of theblob as described by Van Hese et al. [7]. An added constraintis that the pixel value at the origin point of the structuringelement, has to be higher than a certain threshold. This isdone to prevent the filter from returning high values below thedetected blob. As a person gets closer to the camera, the regionhe occupies will get larger as well. This is taken into accountby defining two sizes of filter, one at the furthest region in theimage and one size for the nearest region, for the rest of theimage the size is interpolated between the large en the smallsize.At this stage the foreground mask will consist solely of thelowest region of tall blobs, which we assume are the feet ofpeople in the room. The reason why the lowest region is themost interesting is because that is the most accurate way oftransforming the location in the camera image to a location ona map of the room. The transformation from camera to floorplan would cast ‘shadows’, bright areas on a map as a resultof the projection onto the floor plan.3) Gaussian modeling: To further prevent this projectioneffect, and reduce the consumed bandwidth, the filtered fore-ground mask is described using Gaussian kernels. The kernelsthat are used are circular 2D Gaussian functions. To modela binary image with Gaussian functions, we make some(a) (b) (c) (d)Fig. 1. The steps of the visual preprocessing. (a) the original image. (b) The foreground mask returned by the background subtraction. (c) Human filteringapplied to the foreground mask.(d) The Gaussian kernel of the blob in image (c) mapped to the floor plan.assumptions and cut corners. For instance, a binary image isnot desired when using a particle filter, a more beneficial shapeis in fact a Gaussian curve, a peak with a gentle slope.With that in mind it is justified to inaccurately model thebinary image with Gaussian functions. Secondly, by choosingcircular Gaussian functions we can further reduce the ‘shadow’effect created by projecting the image. By modeling theforeground mask before it is fitted to the floor plan, wecan maintain the circular nature of the blobs. The image ismodeled by Gaussian curves with coordinates x and y and aσ parameters, only its coordinates are completely transformedwhile the standard deviation is scaled accordingly, resulting incircular Gaussian functions on the floor plan as seen in Figure1 (d), which is what is desired.A method for finding Gaussian distributions in data isExpectation Maximization algorithm [8]. Here a number ofGaussian distributions are mapped to the data. The drawbackof this is that the number of separate clusters has to be known,this is not feasible in this setup. Thus a separate algorithm isdevised as shown in Algorithm 1.The proposed algorithm starts from a binary image, wherefor every white pixel a Gaussian kernel, with a standarddeviation, is added to an array of Gaussian kernels. Then everykernel in that list is compared against every other kernel. Iftwo kernels are not c-separated [9] the kernels are combined,meaning their location is averaged and standard deviation isconvoluted according to equation 1. This is done until no newcombinations are made. This method is illustrated in Figure2.new σ =√σ21 + σ22 (1)f(x, y) =12piσ2e−12 [(x−µxσ )2+(y−µyσ )2] (2)This algorithm gives Gaussian functions located at placeswith a high probability of having a person there. The formulaof a two dimensional circular Gaussian curve is as shown inEquation 2, with σ = σx = σy . The normalizing constant1√2piσ2is there to insure that the integral of the curve is one,it causes the intensity of the peak to decline as the standarddeviation gets larger. Large blobs in the image make for largestandard deviations in the gauss kernel that represents it, butthe larger the blob, the larger the probability of a personAlgorithm 1 Mapping Gauss curves to blobs in an imagefor all pixelvalues ≥ threshold doGaussList← newgaussKernel {pixelcoord, defaul σ}end forunstable = truewhile unstable dofor all gaussKernelsinGaussList dofor all OthergaussKernel inGaussList doDistance = ‖gaussKernel −OthergaussKernel‖Totalσ = gausskernel.σ +Othergausskernel.σif distance ≤ Totalσ thenCombine(gaussKernel, OthergaussKernel)not stable = trueend ifend forend forif noCombinationsOccured thenunstable = falseend ifend whilebeing there. Therefore we can disregard the normalizingconstant, knowing that the particle filter normalizes itself aftermeasurement.III. FUSIONCombining the data from Wi-Fi and video is an importantstep, here it is attempted to increase the amount of valuableinformation. The benefit of fusing these two measurementsis that Wi-Fi is data that only refers to the client while thecamera image has data that refers to all persons in its view. Acamera provides a submeter accurate location but Wi-Fi onlyhas a zone [3]. However the camera has blind spots, is notlocated in every room, and because of the adaptive backgroundsubtraction a stationary person will eventually be absorbed inthe background. Therefore it is critical to determine what thestate of the sensors are.Wi-Fi as only sensor can be used as a measurement and willlocate a person up to room level, but since this vision system’smeasurement has no concept of identification it is ill advisedto use it as measurement on its own. There would be no way(a) (b) (c) (d) (e)Fig. 2. The results of Gaussian modeling. (a) a test image with white blobs with increasing size. (b) The resulting image from the human filtering. notice thatthe blob in the center is about the same size as the blob on the left, despite their difference in size in the original image. (c) Initial state of Gauss modelingalgorithm. (d) The third iteration of the algorithm. ?? Eight and in this case final iterationto determine that the correct person has been located. Becausethe nature of the transmitted data from a camera server, thereis either current sensor data or there is not, it is possible todecide in real-time which variation of measurement model touse. In the case where only Wi-Fi data is received, obviouslyonly the measurement model for Wi-Fi is used.When both Wi-Fi and camera data are available, then thetwo measurements are combined with a naive Bayesian witha confidence measure 3. After this occurrence the same Wi-Fimeasurement is repeated when newer camera data is available.Initially, the confidence measure α for the Wi-Fi measurementis one, i.e. very confident since the measurement has just beentaken. As the Wi-Fi data becomes older the confidence in thatmeasurement decreases, so that eventually when α is zero, theentire probability, P (Wi-Fi|loc) is reduced to 1, and effec-tively removed from the equation. Similarly, the confidencemeasure β is determined by the amount and distribution ofkernels, where β will be closer to one when there are fewerkernels and these are bunched close together, and closer tozero when there are a lot of kernels that are spread over alarger area.P (loc|Wi-Fi, cam) = α ∗ P (Wi-Fi|loc)α ∗ P (Cam|loc)β(3)IV. RESULTThe resulting data are compared to the ground truth, anddifferences in measurement models as to compare the perfor-mance of Wi-Fi alone, camera alone and the both combined.The resulting 2 dimensional error is represented as a cumula-tive distribution function (CDF) shown in Figure 3. This allowsfor fast analysis of both the accuracy and precision.The conditions that were tested included a person with aWi-Fi client moving around in the test area alone with nointerference, this situation is represented by Figure 3(a). Otherconditions include a stationary Wi-Fi client while a personwalks around, and a cluttered scene where one Wi-Fi clientand several others walk in the test area. This is shown in Figure3(b), displaying no significant increase in accuracy to Wi-Fi,but the location error is seldom worse than Wi-Fi alone.V. CONCLUSIONThe results indicate that by combining the two measure-ments the accuracy can be increased while never dippingbelow the best accuracy the of a single measurement.(a)(b)Fig. 3. (a) The cummulative distribution function of the user walking aroundthe test area without interference.(b) all other situationsREFERENCES[1] R. Zetik, J. Sachs, and R. Thom, “UWB localization - active and passiveapproach.”[2] M. Weyn, “Opportunistic seamless localization,” Ph.D. dissertation, Uni-versity of Antwerp, Mar. 2011.[3] J. Torres-Solis, T. H. Falk, and T. Chau, A review of indoor localizationtechnologies: towards navigational assistance for topographical disori-entation. In-Tech Publishing, 2010, ch. Chapter 3, pp. pp. 51–84.[4] M. Piccardi, “Background subtraction techniques: a review,” The ARCCentre of Excellence for Autonomous Systems (CAS) Faculty of Engi-neering, UTS, Tech. Rep., 2004.[5] G. Mori, Particle Filter Notes, Simon Fraser University, ComputingScience, 2005.[6] F. Gustafsson, F. Gunnarsson, N. Bergman, U. Forssell, J. Jansson,R. Karlsson, and P.-J. Nordlund, “Particle filters for positioning, navi-gation, and tracking,” Signal Processing, IEEE Transactions on, vol. 50,pp. 425 – 437, 2002.[7] P. Van Hese, S. Gruenwedel, V. Jelaca, J. Nino, and W. Philips, “Evalu-ation of Background/Foreground Segmentation Methods for multi-viewOccupancy Maps,” in Positioning and Context-Awareness Conference,2011.[8] J. A. Bilmes, A Gentle Tutorial of the EM Algorithm and its Application toParameter Estimation for Gaussian Mixture and Hidden Markov Models,1998.[9] S. Dasgupta, “Learning Mixtures of Gaussians,” Computer Science Divi-sion (EECS) University of California, Tech. Rep., 1999.

Combining wireless and visual tracking for an indoor environment

https://repository.uantwerpen.be/docman/irua/192f6a/74087a83.pdf

Combining wireless and visual tracking for an indoor environment

Abstract

Similar works

Full text

Available Versions

Ghent University Academic Bibliography