This paper proposes a new trajectory matching method using logic operations on binary images. By using small binary images we are able to effectively utilize the large word size offered in modern CPU architectures, resulting in a very efficient evaluation of similarities between trajectories. The efficiency is caused by the fact that all bits in the same word are processed in parallel. Representing trajectories as small binary images has other advantages, such as a low space requirement and good noise resistance. The proposed method is evaluated on a publicly available dataset, and is compared to the more sophisticated Longest Common Subsequence (LCSS) method. In addition, synthetic experiments show the good efficiency and accuracy of the proposed method, enabling real time trajectory retrieval on databases with millions of trajectories.postprin

Schnieders, D

Wong, KKY

Zhuo, W

HKU Scholars Hub

Title Fast trajectory matching using small binary imagesAuthor(s) Zhuo, W; Schnieders, D; Wong, KKYCitation The 3rd International Conference on Multimedia Technology(ICMT 2013), Guangzhou, China, 29 November-1 December 2013.Issued Date 2013URL http://hdl.handle.net/10722/189619Rights Creative Commons: Attribution 3.0 Hong Kong LicenseFast Trajectory MatchingUsing Small Binary ImagesWei Zhuo1,2, Dirk Schnieders1, and Kwan-Yee K. Wong11 Department of Computer Science, The University of Hong Kong, Hong Kong2 School of Electric and Information Engineering, Xian Jiaotong University, ChinaAbstract. This paper proposes a new trajectory matching method us-ing logic operations on binary images. By using small binary images weare able to effectively utilize the large word size offered in modern CPUarchitectures, resulting in a very efficient evaluation of similarities be-tween trajectories. The efficiency is caused by the fact that all bits in thesame word are processed in parallel. Representing trajectories as smallbinary images has other advantages, such as a low space requirement andgood noise resistance.The proposed method is evaluated on a publicly available dataset, andis compared to the more sophisticated Longest Common Subsequence(LCSS) method. In addition, synthetic experiments show the good effi-ciency and accuracy of the proposed method, enabling real time trajec-tory retrieval on databases with millions of trajectories.Keywords: Trajectory Matching, Video Surveillance, Image Processing1 IntroductionGovernments and private institutions capture and store large quantities of videosurveillance footage for later retrieval. Video surveillance has received great at-tention in past decades and robust object segmentation and tracking methodsare available. The motion path that an object takes in time, i.e. its trajectory,can be accurately determined from video surveillance footage. Trajectories pro-vide a rich source of information and can be used to classify an object’s behaviorand discover objects that followed a certain motion characteristic or moved ina similar way. For instance, by comparing trajectories of newly tracked objectswith those trajectories previously obtained, abnormal behavior can be detected.In these types of applications it is crucial to query large quantities of trackingdata both quickly and accurately.In this paper, we propose a novel trajectory matching method using logicoperations on binary images. Instead of using raw object trajectory data, i.e. asequence of image points, we approximate trajectories with small binary images.By doing so we are able to save significant cost in both time and space atthe expense of negligible loss in accuracy. The cost saving in time is achievedbecause binary images allow us to design an effective cost function for comparisonof trajectories which effectively utilizes a large word size in modern hardware2 Fast Trajectory Matchingarchitectures. On current hardware architectures, 64 pixels in the binary imagesare compared to a query binary image in parallel. As a result, we are ableto quickly estimate similarities between trajectories. The cost saving in spacefollows from the fact that the raw trajectory representation typically requiressignificantly more space compared to our small binary image representation.The rest of this paper is organized as follows. Sect. 2 reviews the existing liter-ature on trajectory matching and discusses the differences between existing worknd the proposed method. In Sect. 3, we introduce the trajectory representationand in Sect. 4 we propose the similarity measure. In Sect. 5 we present experi-mental results on synthetic and real data, followed by conclusions in Sect. 6.2 Literature reviewThe most widespread prior trajectory representations include chain codes, piece-wise linear approximations, polynomials and splines. Chain code [1] is a simplerepresentation of trajectories, consisting of piecewise linear segments encodedwith 8 basic directions. Other representations, like splines and polynomial inter-polation, are more sophisticated, however often require a more complex compu-tation and are therefore less efficient. Others attempt to represent motion trajec-tory in a feature space. For example, Bashir et al. [2] segmented each trajectoryinto multiple atomic sub-trajectories and characterized each sub-trajectory byits principle component. Zhou et al. [3] extracted features using tensor subspaceanalysis and then performed standard mean shift to cluster the trajectory fea-tures. Chen et al. [4] introduced null-space representation which is invariant toviewpoints. Chen and Chang [5] proposed a wavelet-based approach that decom-posed a trajectory into sub-trajectories which were described by a feature vectorthat includes the acceleration, initial velocity, arc length, order and multi scaleedge points. In contrast to the existing work above, we represent a trajectory asa small binary image.To retrieve similar trajectories, a proper matching technique and similaritymetric should be developed. The similarity metric will depend on the trajec-tory representation. Several types of trajectory matching methods have beenproposed in the literature, and we will briefly discuss the following: DynamicTime Warping (DTW), Longest Common Subsequence (LCSS), vector match-ing and statistical matching. DTW [6] is a dynamic matching technique thatuses dynamic programming and has the advantage of being conceptually verysimple, but unfortunately is quite sensitive to noise. LCSS [7] has been shownto achieve a very high accuracy and we will use it for experimental compari-son in this paper. Vector matching techniques perform matching using a sim-ilarity metric. Commonly used distances include Euler-distance, city-distanceand Minkowski-distance. Other methods, such as [8] and [9], employ statisticalmatching techniques to find appropriate trajectories.The existing methods in the literature achieve quite remarkable trajectorymatching accuracy and are suitable to obtain similar (scale, translation and ro-tation invariance) trajectories on small databases. However with the increasingFast Trajectory Matching 3amount of tracking information, comparing millions of database items quicklyis very desirable. To achieve such a high performance we assume scale, trans-lation and rotation variance in our approach, which is of particular interest invideo surveillance where the camera is stationary. A strong robustness to noiseis achieved by utilizing relatively small binary images.3 Trajectory RepresentationLet a raw trajectory be given as a sequence of absolute 2D points in an imageof size m′ × n′Γ = [x0 y0 ], [x1 y1 ], . . . , [xe ye ]. (1)We represent this trajectory as an 8-adjacent path in a small m×n binary imageI =I0 I1 . . . In−1In In+1 . . . I2n−1.........I(m−1)n I(m−1)n+1 . . . Imn−1 . (2)To consistently transfer a trajectory from the modality in Eq. (1) to our modalityin Eq. (2), we assume a constant aspect ratio, i.e. m/n = m′/n′. In ourapproach, hardware bit-level parallelism is effectively exploited by representingI as a bit vectorT =[w0 w1 . . . wk], (3)which consists of k words of length w given bywi =[Iwi . . . Iw(i+1)−1]0 < i < k. (4)To avoid internal fragmentation we letmn be a multiple of w. This representationis motivated by the fact that on modern hardware w is usually quite large, i.e.on 64-bit architectures w = 64, resulting in the following key benefits of ourapproach:1. a relative small number of words per trajectory, and2. an efficient computation by processing w bits in parallel.4 Trajectory MatchingA simple approach for matching a query trajectory T to a target trajectory T′is to maximize the overlapping trajectory segments. This can be achieved byformulating a similarity measures(T,T′) = |T ∧T′| (5)= |w0 ∧w′0|+ . . .+ |wk ∧w′k|,4 Fast Trajectory Matchingwhere |w| represents the population count operation, i.e. the number of set bitsin the word w, and ∧ represents the bitwise AND operator. This equation ismotivated by the fact that both bitwise AND and population count are availableon modern microprocessors enabling us to evaluate Eq. (5) quickly. We find thetrajectory that maximizes this similarity measure asTˆ = maxT′s(T,T′). (6)Unfortunately, the dissimilarity measure in Eq. (5) is not optimal because it willonly measure the length of overlapping trajectory segments but it completelyignores non-overlapping trajectory segments. Alternatively, we may formulate adissimilarity functiond(T,T′) = |T⊕T′| (7)= |w0 ⊕w′0|+ . . .+ |wk ⊕w′k|,wherewi ⊕w′i = (wi ∨w′i) ∧ ¬(wi ∧w′i) (8)represents bitwise exclusive disjunction. Note that Eq. (7) measures how longthe non-overlapping trajectory segments are, i.e. it measures the absolute lengthof those paths which appear in T′ or T but not in both. This measurement isalso known as the Hamming distance and determines the minimum number ofbit substitutions required to transform T to T′ or vice versa.To find the best matched trajectory we minimize this dissimilarity functionTˆ = minT′d(T,T′). (9)Unfortunately, the similarity measure in Eq. (7) is also not optimal becauseit measures the absolute number of bit substitutions required. It does not di-rectly take into account overlapping path segments and two paths that are non-overlapping may achieve a lower dissimilarity score than two paths that arepartially overlapping. To overcome this problem we propose to indirectly takethe overlapping path segments into account. Note that the dissimilarity functionin Eq. (7) satisfies the following inequality0 ≤ d(T,T′) ≤ |T ∨T′|, (10)where ∨ represents bitwise OR. In this paper, we propose to optimize for thelargest ratio between non-overlapping and all paths and find the best matchedtrajectory as followsTˆ = maxT′d(T,T′)|T ∨T′| . (11)Fast Trajectory Matching 5240x320 96x128 48x64 24x32 12x16(a)12x16 24x32 48x64 96x12824681012x 106image sizeoperations / s  Eq. (7)Eq. (11)12x16 24x32 48x64 96x128050100150200number of wordsimage size(b) (c)Fig. 1. Efficiency of the proposed method (a) Raw trajectory rendered on imageobtained from a surveillance camera [10] and four binary images used in the proposedmethod (plotted to scale). (b) Theoretical efficiency of Eq. (7) and Eq. (11) using thebinary images shown in (a) as the query. (c) Number of words used per binary imageagainst image size.5 Experimental Evaluation5.1 Synthetic DataWe first measure the efficiency of the proposed method by measuring the numberof times the matching function can be executed per second. This theoreticalefficiency is expected to be very high due to the simplicity of the matchingfunction and the availability of hardware implementations. We have implementedEq. (5), Eq. (7), Eq. (11) and measured their efficiency on an Intel i7 CPUsystem that natively supported population count. We measured the performancein number of evaluations per second and plot results for different image sizes inFig. 1. Since we found the performance of Eq. (5) and Eq. (7) to be equivalentwe have only plotted the performance of Eq. (7). As expected, the efficiency ofEq. (11) is slightly less than Eq. (7) due to the additional operations performed.Nevertheless, it can be seen that Eq. (11) can be evaluated around 10 milliontimes per second for small binary images of size 12 × 16. This large numberfollows from the fact that a binary image of size 12 × 16 just required k = 3words.In a second experiment we synthetically generated 30 original trajectoriesand generated 50 noisy variations for each of these trajectories. To generatenoisy trajectory from a given raw trajectory, we first add uniformly distributednoise to the points of the trajectory. The magnitude of this noise was 20% of thestandard deviation of all the trajectory points. A bezier-spline was subsequentlyfitted to these noisy trajectory points. Fig. 2 shows an example of an originaltrajectory and a synthetically generated noisy trajectory. The generated binary6 Fast Trajectory Matching180 230 28050100150200250xy  Original trajectoryGenerated noisy trajectory(a)12× 16 24× 32 48× 64 96× 128(b)12× 16 24× 32 48× 64 96× 128(c)Fig. 2. Noisy trajectory generated for the synthetic experiment (a) An ex-ample of an original and noisy raw trajectory from the synthetic experiment. (b) Theoriginal trajectory represented as binary images. (c) The noisy trajectory representedas binary images.12× 16 24× 32 48× 64 96× 128Eq. (5) 87% 91% 92% 89%Eq. (7) 88% 86% 67% 41%Eq. (11) 94% 97% 96% 95%(a)12× 16 24× 32 48× 64 96× 128Eq. (5) 89% 95% 99% 98%Eq. (7) 98% 97% 82% 54%Eq. (11) 98% 100% 100% 100%(b)Fig. 3. Accuracy of the proposed method (a) Percentage of top 50 noisy retrievalsthat belong to the same set as the query trajectory. (b) Percentage of top 10 noisyretrievals that belong to the same set as the query trajectory.Fast Trajectory Matching 7images are also shown for reference. There are a total of 1500 noisy trajectoriesthat are clustered into 30 groups. We randomly selected a trajectory from theset of original trajectories and query the database to select the top 50 retrievalsbased on Eq. (5), Eq. (7) or Eq. (11). To test the accuracy of the proposedmethod, we determined how many retrievals belong to the same cluster as thequery trajectory. We repeated this experiment for different sizes of the binaryimages. The results are shown in Fig. 3. The experiment has also been repeatedusing just the top 10 retrievals. It can be seen that reasonable good results canbe obtained with a very small 12× 16 binary image size and that results obtainby Eq. (11) are consistently better than those obtained by Eq. (7) and Eq. (5).This follows from the fact that neither Eq. (7) nor Eq. (5) perform normalizationand only compare absolute similarity and dissimilarity respectively. We concludethat a small binary image of size 12 × 16 results in high accuracy and quickcomputation.5.2 Real DataWe evaluated the proposed method on 3206 public available trajectories fromthe database reported in [10]. Given the high efficiency and accuracy, we gener-ated small binary images of size 12× 16 for all the trajectories in the database.Fig. 4 shows the top 10 retrieval results using Eq. (7), Eq. (5) and Eq. (11) fora specific query. To query the whole database took just ∼ 0.5 milliseconds whenusing Eq. (11), whereas LCSS required several seconds. It can be seen that theresults obtained by Eq. (7) and Eq. (11) are good and the latter performs supe-rior because overlapping as well as non-overlapping trajectories are considered.Results retrieved by Eq. (5) are not as good because just overlapping trajectorysegments are considered. Similarly, LCSS focuses on finding the longest commonsubsequence and does not always return desirable results.6 ConclusionsIn this paper, we presented a novel trajectory matching method using logicoperations on small binary images. By approximating the trajectories using thisnovel representation, we were able to save significant cost in both time and spaceat the expense of retrieval accuracy that is negligible in many applications.References1. Kaneko, T., Okudaira, M.: Encoding of arbitrary curves based on the chain coderepresentation. Transactions on Communications 33 (1985) 697–7072. Bashir, F., Khokhar, A., Schonfeld, D.: Segmented trajectory based indexing andretrieval of video data. In: ICIP. (2003)3. Zhou, H., Tao, D., Yuan, Y., Li, X.: Object trajectory clustering via tensor analysis.In: ICIP. (2009)8 Fast Trajectory Matching(a)Rank Eq. (5) Eq. (7) Eq. (11) LCSS[7]12345678910(b)Fig. 4. Experimental results on real data: Retrieval results obtained on a querytrajectory (a), and top 10 retrieval results using binary images of size 12 × 16 andEq. (5), Eq. (7), Eq. (11) and LCSS[7] (b).Fast Trajectory Matching 94. Chen, X., Schonfeld, D., Khokhar, A.: Robust null space representation and sam-pling for view-invariant motion trajectory analysis. In: CVPR. (2008)5. Chen, W., Chang, S.: Motion trajectory matching of video objects. In: ElectronicImaging, International Society for Optics and Photonics (1999) 544–5536. Hu, W., Tan, T., Wang, L., Maybank, S.: A survey on visual surveillance of objectmotion and behaviors. IEEE Transactions on Systems, Man, and Cybernetics 34(2004) 334 –3527. Vlachos, M., Kollios, G., Gunopulos, D.: Discovering similar multidimensionaltrajectories. In: International Conference on Data Engineering. (2002)8. Johnson, N., Hogg, D.: Learning the distribution of object trajectories for eventrecognition. Image and Vision Computing 14 (1996) 583–5929. Porikli, F.: Trajectory distance metric using hidden markov model based repre-sentation. In: ECCV Workshops. (2004)10. Basharat, A., Gritai, A., Shah, M.: Learning object motion patterns for anomalydetection and improved object detection. In: CVPR. (2008)

Fast trajectory matching using small binary images

http://hub.hku.hk/bitstream/10722/189619/1/Content.pdf

Fast trajectory matching using small binary images

Abstract

Similar works

Full text

Available Versions

HKU Scholars Hub