Fingerprint techniques have a significant advantage in respect of watermarking: a fingerprint can be extracted in each moment of the lifetime of a multimedia content. This aspect is fundamental to solve the problem of copy detection mainly because many copies can be available in huge amount of data in circulation and because each copy can be attacked in several ways (compression, re-encoding, text-overlay, etc.). In this paper the problem of copy detection is studied and tested from two different point of views: content based and identification approaches. The results show that the proposed system is quite robust to some copy modifications and most of all show that the overall results depend on the evaluation method used for testing

Corvaglia M.

Guerrini F.

Leonardi R.

Rossi E.

Corvaglia, Marzia

Guerrini, Fabrizio

Leonardi, Riccardo

Rossi, Eliana

Archivio istituzionale della ricerca - Università di Brescia

Identification versus CBCD: a comparison of different evaluation techniques

CORVAGLIA, Marzia

GUERRINI, Fabrizio

LEONARDI, Riccardo

ROSSI, Eliana

IDENTIFICATION VERSUS CBCD: A COMPARISON OF DIFFERENT EVALUATIONTECHNIQUESM. Corvaglia, F. Guerrini, R. Leonardi, E. RossiDEA-SCL, University of Brescia, ItalyABSTRACTFingerprint techniques have a significant advantage in re-spect of watermarking: a fingerprint can be extracted in eachmoment of the lifetime of a multimedia content. This aspectis fundamental to solve the problem of copy detection mainlybecause many copies can be available in huge amount of datain circulation and because each copy can be attacked in sev-eral ways (compression, re-encoding, text-overlay, etc.). Inthis paper the problem of copy detection is studied and testedfrom two different point of views: content based and identifi-cation approaches. The results show that the proposed systemis quite robust to some copy modifications and most of allshow that the overall results depend on the evaluation methodused for testing.1. INTRODUCTIONThe amount of digital video content present in TV Channels,Internet and Video Web Servers is ever growing thanks to theprogress of multimedia and networking technologies. Due tothe incredible amount of data, the monitoring of media usagebecomes of great importance for various reasons.First of all, the media should be controlled for copyrightmanagement: both the owners of web servers and the pro-prietors of copyright protected media should check if a givencontent has the right to be distributed, how and where it isused. Another application is media usage monitoring, whichaims at supervising the correct placement of visual items suchas advertising. Finally, the same material can be present onvarious sites and there should be the chance to create links byanalyzing video and visual objects instead of text annotation.These aspects can be seen in a general way as the detec-tion and the correct identification of duplicated visual media.The solution is a tricky task because of the large amount ofmultimedia content to deal with and because each visual itemcan suffer different editing operations and modifications interms of color, gamma, text or logo insertion geo,metric trans-formations, etc. that make a copy often less similar to theoriginal video than another video.A well known approach for the detection of duplicatedvisual media is given by watermarking, which consists in theinsertion into the video stream of an imperceptible digital signcalled watermark that can be later retrieved to uniquely iden-tify the content origin and/or to establish its ownership. Wa-termarking presents two main disadvantages. First, the videoshould be pre-processed before its distribution to insert thewatermark which is not a realistic condition for huge amountof video data in circulation. Second, watermarking is not ro-bust enough to properly identify multimedia material that hasbeen severely attacked (re-encoding, text-overlay, etc.)Recently alternative approaches have been considered inorder to overcome the evident limitations of watermarking inthe considered field of interest. Fingerprinting approach is apassive technique and does not require any pre-processing be-cause the content itself and intrinsic measurements are used toidentify the uniqueness of a video. Given a certain video, itsfingerprint consist of a significant set of features opportunelyextracted and combined with the purpose to be robust acrossthe common editing operations and sufficiently different forevery original content to identify it uniquely and reliably.In this paper a fingerprinting technique based on visualfeatures is proposed. Its effectiveness has been evaluated un-der two different perspectives. In the first one the fingerprintis used to detect the transformed version of an original video;in literature this problem is called Content Based Copy De-tection (CBCD). In the second case the fingerprint is used toverify its uniqueness and robustness in the identification pro-cess.Several techniques have been proposed to solve the prob-lem of CBCD. They can be divided into two main groups,namely those relying on global descriptors and those basedon local descriptors. The first kind of descriptor is extractedfrom the whole frame. An example of global descriptor isthe ordinal measure [1] , which consists in dividing the im-age into small blocks and then sorting each block dependingon its average gray level: the signature is the rank of eachblock. Other global descriptors are the YUV color histogramof each frame or the block-based motion direction [2]. Themain disadvantage of global descriptors is the lack of robust-ness against some attacks, for example caption insertion andgeometric transformations, such as zoom, crop and letter-box.To deal with this problem, the approaches based on local de-scriptors compute features only on selected points of a frame,also called points of interest, which can be detected by Harrisinterest point detector [3] or by the frame SIFT descriptor [4].In [5], some points of interest are tracked along the video se-quence to reduce the amount of information and to take intoaccount both spatial, temporal and dynamic behaviors of thelocal descriptor. The main drawback of local features basedmethods is their high computational cost.In the case of identification, the first techniques were in-troduced for image identification [6] [7] etc. and were mainlybased on different transform (wavelet, DCT, etc.) dependingon application. Then many extensions for video and audiohave been proposed [8].The brief overview of the existing techniques shows thatthe approaches differ quite a lot according to the fact they tryto solve the CBCD problem rather than the identification one.In this paper, the same approach is evaluated in both environ-ments in order to study how the performance of a techniquecan vary depending on the selected approach.This paper is organized al follows. Section 2 presents dif-ferent kind of evaluation methods; Section 3 describes theproposed method; in Section 4 results are presented and dis-cussed. Finally, in Section 5 conclusions are drawn.2. EVALUATION TECHNIQUESDepending on the perspective, i.e., identification or CBCD,the performance evaluation technique is different.2.1. Content based approachIn the case of CBCD, the approach is usually the exhaustiveone. Given a database DBm of N multimedia items (e.g.,images, videos or audio clips) Ii with i ∈ [1, N ], given a setof features FIi that characterizes each item Ii, given a queryitem Q extracted from a modified item of the database, giventhe feature FQ extracted from the query, chosen an appropri-ate distance measure D between the features, the exhaustiveapproach consists in computing all distances Di (i ∈ [1, N ]),between FQ and FIi and to select the most similar item Ijwhose feature FIj has the minimum distance with FQ : Dj =min(FIi, FQ) with i ∈ [1, N ].Two common parameters are usually considered, i.e., pre-cision, which is an indicator of the retrieved relevant items inthe retrieved items, and recall, which indicates the retrievedrelevant items in the overall number of relevant items in thedatabase. In some cases these parameters are summarized ina single parameter F which is given by the weighted harmonicmean of the previous two.Other parameters are the rate of false alarms and the prob-ability of miss. TRECVID [9] evaluation introduces a weightedsum of these parameters at different operating points.2.2. Identification approachIn the case of identification, the aim is to verify if a certainquery itemQ, described by its features FQ, has been extractedfrom a multimedia item Ii, described by its features FIi withi ∈ [1, N ], which is considered to be the right one available inthe databaseDBm. This verification is performed by comput-ing the distance DQi between FQ and FIi and then compar-ing it with a selected threshold τ . If the condition DQi ≤ τ istrue, then Q is considered to be extracted from item V .The key point of this approach is the setting of the thresh-old. In this sense, MPEG community for the evaluation ofMPEG-7 Video Signature Tool proposed to proceed in twosteps [10]: in the fist step the threshold is defined while in thesecond one the identification comparison is performed.More in details, at the fist stage, called Independence test,given a large database of videosDBI considered independentamong them and chosen a certain distance measure betweenthe relative features, the threshold τ is determined by the dis-tance value which corresponds to the maximum false positiverate. MPEG proposes to set the false positive rate equal to 5part per millions (ppm). At the subsequent stage, called Ro-bustness test, a set of query items Q are available at differentlevel and kind of modifications (database DBR). Each queryQ described by FQ is compared with the original item Ii de-scribed by FIi. Thus they are considered related only if thedistance is below the threshold τ computed by the Indepen-dence test.3. SYSTEM OVERVIEWThe proposed system is based on the use of multiple visualfeatures, considering their development in time along bothqueries Q and original items Ii. The basic idea consists incomparing the sequence of each feature extracted from thequery with the same feature in the original item, through theuse of a sliding window.Feature extractionThreshold extraction Matching outDBm DBI DBRfalse positive[ppm]Fqτfeature DBDBm DBI DBRmedia DBFIiFig. 1. Framework of the proposed systemThe framework is shown in Figure 1. The set of featuresFIi and FQ respectively of each original item and query isextracted and then stored in three separate databases. The de-velopment in time of FQ is then compared with FIi accordingto the evaluation technique (content based or identification)and considering a sliding window of the same duration of thequery. The distance in each point of the sliding window isobtained averaging the local distances obtained applying themetric operator. When the sliding window has spanned thewhole video item, it is possible to plot the distance betweenthe query and the window of the original video item, withrespect to the temporal position of the window in the videoitem. The global minimum of this function is considered thepoint where the query is positioned into the video.The visual low-level features considered in this work are:MPEG-7 Dominant Color (DC): compact representationof the representative colors in a still image or video frame[11] . More specifically, it consists of the representative colorsand their percentages in the image, plus spatial coherency andcolor variance for each dominant color. The distance measureconsidered to evaluate the distance between two video frames,described by their DCs, is the Earth Mover’s Distance [12].MPEG-7 Color Layout (CL): compact and resolution-invariant representation that indicates the distribution of col-ors in a still image or video frame. This descriptor is obtainedby applying the DCT transform on a 2-D array of local rep-resentative colours in Y Cb Cr color space [11]. The dis-tance measure considered to evaluate the distance betweentwo video frames, described by their CLs, is the MPEG-7standard distance.Luminance Layout (LL): representation of the distribu-tion of luminance in an a still image or video frame. This de-scriptor, which a simplification of the Color Layout, has beenintroduced mainly to deal with monochrome videos. The dis-tance measure considered to evaluate the distance betweentwo video frames, described by their LLs, is the L1 norm.4. EXPERIMENTAL RESULTSExperiments have been performed using a part of the data setavailable for MPEG-7 Video Signature Tool standardization.The main databaseDBm is composed by 1900 video items of3 minutes. The Independent database DBI is composed by a11300 clips of 2 seconds, which have been extracted fromthe videos available in DBm and which have been dividedby source in oder to guarantee the independency in thresholdsetting. The Robustness database DBR contains 545 queriesof 2 seconds subjected to 9 transformations (VCR recording,brightness change, camera recapturing, interlaced/progressiveconversion, grayscale conversion, frame-rate reduction, reso-lution reduction, severe compression and text/logo overlay)with Light strength of attack; in total the number of consid-ered queries is 4905. For each video of the three databases,features DC, CL and LL are extracted and stored in identicaldatabases (Figure 1).The system proposed in Section 3 has been tested in threeconditions:– Content Based (CB-min): as described in Section 2.1,each query of DBR is compared with all the item of DBm(exhaustive approach) and the query candidate is obtainedconsidering the minimum distance;– Identification (ID): as described in Section 2.2, usingDBI three thresholds have been extracted by setting the falsepositive rate to 5 ppm, 50 ppm and 500 ppm; then each queryof DBR is compared with the relative item of DBm in orderto decide if the query is related or not;– Content Based with threshold (CB-thr): exhaustive ap-proach (CB-min) is refined considering the minimum distanceonly if it is below a threshold defined in the identification ap-proach (ID).The performance results of the different features in termsof Detection Rate (%) for each modification are shown in Fig-ure 2, 3 and 4. The classical parameters recall and precisionhave not been used because they are not applicable in the caseif Identification test.avc bri cam int mon res sev tex0102030405060708090100DC performance  CB−minID 5 ppmCB−thr 5 ppmID 50 ppmCB−thr 50 ppmID 500 ppmCB−thr 500 ppmFig. 2. Performance of DC feature varying the evaluationmethodIf we consider the results for CB-min evaluation method,we can observe that the features act similarly. The DetectionRate is higher that 70% for all modification with exception forcamera recapturing (cam) and grayscale conversion (mono).We can also note that LL provides the best performance forgrayscale conversion (mono) and that CL reaches the 90% formost of the modifications.If we consider the results for the other two approaches(CB-thr and ID), we can clearly see that the performance areseriously compromised. As mentioned above the choice ofthe threshold is a crucial aspect that should be carefully dealtwith. For example, the threshold obtained by the Indepen-dence test with false positive rate set to 5ppm provides per-avc bri cam int mon res sev tex0102030405060708090100CL performance  CB−minID 5 ppmCB−thr 5 ppmID 50 ppmCB−thr 50 ppmFig. 3. Performance of CL feature varying the evaluationmethodavc bri cam int mon res sev tex0102030405060708090100LL performance  CB−minID 5 ppmCB−thr 5 ppmID 50 ppmCB−thr 50 ppmID 500 ppmCB−thr 500 ppmFig. 4. Performance of LL feature varying the evaluationmethodformance closed to null, while with false positive rate set to50/500ppm the performance significantly increases. In thissense the Independence test forces the threshold to a valuethat is not the optimum for every technology.5. CONCLUSIONSIn this paper a comparison of two evaluation techniques forcopy detection is reported: one method is based on the classicexhaustive approach while the other one on the identificationapproach.The tests have been performed using the video datasetprovided by MPEG for MPEG-7 Video Signature Tool stan-dardization. The results show that the evaluation method canseriously compromise the performance evaluation (the exhaus-tive approach provides significantly better results than the iden-tification one). Moreover the choice of the threshold is crucialand the proposed system needs to be improved for the criticalquery modifications.6. REFERENCES[1] D. Bhat and S. Nayar, “Ordinal measures for image cor-respondence,” IEEE Trans. on Pattern Analysis and Ma-chine Intelligence, vol. 20, no. 4, pp. 415–423, 1998.[2] A. Hampapur, K. Hyun, and R. Bolle, “Comparisonof sequence matching techniques for video copy detec-tion,” in SPIE Conf. on Storage and Retrieval for MediaDatabases, San Jose, CA, USA, Jan. 2002, pp. 194–201.[3] A. Joly, O. Buisson, and C. Frelicot, “Content-basedcopy retrieval using distortion-based probabilistic simi-larity search,” IEEE Trans. on Multimedia, vol. 9, no. 2,pp. 293–306, Feb. 2007.[4] D. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. Journal of Computer Vision,vol. 60, pp. 91–110, 2004.[5] J. Law-To, O. Buisson, V. Gouet-Brunet, and N. Bouje-maa, “Robust voting algorithm based on labels of be-havior for video copy detection,” in ACM 14th AnnualInt. Conf. on Multimedia, 2006, pp. 835–844.[6] R. Venkatesan, S. M. Koon, M. H. Jakubowski, andP. Moulin, “Robust image hashing,” in ICIP’00 - IEEEInt. Conf. on Image Processing, Vancouver, Sept. 2000.[7] P. Brasnett and M. Bober, “Fast and robust image iden-tification,” in ICPR08, 2008, pp. 1–5.[8] B. Coskun, B., and N. D. Memon, “Spatio-temporaltransform based video hashing,” IEEE Tran. on Multi-media, vol. 8, no. 6, pp. 1190–1208, 2006.[9] A. Smeaton, P. Over, and W. Kraaij, “Evaluation cam-paigns and TRECVid,” in ACM 8th Int. Workshop onMultimedia Information Retrieval, 2006, pp. 321–330.[10] MPEG Doc. No. W10154, Updated Call for Proposalson Video Signature Tools, Lausanne, Switzerland, Oct.2008.[11] B.S. Manjunath, P. Salembier, and T. Sikora, Introduc-tion to MPEG-7 - Multimedia Content Description In-terface, John Wiley And Sons, LTD, 2002.[12] Y. Rubner, C. Tomasi, and L. J. Guibas, “A metric fordistributions with applications to image databases,” inIEEE Int. Conf. on Computer Vision, Bombay, India,1998.

Identification versus CBCD: a comparison of different evaluation techniques

Abstract

Similar works

Full text

Available Versions

Archivio istituzionale della ricerca - Università di Brescia

Archivio istituzionale della ricerca - Università di Brescia