Current multimedia applications in Web 2.0 have generated a massive amount of multimedia resources, but most search results for multimedia resources still focus on the whole re-source level. Media fragments expose the inside content of multimedia resources for annotations, but they are yet fully explored and indexed by major search engines. W3C has published Media Fragment 1.0 as a standard way to describe media fragments on the Web. In this proposal, we make use of Google's Ajax Application Crawler to index media fragments represented by Media Fragment URIs. Each media fragment with related annotations will have an individual snapshot page, which could be indexed by the crawler. Initial evaluation has shown that the snapshot pages are successfully fetched by Googlebot and we are expecting more media fragments to be indexed using this method, so that the search for multimedia resources would be more efficient

Li, Yunjia

Wald, Mike

Wills, Gary

English

Southampton (e-Prints Soton)

Let Google Index Your Media FragmentsYunjia Li,Mike Wald and Gary WillsSchool of Electronics and Computer ScienceUniversity of SouthamptonUK{yl2,mw,gbw}@ecs.soton.ac.ukABSTRACTCurrent multimedia applications in Web 2.0 have generateda massive amount of multimedia resources, but most searchresults for multimedia resources still focus on the whole re-source level. Media fragments expose the inside content ofmultimedia resources for annotations, but they are yet fullyexplored and indexed by major search engines. W3C haspublished Media Fragment 1.0 as a standard way to describemedia fragments on the Web. In this proposal, we make useof Google's Ajax Application Crawler to index media frag-ments represented by Media Fragment URIs. Each mediafragment with related annotations will have an individualsnapshot page, which could be indexed by the crawler. Ini-tial evaluation has shown that the snapshot pages are suc-cessfully fetched by Googlebot and we are expecting moremedia fragments to be indexed using this method, so thatthe search for multimedia resources would be more ecient.Categories and Subject DescriptorsH.5.1 [Multimedia Information Systems]: Audio, Videoand Hypertext Interactive SystemsGeneral TermsWeb,Search,DesignKeywordsmedia fragment,annotation,microdata,Google search1. INTRODUCTIONThe term\media fragment"refers to the inside content ofmultimedia objects, such as a certain area within an image,or a ve minutes segment within a one-hour video. Withthe rapid development of multimedia applications, such asYouTube and Flickr, end users could easily upload, shareand tag the multimedia resources online. However, mostsearch results for multimedia resources still focus on thewhole resource level and media fragments are not fully in-dexed yet. Multimedia resources are seldom displayed aloneon the Web page. Usually, a multimedia player is embeddedin the replay page (also called\landing page") together withall media fragments, metadata and annotations. Search en-gines thus will not be able to distinguish which annotation isCopyright is held by the author/owner(s).WWW2012 Developer Track, April 18-20, 2012, Lyon, France.related to which media fragment and it leads to the dicultyof searching and reusing the inside content of multimedia.The W3C Media Fragment Working Group has proposedMedia Fragment URI 1.0 (MFURI) [6], which denes hashURI syntax to address media fragments from temporal, spa-tial, track and named section dimensions. It has been statedby Troncy et al. [6] that \enabling the addressing of mediafragments ultimately creates a means to attach annotationsto media fragments". In this proposal, we introduce a modelto enable Google to index media fragments using relatedannotations, so that they could be found by traditional key-word search. This model uses MFURI syntax and Google'scrawling infrastructure for Ajax applications. A demo is im-plemented on top of the Synote system [3]. Synote is a Web2.0 application, which allows users to embed audio-visualresources from other domains and make synchronised anno-tations. We mainly implement the temporal dimension foraudio-visual resources, but the concept could be easily ex-tended to spatial and other dimensions dened in MFURI.Some initial evaluation results have shown that the annota-tions related to media fragments are indexed by Google andusers could play the media fragment directly from the linksprovided by Google search results.2. PROBLEM ANALYSISThe online presence of media fragments is very poor atthe moment. This is partially because a large portion ofapplications do not provide media fragments at all. Thetags, descriptions and other forms of annotations are on thewhole multimedia level. Another important reason is thateven though some applications provide synchronised anno-tations, they are loaded together with the whole multime-dia resource on the same logical page. This is reasonableas it provides an interactive experience for users. For ex-ample, TED Talks1 and YouTube's interactive transcript2allow users to click on the transcript block and the mediaplayer embedded on the page will start playing from thattime point.This user-friendly function is not search-engine-friendlyfor two reasons. Firstly, most search engines only fetch pagesas a direct response from the server. So any dynamicallygenerated content on the client-side is ignored. YouTubeload interactive transcript by Ajax, so the transcript willnot be indexed by Google from the replay page. TED doesnot have this problem because the transcript is generated1http://www.ted.com/talks2http://goo.gl/t1nMjFigure 1: Google Ajax Application Crawlerby server-side script. Secondly, as media fragments and an-notations share the same HTML page with the whole multi-media resource and annotations, search engines cannot nda unique logical page for each media fragment. When akeyword is searched, the search engine can only return thewhole page and users will lose track of which media fragmentis related to the keyword. This also causes accessibility prob-lems for devices with low bandwidth, because much tracis wasted on downloading unnecessary data.One solution to these two problems is slicing the wholepage into dierent pages according to media fragments, butthe interactive experience will be lost in that users cannotwatch dierent media fragments on the same page. Theacceptable solution for both multimedia applications andend users must satisfy the following criteria: Keyword search should only return the documents di-rectly related to the media fragments, i.e. all the con-tent in the document should annotate this media frag-ment. It is better if the media fragment could be high-lighted in the search results. The interactive experience should be kept. But whenusers click the link in the search result, the media frag-ment corresponding to the link should be highlighted Few changes to the server are required3. IMPLEMENTATIONGoogle has developed a framework to crawl Ajax applica-tions (Figure 1). If the \hashbang" token (\#!") is includedin the original URL3, Google crawler will know that thispage contains Ajax content. Then the crawler will request\ugly URL". On receiving this "Ugly URL" request, theserver can return the snapshot page after the dynamic in-formation is fully generated by javascript. The content inthe snapshot page will be indexed in for the original\prettyURL".In this framework, the crawler does not care how theserver generates the snapshot page, so we make use of this2http://goo.gl/dPc813The HTTP protocol and domain names will be ignored inthe URLs for short in this paperframework by keeping two sets of pages. We do not discardthe existing replay pages, but we duplicate this page by cut-ting it into many snapshot pages based on the time span de-ned in\ escaped fragment "parameter in the\ugly URL".Each snapshot page contains metadata and annotations onlyrelated to the media fragments. The hash in MFURI syn-tax, however, is replaced by hashbang as\#!"is the requiredtoken by the crawler. When the request URL containing\ escaped fragment ", the server returns the snapshot page.If not, the normal replay page will be returned.Figure 2 explains how this model could be used to indexmedia fragments. As a pre-condition, the time informationmust be available for the developers so that the MFURI canbe constructed. The returned page in step 4 only containskeywords related to fragment \t=3,7". In the Google in-dex, the \pretty media fragment URLs" are associated withthe snapshot page. So what Google actually indexed is theURL of the replay page with hashbang and MFURI syn-tax attached. Step 8 still returns the whole page, but instep 9, the fragment will be passed to the URL representingthe real location or the service which delivers the multime-dia le. For example, if the request URL is example/re-play/1#!t=3,7, the fragment \#t=3,7" will be attached atthe back of example2/1.ogv, which is the video embedded inthe replay page. Hashbang is not a valid syntax in MFURIspecication, so developers need to parse the informationin the hashbang URL before attaching the fragment to thethe URL of the actual multimedia le. Then we control theembedded player to play the fragment from 3s to 7s usingjavascript and the corresponding annotations are highlightedstraight away. In this case, step 6 will return the URL of themedia fragment instead of the replay page. This design notonly makes sure media fragments are indexed precisely withthe keywords related to it, but also preserves the existinguser interface and the interactive experience.On the server side, necessary changes needs to be made.The rst one is the programme to detect\ escaped fragment "parameter in the request URL from Google and redirect itto the snapshot generation programme, which is the secondprogramme we need to add to the server. The snapshotpage does not need to be user-friendly, but the metadataand annotations related to the media fragment should bepresented in a well-structured manner. We also embed Mi-crodata [2] dened in schema.org4, into the page. Each me-dia fragment is dened as either\Audio"or\Video", and theannotations are dened as \keywords" and other propertiesin schema.org.The third component developers need to write is the pro-gramme to highlight the corresponding media fragment whenthe page is opened. For this demo, we developed SynoteMedia Fragment Player5 (SMFP) to full the visual outputdened in Media Fragment User Agent (UA) Test Cases6on the temporal dimension of MFURI. MFURI should beprocessed via HTTP protocol with the help of \smart useragents", \smart servers" and proxy caches [4]. Even thoughsome UAs support part of the functions7 dened in MFURI,the general support for media fragment highlighting is verylimited. So we developed SMFP to play the temporal di-4http://schema.org5http://goo.gl/NO0t76http://goo.gl/yexpy7https://bugzilla.mozilla.org/show_bug.cgi?id=648595Figure 2: The model to improve media fragment presence based on Google crawlermension of media fragments for dierent formats of videoand audio resources. In the future, if major browsers couldnatively support the media fragment highlighting and re-trieval dened in MFURI specication, it would not be nec-essary for developers to implement this component. Exceptfor these changes, we also need to include the newly createdhashbang URLs in the sitemaps so that they could be easilyfound out by Google.4. EVALUATION AND DISCUSSIONWe implemented the model demonstrated in Figure 2 inSynote and pre-dened some media fragments and annota-tions. Sitemaps containing URIs like replay/1#!t=3,7 havebeen submitted to Google for indexing. To have a quickevaluation of what Googlebot fetches from these URIs, wesubmit several URIs into Google Web Master Tools8. Theresult shows that on fetching this URL:http://linkeddata.synote.org/synote/recording/replay/36513#!t=00:00:01.000,00:00:14.000The snapshot page is returned with annotations only relatedto fragment \#t=00:00:01.000,00:00:14.000". We also putsome Microdata in the snapshot page and the semantic in-formation can be successfully recognised by Live Microdata9and Linter Structured Data10. If the keyword\Terrace The-ater", for example, is searched in Google, the snapshot pagecan be successfully found in the search results instead of thewhole replay page. When we click the link in search results,the Synote Player page in Figure 3 will be opened and thebutton in top-left corner indicates that a media fragment isrequested. On opening the page, the video will start play-ing from 1s to 14s and the related annotation on the rightcolumn will be highlighted.As another example for evaluation, we search the sen-tence \All kinds of conceptual things, they have names nowthat start with HTTP" in Google. This sentence is includedin the transcript of the video resources for both TED Talksand Synote. The rst result in Figure 4 is from TED Talks.Clicking on the link will open the replay page of the talk,but you still need to manually nd the sentence in the in-teractive transcript. So even though the sentence logically8http://www.google.com/webmasters/tools/9http://foolip.org/microdatajs/live/10http://linter.structured-data.org/Figure 3: Screenshot of Synote replay pageannotates part of the video, the search engine still associatesit with the whole document, i.e. the replay page. The sec-ond result in Figure 4 indicates that this sentence in Synoteis directly related to the fragment dened in the replay page.On clicking this search result, the video embedded in the re-play page will start playing from the start time of this mediafragment.The main idea of this model is generating a set of search-engine-friendly snapshot pages on the y according to thetime span dened in the fragment of request URL. For ap-plications heavily relied on Ajax or Flash, it is useful to keeptwo sets of pages for both rich user interaction and searchengine optimisation (SEO). Considering Web Content Ac-cessibility Guidelines11, it is also a good practice to providesuch snapshot pages because the interactive feature usuallyis not accessible to screen readers and keyboard users sinceit depends much on javascript.This model relies on hashbang URLs, which has beenwidely used in Facebook and Twitter to allow the index-ing of Ajax content. But hashbang is claimed not to be a11http://www.w3.org/TR/WCAG20/Figure 4: Search results comparison between TEDTalks and Synotegood practice of URL design12 and it should be replacedby HTML5 PushState13. Twitter recently also announcedthat they would remove all the hashbang URLs14. In orderto use Google Ajax Application Crawler, we have to imple-ment the hashbang URLs, but we also need to consider thedisadvantages of such URLs in the future work.Another limitation of this model is that it only works forGoogle and therefore further work is required to investigatesimilar solutions for other search engines, such as Yahoo!and Bing. The general solution of media fragment index-ing might be embedding Rich Snippet [5], such as Micro-data and RDFa [1], into the page so that search engines canhighlight them in the search results. But currently, majorsearch engines are still far from reaching an agreement onthe vocabularies to describe media fragments.5. CONCLUSIONMedia fragment is important on the Web as it describesthe inside content of multimedia resource. However, majorsearch engines currently are not able to index media frag-ments using the related annotations. We analysed this prob-lem from both multimedia application and search engine'spoint of views. The key problem is that media fragmentsand annotations do not have their own page on the Web andthey share the same page as the parent resource. We devel-oped a model, which uses MFURI syntax and Google AjaxApplication Crawler, to let Google index snapshot pages foreach media fragment. In this way, media fragments willhave their own links in the search results and on followingthe links, users still visit the same replay pages as before.We have evaluated the implementation and the result showsthat the media fragments could be indexed by Google. Weenvision that by using this model, more and more mediafragments and annotations will be indexed and the searchfor multimedia resources will be more ecient. The screen-cast15 of this evaluation and the live demo of Synote16 areavailable online.6. REFERENCES[1] B. Adida and M. Birbeck. RDFa Primer, Oct. 2008.http://www.w3.org/TR/xhtml-rdfa-primer/.12http://goo.gl/xfvwT13http://diveintohtml5.info/history.html14http://storify.com/timhaines/hashbang-conversation15http://goo.gl/4zl1V16http://linkeddata.synote.org/synote/[2] I. Hickson. HTML Microdata, Feb. 2012.http://dev.w3.org/html5/md/.[3] Y. Li, M. Wald, G. Wills, S. Khoja, D. Millard,J. Kajaba, P. Singh, and L. Gilbert. Synote:development of a web-based tool for synchronizedannotations. New Review of Hypermedia andMultimedia, 17(3):295{312, 2011.[4] E. Mannens, D. Van Deursen, R. Troncy, S. Pfeier,C. Parker, Y. Lafon, J. Jansen, M. Hausenblas, andR. Van de Walle. A uri-based approach for addressingfragments of media resources on the web. MultimediaTools and Applications, pages 1{25.10.1007/s11042-010-0683-z.[5] T. Steiner, R. Troncy, and M. Hausenblas. How Googleis using Linked Data Today and Vision For Tomorrow.In S. Auer, S. Decker, and M. Hauswirth, editors,Linked Data in the Future Internet 2010,Ghent,Belgium, 2010.[6] R. Troncy, E. Mannens, S. Pfeier, and D. V. Deursen.Media fragments URI 1.0 (basic), Mar. 2012.http://www.w3.org/TR/media-frags/.

Let Google index your media fragments

Abstract

Similar works

Full text

Available Versions

Southampton (e-Prints Soton)