HU, Meishan

LIM, Ee Peng

SUN, Aixin

Institutional Knowledge at Singapore Management University

Singapore Management UniversityInstitutional Knowledge at Singapore Management UniversityResearch Collection School Of Information Systems School of Information Systems7-2008Searching blogs and news: A study on popularqueriesAixin SUNNanyang Technological UniversityMeishan HUEe Peng LIMSingapore Management University, eplim@smu.edu.sgFollow this and additional works at: https://ink.library.smu.edu.sg/sis_researchPart of the Computer Sciences Commons, and the Digital Communications and NetworkingCommonsThis Conference Paper is brought to you for free and open access by the School of Information Systems at Institutional Knowledge at SingaporeManagement University. It has been accepted for inclusion in Research Collection School Of Information Systems by an authorized administrator ofInstitutional Knowledge at Singapore Management University. For more information, please email libIR@smu.edu.sg.CitationSUN, Aixin; HU, Meishan; and LIM, Ee Peng. Searching blogs and news: A study on popular queries. (2008). 31st AnnualInternational ACM SIGIR Conference on Research and Development on Information Retrieval (SIGIR 2008). Research Collection SchoolOf Information Systems.Available at: https://ink.library.smu.edu.sg/sis_research/420Searching Blogs and News: A Study on Popular QueriesAixin Sun, Meishan Hu, and Ee-Peng LimSchool of Computer EngineeringNanyang Technological University, Singapore{axsun, hu0004an, aseplim}@ntu.edu.sgABSTRACTBlog/news search engines are very important channels toreach information about the real-time happenings. In thispaper, we study the popular queries collected over one yearperiod and compare their search results returned by a blogsearch engine (i.e., Technorati) and a news search engine(i.e., Google News). We observed that the numbers of hitsreturned by the two search engines for the same set of querieswere highly correlated, suggesting that blogs often providecommentary to current events reported in news. As manypopular queries are related to some events, we further ob-served a high cohesiveness among the returned search resultsfor these queries.Categories and Subject Descriptors: H.3.3 [Informa-tion Storage and Retrieval] Information Search and RetrievalGeneral Terms: ExperimentationKeywords: Blog, News, Search engine, Popular queries,Search results1. INTRODUCTIONSearch engines dedicated for blogs and news enable usersto reach the most up-to-date information. Very often, the 1 10 1001K10K100K1M10M100Myoutubemyspaceron paulgoogleparis hiltonnoeliaiphonegalilea montijobritney spearsbebofacebookstudivzmusicjoostweb 2.0second lifevideoutubetammy nyptwitterFrequency / Number of hits Query frequencyNum of hits (TR)Num of hits (GN)Figure 1: Top 20 queries with highest frequenciesCopyright is held by the author/owner(s).SIGIR’08, July 20–24, 2008, Singapore.ACM 978-1-60558-164-4/08/07.popular queries submitted to blog/news search engines serveas good indications of the information needs of web usersat the time of searching. In this paper, we ﬁrst give anoverview of the popular queries collected from Technorati(http://www.technorati.com/), a well-known blog searchengine, over one year period. We then compare the searchresults to these popular queries returned by Technorati andGoogle News (http://news.google.com/), the latter is apopular news search engine. The results from the two searchengines represent the information provided by bloggers andjournalists respectively. For each query, the search resultsare studied from two perspectives: (i) the number of hits,and (ii) the cohesiveness of the returned results. Such ananalysis will facilitate many web/text mining tasks, includ-ing trend analysis in blogosphere, event detection, etc.2. DATA COLLECTIONWithout the access to query logs of commercial blog/newssearch engines, we chose to collect the popular queries pub-lished by Technorati. From 08 Nov 2006 1AM to 30 Nov2007 10PM, we crawled the top 15 most popular queries1from Technorati every 3 hours. Each of the 15 popularqueries was extracted and submitted to two search engines,i.e., Technorati and Google News. The number of hits andthe top 50 results from each search engine were recorded.Among the 46,545 popular queries scheduled to be crawledin the given period, we managed to collect 44,594 (or 95.81%).We missed some queries (and their search results) due to oc-casional network/power outage and errors in query extrac-tion due to page format change. Note that, the same querymay be popular at various time points but were evaluatedmultiple times as the search results could be diﬀerent at eachtime. After punctuation removal (e.g., “second life”becomessecond life), the number of distinct query keywords is 1530.3. DATA ANALYSIS3.1 Popular QueriesThe query frequency of a popular query is deﬁned by thenumber of its occurrences in our dataset. The distributionof popular queries with respect to their query frequencies isplotted in Figure 2(a) with log-scale axes. It is observed that63% of queries have query frequencies smaller than 10 anda small set of queries appears to be popular for fairly longperiods. Among the latter, youtube has query frequency of2766, being popular for about 346 days over the 388 dayscovered by our dataset.The top 20 queries with the highest query frequencies (see1http://www.technorati.com/pop/729 1 2 4 8 16 32 64 128 256 1  10  40  160  640  2560Number of queriesQuery frequency(a) Query distribution 1 10 1001K10K100K1M10M 1  10  40  160  640  2560Number of hitsQuery frequencyTRGNAvg(TR)Avg(GN)(b) Number of hits 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 1  10  40  160  640  2560Search result cohesivenessQuery frequencyGNTRAvg(GN)Avg(TR)(c) Search result cohesivenessFigure 2: Popular queriesFigure 1) can be broadly grouped into three categories: web-site, person name, and topic/concept.Almost half of the top 20 queries, i.e., YouTube, MySpace,Google, Bebo, Facebook, StudiVZ, Joost, Second Life, andTwitter, are websites providing services mainly on social-networking, sharing, searching, and gaming (see their cor-responding articles in Wikipedia2). Quite a few top queriesare person names, including Ron Paul, Paris Hilton, Noelia,Galilea Montijo, Britney Spears, and Tammy NYP. Prob-ably because of their popularity, articles about them havebeen created in Wikipedia. Many of these persons becamepopular due to some events in which they involved (e.g.,presidential election for Ron Paul). A few queries are top-ics/concepts, such as Web 2.0, video, and music, suggest-ing that many users are interested in the discussions amongbloggers on these topics. Finally, iphone is a product re-leased in 2007 which received much public attention; “utube”may refer to (i) a misspelt “youtube”, or (ii) the lawsuit casebetween utube.com and youtube.com.Compared to the top 20 most popular queries for Blogdig-ger.com reported in [1], the major diﬀerences in our obser-vation is the large number of websites among top popularqueries. All these websites are highly dynamic in nature.Web users may want to discover through blog search “whatother people say” about these websites or the popular top-ics discussed at these websites (e.g., an interesting videosurfaced on YouTube). Among the 1530 distinct popularqueries in our dataset, many are named entities and areevent-related, which is consistent with the earlier observa-tion [1]. For instance, harry potter (for the release of thebook), saddam hussein (mainly for the execution video), andWindows Vista (for the release of the software).3.2 Number of HitsFigure 1 also reports the average number of hits wheneach query was submitted to Technorati (TR) and GoogleNews (GN). It is interesting to observe that, the numbersof hits from TR and GN are highly correlated (coeﬃcient =0.965). The average numbers of hits for all queries againsttheir query frequencies are plotted in Figure 2(b). Similarly,high correlation (coeﬃcient=0.798) between the numbers ofhits is observed for all popular queries. That is, if a query(e.g., a person name) receives high coverage among news, itis likely to be mentioned by a large number of blog posts. Inother words, many blogs provide commentary or additionalinformation about current events. For a given query, thenumber of hits from TR is always larger than that from2http://en.wikipedia.org/GN, as shown in Figure 2(b). The large number of inputsources (i.e., millions of blogs traced by TR) could be thereason.Observed from Figure 2(b), queries having high query fre-quency do not always lead to more hits from the two searchengines. For TR, queries having query frequency in therange from 40 to 160 have higher probability of receivingsmall number of hits compared to other queries. For GN,many queries having query frequency larger than 40 matchvery few news articles. One possible reason is that suchqueries are related to some bursty events lasting for onlya short period. At the time of searching, an event maybein its very early stage with few blog posts (or news arti-cles) matching it. It is also observed that some queries areless covered in news articles despite their popularity. Forinstance, among the top 20 most popular queries shown inFigure 1, 4 of them, i.e., Galilea Montijo, StudiVZ, utube,and Tammy NYP, have less than 10 hits on average fromGoogle News.3.3 Search Result CohesivenessGiven a query searched at a particular time, the cohe-siveness of its search result is deﬁned by the average cosinesimilarity between each of the 50 returned documents (i.e.,the snippets) to their centroid. Figure 2(c) plots the aver-aged cohesiveness of all queries having the same query fre-quency. On average, results from GN demonstrated highercohesiveness than that from TR. For GN, the search resultcohesiveness for queries having query frequency between 40and 160 are higher than others. Recall that most of thesequeries are likely related to some bursty events. The resultsreturned are therefore similar to each other, as most of themreport the same event. Comparing the same set of queries,Figure 2(c) shows that the topics discussed in blog posts aremuch more diverse than that in news articles.4. CONCLUSIONIn this paper, we report the observations made from pop-ular queries published by Technorati over one year period.The search results to these popular queries returned by Tech-norati and Google News are compared by examining thenumber of hits and the cohesiveness of the returned doc-uments. Our observations include: many of the popularqueries are event-related; there exists high correlation be-tween the numbers of hits returned from Technorati andthat from Google News for these popular queries.5. REFERENCES[1] G. Mishne and M. de Rijke. A study of blog search. In Proc. ofECIR, pages 289–301, London, UK, 2006.730

Searching blogs and news: A study on popular queries

https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=1419&amp;context=sis_research

Searching blogs and news: A study on popular queries

Abstract

Similar works

Full text

Available Versions

Institutional Knowledge at Singapore Management University