Language use is overlaid on a network of social con-nections, which exerts an influence on both the top-ics of discussion and the ways that these topics can be expressed (Halliday, 1978). In the past, efforts t

Cohen, Shay

Eisenstein, Jacob

Puniyani, Kriti

Xing, Eric P.

English

Kriti Puniyani

Jacob Eisenstein

Shay Cohen

Eric P. Xing

CiteSeerX

Social links from latent topics in Microblogs

Edinburgh Research Explorer

     Edinburgh Research Explorer                                      Social Links from Latent Topics in MicroblogsCitation for published version:Cohen, S, Puniyani, K, Eisenstein, J & Xing, EP Social Links from Latent Topics in Microblogs.Link:Link to publication record in Edinburgh Research ExplorerDocument Version:Author final version (often known as postprint)General rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 20. Feb. 2015Social Links from Latent Topics in Microblogs∗Kriti Puniyani and Jacob Eisenstein and Shay Cohen and Eric P. XingSchool of Computer ScienceCarnegie Mellon University{kpuniyan,jacobeis,scohen,epxing}@cs.cmu.edu1 IntroductionLanguage use is overlaid on a network of social con-nections, which exerts an influence on both the top-ics of discussion and the ways that these topics canbe expressed (Halliday, 1978). In the past, efforts tounderstand this relationship were stymied by a lackof data, but social media offers exciting new oppor-tunities. By combining large linguistic corpora withexplicit representations of social network structures,social media provides a new window into the inter-action between language and society. Our long termgoal is to develop joint sociolinguistic models thatexplain the social basis of linguistic variation.In this paper we focus on microblogs: internetjournals in which each entry is constrained to a fewwords in length. While this platform receives high-profile attention when used in connection with ma-jor news events such as natural disasters or polit-ical turmoil, less is known about the themes thatcharacterize microblogging on a day-to-day basis.We perform an exploratory analysis of the con-tent of a well-known microblogging platform (Twit-ter), using topic models to uncover latent semanticthemes (Blei et al., 2003). We then show that theselatent topics are predictive of the network structure;without any supervision, they predict which othermicroblogs a user is likely to follow, and to whommicrobloggers will address messages. Indeed, ourtopical link predictor outperforms a competitive su-pervised alternative from traditional social networkanalysis. Finally, we explore the application of su-pervision to our topical link predictor, using regres-sion to learn weights that emphasize topics of par-∗We thank the reviewers for their helpful suggestions andBrendan O’Connor for making the Twitter data available.ticular relevance to the social network structure.2 DataWe acquired data from Twitter’s streaming “Garden-hose” API, which returned roughly 15% of all mes-sages sent over a period of two weeks in January2010. This comprised 15GB of compressed data; weaimed to extract a representative subset by first sam-pling 500 people who posted at least sixteen mes-sages over this period, and then “crawled” at most500 randomly-selected followers of each of theseoriginal authors. The resulting data includes 21,306users, 837,879 messages, and 10,578,934 word to-kens.Text Twitter contains highly non-standard orthog-raphy that poses challenges for early-stage text pro-cessing.1 We took a conservative approach to tok-enization, splitting only on whitespaces and apostro-phes, and eliminating only token-initial and token-final punctuation characters. Two markers are usedto indicate special tokens: #, indicating a topic (e.g.#curling); and @, indicating that the message isaddressed to another user. Topic tokens were in-cluded after stripping the leading #, but address to-kens were removed. All terms occurring less than50 times were removed, yielding a vocabulary of11,425 terms. Out-of-vocabulary items were clas-sified as either words, URLs, or numbers. To ensurea fair evaluation, we removed “retweets” – when auser reposts verbatim the message of another user– if the original message author is also part of the1For example, some tweets use punctuation fortokenization (You look like a retiredpornstar!lmao) while others use punctuation insidethe token (lOv!n d!s th!ng call3d l!f3).Figure 1: Mean rank of test links (lower is better), reported over 4-fold cross-validation. Common-neighbors is anetwork-based method that ignores text; the LDA (Latent Dirichlet Allocation) methods are grouped by number oflatent topics.dataset.Links We experiment with two social graphs ex-tracted from the data: a follower graph and a com-munication graph. The follower graph places di-rected edges between users who have chosen to fol-low each other’s updates; the message graph placesa directed edge between users who have addressedmessages to each other (using the @ symbol). Hu-berman et al. (2009) argue that the communicationgraph captures direct interactions and is thus a moreaccurate representation of the true underlying socialstructure, while the follower graph contains moreconnections than could possibly be maintained in arealistic social network.3 MethodWe constructed a topic model over twitter messages,identifying the latent themes that characterize thecorpus. In standard topic modeling methodology,topics define distributions over vocabulary items,and each document contains a set of latent topicproportions (Blei et al., 2003). However, the aver-age message on Twitter is only sixteen word tokens,which is too sparse for traditional topic modeling;instead, we gathered together all of the messagesfrom a given user into a single document. Thus ourmodel learns the latent topics that characterize au-thors, rather than messages.Authors with similar topic proportions are likelyto share interests or dialect, suggesting potential so-cial connections. Author similarity can be quan-tified without supervision by taking the dot prod-uct of the topic proportions. If labeled data isavailable (a partially observed network), then re-gression can be applied to learn weights for eachtopic. Chang and Blei (2009) describe such aregression-based predictor, which takes the formexp(−ηT (z¯i − z¯j) ◦ (z¯i − z¯j)− ν), denoting thepredicted strength of connection between authors iand j. Here z¯i (z¯j) refers to the expected topic pro-portions for user i (j), η is a vector of learned re-gression weights, and ν is an intercept term whichis only necessary if a the link prediction functionmust return a probability. We used the updates fromChang and Blei to learn η in a post hoc fashion, aftertraining the topic model.4 ResultsWe constructed topic models using an implemen-tation of variational inference2 for Latent Dirich-let Allocation (LDA). The results of the runwith the best variational bound on 50 topics canbe found at http://sailing.cs.cmu.edu/socialmedia/naacl10ws/. While many ofthe topics focus on content (for example, electron-ics and sports), others capture distinct languages andeven dialect variation. Such dialects are particu-larly evident in stopwords (you versus u). Struc-tured topic models that explicitly handle these twoorthogonal axes of linguistic variation are an intrigu-ing possibility for future work.We evaluate our topic-based approach for linkprediction on both themessage and follower graphs,comparing against an approach that only consid-ers the network structure. Liben-Nowell and Klein-berg (2003) perform a quantitative comparison ofsuch approaches, finding that the relatively simpletechnique of counting the number of shared neigh-2http://www.cs.princeton.edu/∼blei/lda-cbors between two nodes is a surprisingly compet-itive predictor of whether they are linked; we callthis approach common-neighbors. We evaluate thismethod and our own supervised LDA+regression ap-proach by hiding half of the edges in the graph, andpredicting them from the other half.For each author in the dataset, we apply eachmethod to rank all possible links; the evaluationcomputes the average rank of the true links that wereheld out (for our data, a random baseline wouldscore 10653 – half the number of authors in thenetwork). As shown in Figure 1, topic-based linkprediction outperforms the alternative that consid-ers only the graph structure. Interestingly, post hocregression on the topic proportions did not consis-tently improve performance, though joint learningmay do better (e.g., Chang and Blei, 2009). The text-based approach is especially strong on the messagegraph, while the link-based approach is more com-petitive on the followers graph; a model that cap-tures both features seems a useful direction for fu-ture work.ReferencesD. Blei, A. Ng, and M. Jordan. 2003. Latent Dirich-let allocation. Journal of Machine Learning Research,3:993–1022.J. Chang and D. Blei. 2009. Hierarchical relational mod-els for document networks. Annals of Applied Statis-tics.M.A.K. Halliday. 1978. Language as social semiotic:The social interpretation of language and meaning.Hodder Arnold.Bernardo Huberman, Daniel M. Romero, and Fang Wu.2009. Social networks that matter: Twitter under themicroscope. First Monday, 14(1–5), January.D. Liben-Nowell and J. Kleinberg. 2003. The linkprediction problem for social networks. In Proc. ofCIKM.Conversations(outof10) Chit-chatgeting ready go to dinner and movie with some friends!!!!!!!!! -OOV-, my, to, and, in, with, at, for, the, a, new, going,day, work, so, am, night, today, time, isSlang @MsGumdrop lmao das ok now u 1 step smarter den da nextperson onlything i remember is math n science smh-OOV-, u, lol, i, me, my, a, 2, im, ur, on, up, lmao, like, in,shit, its, get, got, nWeather @JPRennquist concerned on the amount of ice that may formover Lake Superior, that would play a bigger role on ourweather by March thru Maythe, a, -OOV-, in, it, of, on, is, to, ’s, and, cold, for, this,snow, just, here, good, morning, today, weatherOptimism Feeling the hope that a new day brings. -OOV-, for, the, of, in, thanks, ’s, to, great, at, a, it, out,-URL-, today, new, so, 2, you, seeSPAM(9) Socialmediasay hi to some new #followfriday frenz: @Kdark74@MAKEAMILLION1 @janettefuller @The Scallywags@therealamaru @mslegalhelpyou, follow, thanks, ff, for, thank, rt, the, to, back, your,me, are, followfriday, a, great, u, -OOV-, is, hiWeightlossStart Losing Weight NOW. Shed holiday pounds safelyand effectively. Free Trial Offers. Act today! sponhttp://tinyurl.com/yd9m8po-URL-, -OOV-, free, weight, to, for, the, on, loss, and,your, get, save, off, a, with, fitness, diet, at, lose, fatNews(6) Financial Market breadth is very negative; a bearish indication seehttp://bit.ly/lPIyW #News #Trading #Investing #Finance #SPX#Mkt #Business #Stocks-URL-, for, -OOV-, business, to, in, home, real, the, estate,credit, a, mortgage, on, finance, economy, money, market,andGadgets The Switch From iPhone To Android, And Why Your FirstImpression Is Wrong #gadgets http://bit.ly/4GTPmo-URL-, -OOV-, the, to, iphone, google, for, and, of, a, on,apple, ces, app, one, ’s, in, with, nexus, newMarket(4)Jobs #jobs Oracle Project Manager 12i in Denver, Coloradohttp://jobshouts.com/job/10128/-OOV-, -URL-, job, jobs, for, a, to, me, and, follow, the,in, need, looking, we, php, at, little, rock, iGadgets don’t miss : Canon PowerShot SD960 IS Digital ELPH Cam-era (Pink) + 4GB SD Card + Case + NB-4L + AccessoryKithttp://bit.ly/60JFM2digital, card, 2.0, new, usb, reader, in, one, memory, multi,camera, sdhc, lcd, zoom, mp, -URL-, 2.4, 4x, coffee, pinkEntertain.(3)Television RT @joshmmartin: How about nearly everyone on the Atlantaepisode of #americanidol tonight w/ a country accent hasn’tbeen from GAha, -OOV-, la, idol, flu, a, american, boxing, of, the, show,fight, castle, floyd, hee, on, swine, americanidol, you,dudeFilm andmusicRARE Michael Jackson interview with Soulbeat 1979http://bit.ly/RBsDk Pls ReTweet & #FollowFriday us ! MJ mu-sic 4ever!-OOV-, -URL-, the, ’s, music, of, new, in, and, a, michael,to, on, movie, for, jackson, with, guitar, by, isLifestyle(7) Prayer Lord someone is in that rubble in Haiti who needs to hang onand believe for a miraculous rescue. Breathe on them!Be theirrescue!#tcot#pray-OOV-, for, god, please, love, and, my, haiti, pray, in, to,the, you, praying, of, her, help, hugs, all, is, jesusFood Good Food, Good Wine and a Bad Girl: A Slice ofHeaven - Chocolate Banana Bread: Chocolate Banana Breadhttp://bit.ly/5LiSwp-OOV-, -URL-, the, to, and, a, in, for, haiti, food, of, ’s,with, our, we, recipe, cooking, on, at, wineTable 1: Examples from the 50 topics found by the LDAmodel. There were 8 topics from different languages (French,Dutch, Indonesian, Italian, Portuguese, Spanish, German, mixture of English and Indoneisan). Two topics seemed tocapture stop-words in different communities, and one topic was a mixture of travel and spam. The remaining 39 topicswere manually classified into the six broad categories shown. Sample topics are shown above (a detailed analysis willbe found at http://www.cs.cmu.edu/∼kpuniyan/Twitter-topics.html). In each broad category, thefirst column roughly summarizes the topic, the second shows a representative tweet from the same topic, and the thirdcolumn shows the top 20 words in this topic.

Hierarchical relational models for document networks.

Language as social semiotic: The social interpretation of language and meaning. Hodder Arnold.

Social Links from Latent Topics in Microblogs

Abstract

Similar works

Full text

Available Versions

CiteSeerX

Edinburgh Research Explorer