De Pessemier, Toon

Dooms, Simon

Martens, Luc

Simon Dooms

Toon De Pessemier

Luc Martens

Crossref

English

Mining cross-domain rating datasets from structured data on twitter

Ghent University Academic Bibliography

Mining Cross-Domain Rating Datasets from StructuredData on TwitterSimon DoomsiMinds-Ghent UniversityG. Crommenlaan 8, box 201Ghent, BelgiumSimon.Dooms@UGent.beToon De PessemieriMinds-Ghent UniversityG. Crommenlaan 8, box 201Ghent, BelgiumToon.DePessemier@UGent.beLuc MartensiMinds-Ghent UniversityG. Crommenlaan 8, box 201Ghent, BelgiumLuc1.Martens@UGent.beABSTRACTWhile rating data is essential for all recommender systemsresearch, there are only a few public rating datasets avail-able, most of them years old and limited to the movie do-main. With this work, we aim to end the lack of rating databy illustrating how vast amounts of ratings can be unam-biguously collected from Twitter. We validate our approachby mining ratings from four major online websites focusingon movies, books, music and video clips. In a short miningperiod of 2 weeks, close to 3 million ratings were collected.Since some users turned up in more than one dataset, webelieve this work to be amongst the first to provide a truecross-domain rating dataset.Categories and Subject DescriptorsE.0 [Data]: General; H.1.2 [Information Systems]: Mod-els and Principles—User/Machine Systems, Human Infor-mation ProcessingKeywordsDataset, mining, Twitter, Goodreads, YouTube, Pandora,IMDb1. INTRODUCTIONResearchers are in constant need of data. They need datafor analysis reasons, the validation of algorithms or to driveexperiments. This is especially true in the field of recom-mender systems, where rating data serves as input dataand is therefore a crucial component to calculate recommen-dations. Despite their importance, rating datasets are notabundantly available. Ratings are often considered privateuser data and therefore most online platforms do not makethem publicly available. The lack of rating datasets has con-tributed to the growing gap between academia and industry.While academic researchers are working on algorithms andoften lack user data for testing, industry has data and users,but needs the algorithms to improve their services.To circumvent the lack of public ratings, most recom-mender systems research employs one of the few datasetsthat are available e.g., the MovieLens [1] or the Netflixdataset [2, 3]. Both datasets are slowly becoming outdated(i.e., originating from 2005 and 2007) and furthermore arerestricted to the movie item domain. The most recent moviein the MovieLens 100K dataset, was released as far back as1998. Nevertheless, this dataset is still widely used in recentliterature (e.g., [4, 5]).We aim to end the lack of up-to-date rating data by illus-trating how large amounts of ratings can be unambiguouslycollected from current-day popular websites by mining theTwitter platform for specific pre-formated tweets. In previ-ous work [6], we presented a new movie rating dataset called‘MovieTweetings’, based on a similar approach. With thiswork, we generalize our method to other item domains asbooks, music and video clips, and provide the tools to allowresearchers to collect and build cross-domain rating datasetsfor themselves.2. STRUCTURED DATA ON TWITTERWith over 300 billion tweets in total1, the data avail-able through the Twitter platform is enormously abundant.Twitter users (at the time of writing) post an average of500 million tweets every day, thereby exchanging opinions,ideas, links, jokes and many other types of information. Inthis work, we mine rating datasets by filtering and extract-ing relevant preference indicators contained in these tweets.We avoid dataset ambiguity and the need for complex nat-ural language processing by focusing on structured tweetsoriginating from the social share feature integrated on manypopular online platforms. We illustrate our method focus-ing on four major online platforms with very divergent itemdomains: IMDb2 (movies), Goodreads3 (books), Pandora4(music), YouTube5 (video clips).2.1 Movies - IMDbThe first online platform focuses on movies. IMDb (Inter-net Movie Database) is one of the biggest and most popularwebsites concerning movies. It offers a wide variety of moviecontent information and lets users rate movies on a 10-starscale. Although users have the option to make their ratingspublic, few users actually do this, and so individual ratinginformation is usually not accessible. Users of the IMDb mo-bile app for iOS, are however provided with the option totweet their rating. The app proposes a pre-formatted tweetwith the following structure.I rated <Title> <Rating>/10 #IMDb<Link to IMDb movie page>1http://bit.ly/19YSAmo2http://www.imdb.com3http://www.goodreads.com4http://www.pandora.com5http://www.youtube.comQuerying the Twitter API6 for the term ‘I rated #IMDb’will result in tweets containing movie ratings originatingfrom IMDb. From the tweet the following data fields canbe extracted:• User (Twitter user identifier)• Movie title• Rating (10-star scale)• IMDb URL of the movieThe URL contains the unique IMDb identifier for themovie, which allows easy disambiguation of the movies andfurthermore enables extraction of additional metadata fromthe movie’s IMDb page. In previous work [6], we discussedhow the Twitter API could be queried on a daily basis forsuch tweets, which resulted in a publicly available movierating dataset called ‘MovieTweetings’. The dataset offersthe extracted ratings in a format similar to the MovieLensdataset and also integrates movie genre data.2.2 Books - GoodreadsThe second online platform we extract ratings from (throughTwitter), is Goodreads. Goodreads is one of the largest web-sites for books discussion and discovery. Readers can reviewbooks and receive recommendations based on their personaltaste. This website also offers to tweet about the review orrating a reader has provided. On Goodreads the followingtweet template structure is used.<Rating> of 5 stars to <Title> by<Author> <Link to review on Goodreads>To obtain the tweets originating from the Goodreads web-site, we query the Twitter API for ‘of 5 stars to’. From thetweet the following information can be extracted:• User (Twitter user identifier)• Rating (5-star scale)• Book title• Book author• Goodreads URL of the reviewSince the URL refers to the review on the Goodreads web-site posted by the same user the tweet originates from, addi-tional metadata fields (e.g. Goodreads user id, book id) caneasily be extracted. Ratings provided by users on Goodreadsare publicly available, so if the Goodreads user id is known,(although not implemented in this work) all of that user’sratings could additionally be extracted from the website toexpand the ratings dataset even further.2.3 Music - PandoraMusic is the third item domain this work focuses on.There are many online services for music, one of which isthe online radio service Pandora. Using Pandora, users caneasily stream music and receive song recommendations. Thewebsite offers a similiar social share feature as we foundfor IMDb and Goodreads. Users can tweet about the songthey are currently listening to, using the following predefinedtweet format.6https://dev.twitter.com/docs/api/1.1I’m listening to "<Title>" by <Artist> onPandora <Link to song on Pandora> #pandoraVery similar to the data available from tweets originatingfrom the Goodreads website, the data fields available in thistweet format are:• User (Twitter user identifier)• Song title• Song artist• Pandora URL of the songThe difference here, is the lack of an explicit rating value.Pandora users do not rate the music, they either listen toit, or they do not. The tweets originating from the Pandoraplatform should therefore be considered implicit feedback.The query we use to get the Pandora tweets from the TwitterAPI is ‘I am listening on #pandora’. The Pandora URL isavailable, so again additional metadata (e.g., music genre)can be extracted.2.4 Video clips - YouTubeThe fourth and final platform is YouTube. YouTube iscurrently the biggest provider of short video clips on the In-ternet and is widely famous and well-known all worldwide.While YouTube used to have a 5-star rating scale, in 2009it was replaced with a thumbs up/down system because itmore closely aligned with typically observed rating behav-ior7. When users watch videos on YouTube they can ratethem by clicking a like or dislike button and tweet about it.The pre-formatted tweet in this situation shows the follow-ing structure.I liked a @YouTube video [from @uploader]<Link to YouTube video> <Title>To restrict the Twitter API to results originating fromthese kinds of YouTube related tweets, we employ the query‘I liked a @YouTube video’. The data fields that can beextracted from the resulting tweets are:• User (Twitter user identifier)• The @handle of the video owner (optional)• YouTube URL of the videoWhile there is no star-rating involved in this scenario, thefeedback gathered is explicit feedback (i.e., the user explic-itly expressed that she liked the video). The URL containsthe unique YouTube identifier for the video which can beused to request additional content data (e.g., tags) from theYouTube API.2.5 Other platformsAlthough in this work we focus on the previously discussedplatforms, other websites could be considered for rating ex-traction as well. In table 1 we also list some other onlineplatforms (and suggested Twitter API search queries) wefound to be integrating a social share feature for postingstructured rating data to Twitter.7http://youtube-global.blogspot.be/2009/09/five-stars-dominate-ratings.htmlTable 1: Platforms available for rating extractionPlatform Domain Twitter API queryIMDb movies I rated #IMDbGoodreads books of 5 stars toPandora music I am listening on #PandoraYouTube video clips I liked a @YouTube videoAmazon e-commerce I just bought via @amazonSpotify music #nowplaying on spotifyLast.fm music I’m listening to via @lastfm3. MINING EXPERIMENTTo validate our method of extracting rating datasets fromTwitter, we set up an experiment to automatically buildfour rating datasets, one for each of the online platformsdiscussed in the previous section.3.1 Experimental setupFor each of the online platforms (i.e., IMDb, Goodreads,Pandora and YouTube.) we queried the Twitter API at fixedtime intervals (between 5 and 30 minutes) to download alltweets containing the aforementioned preference indicators.The frequency of querying the Twitter API depended on thetypical number of tweets associated with the specific website.YouTube tweets were much more numerous than tweets fromIMDb, so we had to query the Twitter API more frequently(every 5 minutes) to capture all the relevant tweets, whilerespecting the Twitter API limitations.The data fields as discussed in the previous section wereextracted by means of a series of specific regular expressionsand stored line by line in dataset files. We mined ratings overa period of 2 weeks (from December 19, 2013 to January 2,2014) and processed all the captured tweets in 4 resultingdatasets. The (Python) scripts used for the downloadingand processing of the files, and the resulting datasets areavailable on the Github platform8.3.2 ResultsTable 2 lists the basic characteristics for each of the 4 col-lected datasets. While each dataset was mined on Twitterfor the exact same period of time, the number of extractedratings is significantly different. The most ratings were col-lected from the YouTube platform, the fewest from Pandora(about 2000 times less). For all of the datasets the sparsity,calculated as equation 1, turned out to be very high. Thisis to be expected since the collected datasets are unfiltered.Often public datasets only integrate users with a minimumnumber of ratings while our datasets contain every user withat least one tweet containing a rating. The high sparsity val-ues indicate high numbers of users and items with only littlerating information to link them, which can be a major prob-lem for collaborative filtering recommender systems knownas the sparsity problem [7, 8].sparsity = 1− # ratings(# users)× (# items) (1)On the other hand, filtering a dataset may cause the intro-duction of a systematic bias which may prevent the abilityto generalize any obtained experimental results to real-lifescenarios[9]. Furthermore, the natural datasets that come8https://github.com/sidooms/Twitter-ratingsTable 2: Dataset statistics (2 week mining period)IMDb Goodreads Pandora YouTube#ratings 9,297 43,960 1,468 2,867,182#users 3,412 19,680 1,039 420,373#items 2,689 27,403 425 1,112,292avg rat./day 664 3,140 105 204,799sparsity 0.99899 0.99992 0.99668 0.99999323726190031427644419580IMDb GoodreadsYouTubeFigure 2: Venn diagrams indicating the numbers ofunique users for the IMDb, Goodreads and YouTubedatasets and their intersections.from our approach allow the simulation of a real-life recom-mender system, which often has to take into account manyusers and items with very few ratings or feedback.Fig. 1 again illustrates the extreme difference in numbersof ratings collected from each platform during our miningperiod. The figure shows the daily number of ratings whichvaries day by day mostly depending on the day of the week(more activity in weekends).An emerging research topic in the recommender systemsarea is cross-domain recommendation [10]. In cross-domainrecommendation scenarios the sparsity problem is usuallyalleviated by integrating data from another domain e.g., rec-ommending books based on previous movie ratings. One ofthe main challenges the domain faces is the lack of cross-domain rating datasets, which forces researchers to workwith artificially generated datasets. With our approach wefind ourselves in the unique position of linking rating dataoriginating from the same (Twitter) user across multipleitem domains (books, movies, music, etc.). We analyzed theintersections of the 4 collected datasets, more specificallythe intersection of users (i.e., find Twitter user ids that haveratings in more than one dataset).Fig. 2 shows the result of the intersection analysis forYouTube, Goodreads and IMDb (Pandora was omitted be-cause of the low number of collected ratings). Many usersturned out to actually rate across more than one domain. Intotal, 7 users were even found to rate across all three of thedatasets. Considering the short period of data collection (2weeks), these results look very promising for the creation ofcross-domain rating datasets.4. CONCLUSIONSIn this work we generalized our method of mining ratingdatasets from structured data on Twitter. We focused on4 different online platforms each in another item domain:01000200030004000Day (2 week period)Number of collected ratingsllllllllllllll1 2 3 4 5 6 7 8 9 10 11 12 13 14l GoodreadsIMDbPandoral l l l l l l l l l l l l l050000100000150000200000250000Day (2 week period)Number of collected ratings1 2 3 4 5 6 7 8 9 10 11 12 13 14YouTubeFigure 1: The daily number of collected ratings from Goodreads, IMDb, Pandora and YouTube. YouTube isdisplayed separately because of the large difference in Y-axis scale.IMDb (movies), Goodreads (books), Pandora (music) andYouTube (video clips). We illustrated the application ofour approach for each of these platforms and validated ourmethod by mining ratings into 4 datasets over a period of 2weeks. In total, almost 3 million ratings were collected, withsome users even appearing in multiple datasets, paving theway towards cross-domain datasets. We hope this work endsthe lack of rating data in the recommender systems areaby assisting researchers in tapping into the vast amountsof information that can be easily extracted through socialmedia.5. ACKNOWLEDGMENTSThe described research activities were funded by a PhDgrant to Simon Dooms of the Agency for Innovation by Sci-ence and Technology (IWT Vlaanderen).6. REFERENCES[1] Jonathan L Herlocker, Joseph A Konstan,Al Borchers, and John Riedl. An algorithmicframework for performing collaborative filtering. InProc. 22nd annual int. ACM SIGIR conference onResearch and development in information retrieval,pages 230–237. ACM, 1999.[2] Andreas To¨scher, Michael Jahrer, and Robert M Bell.The bigchaos solution to the netflix grand prize.Netflix prize documentation, 2009.[3] Martin Piotte and Martin Chabbert. The pragmatictheory solution to the netflix grand prize. Netflix prizedocumentation, 2009.[4] Rui Ping Song, Bo Wang, Guo Ming Huang, Qi DongLiu, Rong Jing Hu, and Rui Sheng Zhang. A hybridrecommender algorithm based on an improvedsimilarity method. Applied Mechanics and Materials,475:978–982, 2014.[5] Yong Liu, Fan Jia, and Wei Cao. A degree-basedmethod to solve cold-start problem in network-basedrecommendation. In Advanced Technologies, Embeddedand Multimedia for Human-centric Computing, pages897–904. Springer, 2014.[6] Simon Dooms, Toon De Pessemier, and Luc Martens.Movietweetings: a movie rating dataset collected fromtwitter. In Workshop on Crowdsourcing and HumanComputation for Recommender Systems, CrowdRec atRecSys 2013, 2013.[7] Bin Li, Qiang Yang, and Xiangyang Xue. Can moviesand books collaborate? cross-domain collaborativefiltering for sparsity reduction. In IJCAI, volume 9,pages 2052–2057, 2009.[8] Weike Pan, Evan Wei Xiang, Nathan Nan Liu, andQiang Yang. Transfer learning in collaborative filteringfor sparsity reduction. In AAAI, volume 10, pages230–235, 2010.[9] Guy Shani and Asela Gunawardana. Evaluatingrecommendation systems. In Recommender systemshandbook, pages 257–297. Springer, 2011.[10] Ignacio Ferna´ndez-Tob´ıas, Iva´n Cantador, MariusKaminskas, and Francesco Ricci. Cross-domainrecommender systems: A survey of the state of theart. In Proc. 2nd Spanish conf. on InformationRetrieval. CERI, 2012.

Mining cross-domain rating datasets from structured data on Twitter

https://biblio.ugent.be/publication/5839927/file/5839948

Mining cross-domain rating datasets from structured data on Twitter

Abstract

Similar works

Full text

Available Versions

Crossref

Ghent University Academic Bibliography