de Rijke, M.

de Rooij, O.

Kenter, T.

English

xTAS is an extendable multi-user text analysis service for large scale multi-lingual document analysis developed at the University of Amsterdam. It can process large amounts of documents in a timely manner through a web interface that can be used by multiple users at once. In this demonstration paper we present recent additions which include semanticization, on the fly TF-IDF model generation and on the fly co-occurrence metrics. Furthermore, we demonstrate ThemeStreams, a novel topic monitoring tool built on top of xTAS

International Migration, Integration and Social Cohesion online publications

UvA-DARE is a service provided by the library of the University of Amsterdam (https://dare.uva.nl)UvA-DARE (Digital Academic Repository)xTAS and ThemeStreamsExtendable Text Analysis Service and its Usage in a Topic Monitoring Toolde Rooij, O.; Kenter, T.; de Rijke, M.Publication date2013Document VersionFinal published versionPublished inDIR-2013 : Dutch-Belgian Workshop on Information RetrievalLink to publicationCitation for published version (APA):de Rooij, O., Kenter, T., & de Rijke, M. (2013). xTAS and ThemeStreams: Extendable TextAnalysis Service and its Usage in a Topic Monitoring Tool. In C. Eickhoff, & A. P. de Vries(Eds.), DIR-2013 : Dutch-Belgian Workshop on Information Retrieval: Proceedings of the 13thDutch-Belgian Workshop on Information Retrieval : Delft, The Netherlands, April 26th, 2013(pp. 58-59). (CEUR Workshop Proceedings; Vol. 986). CEUR-WS. http://ceur-ws.org/Vol-986/paper_17.pdfGeneral rightsIt is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s)and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an opencontent license (like Creative Commons).Disclaimer/Complaints regulationsIf you believe that digital publication of certain material infringes any of your rights or (privacy) interests, pleaselet the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the materialinaccessible and/or remove it from the website. Please Ask the Library: https://uba.uva.nl/en/contact, or a letterto: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. Youwill be contacted as soon as possible.Download date:10 Mar 2023xTAS and ThemeStreamsExtendable Text Analysis Service and its Usage in a Topic Monitoring ToolOrk de RooijUniversity of AmsterdamScience Park 9041098 XH Amsterdam, TheNetherlandsorooij@uva.nlTom KenterUniversity of AmsterdamScience Park 9041098 XH Amsterdam, TheNetherlandstom.kenter@uva.nlMaarten de RijkeUniversity of AmsterdamScience Park 9041098 XH Amsterdam, TheNetherlandsderijke@uva.nlABSTRACTxTAS is an extendable multi-user text analysis service forlarge scale multi-lingual document analysis developed at theUniversity of Amsterdam. It can process large amountsof documents in a timely manner through a web interfacethat can be used by multiple users at once. In this demon-stration paper we present recent additions which include se-manticization, on the fly TF-IDF model generation and onthe fly co-occurrence metrics. Furthermore, we demonstrateThemeStreams, a novel topic monitoring tool built on topof xTAS.Categories and Subject DescriptorsI.2.7 [Natural Language Processing]: Text AnalysisGeneral TermsAlgorithms, Performance, ExperimentationKeywordstext analysis, web service, distributed processing, microblogvisualization1. INTRODUCTIONxTAS1 is an integrated set of text analysis services forprocessing documents in a timely manner. It is availablethrough a web API that can be used by multiple usersat once. xTAS includes tools for stemming, tokenization,named entity recognition, part–of–speech tagging, sentimentanalysis and various types of aggregation on top of this. Thepurpose of xTAS is to run text processing tasks as fast aspossible, without concerning users about databases, storageor result caching.The software can run multiple tasks in parallel, possiblyon different machines (nodes). xTAS is built solely withopen source software. It uses Celery [2] to distribute tasks1See http://xtas.netPermission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.DIR 2013, April 26, 2013, Delft, The Netherlands..between nodes. By default MongoDB [4] is used to storedocuments and results though other options are available aswell.The software is extendable. Additional functionality caneasily be added through a plugin architecture.In what follows we describe recent additions to xTAS and wepresent ThemeStreams, a novel topic monitoring tool builton top of xTAS.2. XTASRecent additions and improvements to xTAS include:• Semanticization2xTAS can semantically enrich texts by linking entitiesmentioned in it to their Wikipedia article.• On the fly TF-IDF model generation and applicationTF-IDF models based on a user selected series of doc-uments can be trained on the fly. The models can beused to provide TF-IDF statistics for words in newdocuments.• Co-occurrence metric calculationA variety of co-occurrence metric calculation methodswere added to xTAS, including maximum likelihoodestimate, point wise mutual information, log likelihoodratio and χ2. This enables users to calculate the co-occurrence of entities in a set of documents.• Automatic language identificationIf the language of a document is not supplied xTAScan automatically determine it. Currently this is im-plemented by using TextCat [6].• Support for multiple document storesBesides mongoDB [4], xTAS can communicate directlywith Apache Solr [1] or ElasticSearch [3]. These storescan be used as a document repository as well as a resultcache.• Response time improvementsAnalysis of xTAS usage over time shows that namedentity recognition is a frequently requested and timeconsuming analysis. In order to keep response times to2Semanticization, the process of linking mentions of con-cepts in a text to the articles in an external knowledge basethey denote, is also referred to as entity linking or Wikifica-tion.near-real time speeds xTAS keeps several NER models(for all supported languages) in memory on each xTASnode.3. THEMESTREAMSThemeStreams3 is a visual interface that helps answer thequestion ”Who is talking about what?”. It does so for topicsin the Dutch political landscape by showing the ebb and flowof conversations about particular themes trough time. Whilethere are many topic monitoring tools available, the noveltyof ThemeStreams lies in its ability to present the user with aquick overview of the relative frequency of posts a particulargroup of users issued on a certain subject. ThemeStreams isbased on tweets posted on Twitter by four groups of people:• politicians (ministers, members of parliament, but alsothe local ranks of politicians in municipalities and provinces)• political journalists (news paper journalists as well astalk show hosts of political television shows)• lobbyists (people pushing the people who are active inpolitics)• other influencers (these include (satirical) columnists,politically engaged celebrities and stand-up comedi-ans)The harvesting of these tweets started late 2011. At thetime of writing, we follow about 1400 individual users, who,together with all people participating in conversations withthese inner circle users yield a set of just over 3.9M tweets.The interactive visual interface is aimed at giving insightinto the ownership and dynamics of themes being discussed.It enables users to answer questions such as Who put thisissue on the map?, Who picked up on this topic?, Is thistopic gaining momentum? ThemeStreams allows users toexplore streams of tweets either from a fixed set of predefinedthemes or through a search box. It uses stream graphs [5]to indicate how the four influence groups discuss a specifiedtheme, thereby depicting the volume, the “aliveness” andownership of a topic.The interface indicates the time a tweet was posted, theinfluence group the poster belongs to and the number ofpeople which reacted to a statement (which can be used toestimate the “size” and “lifetime” of statement). Initially acombined word cloud is shown with words colorized by thegroup they originate from. Users can zoom in to parts ofthe stream for more detail. Doing so results in individualword clouds being displayed per influence groups during theselected period.Initial usability studies were carried out with universitystaff members and media analysts working for a communi-cation agent. We found that ThemeStreams was intuitive tounderstand and it was easy to inspect parts of a tweet streamin detail. The combined clouds proved to be insightful fora fast overview of data. The individual clouds proved tobe useful for inspecting relative word usage between groups.We also found a need for depicting the most representedspeakers within a group.4. FUTURE WORKxTAS is actively being used in a number of research andproduction environments. As such, work on xTAS is ongo-ing and features are being deployed in close collaboration3See an online demo at http://themestreams.xtas.net/Figure 1: ThemeStreams - A visual interface thatanswers the question Who is talking about what?.Tweets are shown in a stream graph, categorized bytheir authors and weighted by their conversationalinfluence. Parts of the stream can be selected anddetailed word clouds per group pop up to show whatwas being said by whom during that period in time.with end users. Currently, we focus on adding support fortemporal tagging and for easier deployment on large clus-ters.A more detailed user study of ThemeStreams is currentlyin progress. Also we are looking into additional applica-tion scenarios for ThemeStreams, like discourse analysis overtime in other domains such as news paper archives.5. ACKNOWLEDGEMENTSThis research was partially supported by the EuropeanUnion’s ICT Policy Support Programme as part of the Com-petitiveness and Innovation Framework Programme, CIPICT-PSP under grant agreement nr 250430, the EuropeanCommunity’s Seventh Framework Programme (FP7/2007-2013) under grant agreements nr 258191 (PROMISE Net-work of Excellence) and 288024 (LiMoSINe project), theNetherlands Organisation for Scientific Research (NWO) un-der project nrs 612.061.814, 612.061.815, 640.004.802, 727.-011.005, 612.001.116, HOR-11-10, the Center for Creation,Content and Technology (CCCT), the BILAND project fundedby the CLARIN-nl program, the Dutch national programCOMMIT, the ESF Research Network Program ELIAS, theElite Network Shifts project funded by the Royal DutchAcademy of Sciences (KNAW), and the Netherlands eScienceCenter under project number 027.012.105.6. REFERENCES[1] Apache solr. http://lucene.apache.org/solr/.[2] Celery: Distributed Task Queue.http://celeryproject.org/.[3] elasticsearch. http://www.elasticsearch.org/.[4] MongoDB. http://www.mongodb.org/.[5] L. Byron and M. Wattenberg. Stacked graphs–geometry& aesthetics. Visualization and Computer Graphics,IEEE Transactions on, 14(6):1245–1252, 2008.[6] W. B. Cavnar, J. M. Trenkle, et al. N-gram-based textcategorization. Ann Arbor MI, 48113(2):161–175, 1994.

xTAS and ThemeStreams:Extendable Text Analysis Service and its Usage in a Topic Monitoring Tool

https://pure.uva.nl/ws/files/55504060/paper_17.pdf

xTAS and ThemeStreams:Extendable Text Analysis Service and its Usage in a Topic Monitoring Tool

Abstract

Similar works

Full text

Available Versions

International Migration, Integration and Social Cohesion online publications